The Genesis of Our Algebra 1 Course, part 2: Remote Field Testing in the US

If we want to create something our users truly enjoy, then why not ask what they want directly? As a grantee of Phase 1 Balance the Equation Grand Challenge, Mastory headed straight to the source and interviewed students and teachers from high schools in Illinois, primarily in the Chicago area. Interviews were utilized to adapt our characters and stories to something our users would be interested in. The conversations proved to be exciting and enriching, with fascinating insights from the learners themselves, so keep on reading to find out more.

Our interviewees

We conducted interviews with Title I school students and teachers, who face a plethora of challenges in their daily lives. With the help of Fieldwork Chicago, student, and teacher interviews were set up on video calls. A total of 5 teachers and 11 students were interviewed.

During teacher interviews, we wanted to find out as much as possible about their everyday experiences. What are the biggest challenges they face as title 1 school Algebra 1 teachers? What do they think of Algebra 1? What do they think their students think of Algebra 1? What is important for them? And a lot more.

We had two goals in mind while conducting our student field tests: on the one hand, we wanted to gain general and unbiased insights about our students. We wanted to find out what topics they were naturally interested in. What are their likes and dislikes? What are their problems? What makes them laugh? What encourages them? What bores them? And of course - how do they think and feel about algebra? To find out, we conducted 5 interviews. On the other hand, we wanted to test how they react to Mastory. We wanted to know if they like the characters we have created for our Algebra 1 story if they find our story elements engaging, and most importantly if we are able to motivate them with our story to get active. With this purpose in mind, we conducted 6 student concept tests. Here, we demonstrate the results of our concept tests.

The Concept Tests

In order to get the students engrossed in the story, we needed opinions about the proposed characters and storylines. After getting comfortable with the interviewer and listening to an audio clip introducing the main characters, students were asked their opinions on character personalities and physical attributes. In general, they were on a search for ‘realness’. Not only in their personality and goals but also in the way they look. It was fascinating that when asked to choose out of 11 pictures who they thought the character would look like, 3 out of 5 students picked the same, most real person. The other 10 pictures were all images of models and actors! This discovery highlights that authentic ‘real’ characters are key for relatability. It is vital to create characters that are viewed as genuine members of their community and that can serve as role models. So it was important to hear that we managed to create characters that sounded real: “I like how that character was made, and it felt like you were having a conversation with a real person.”

Students showed a keen interest in genuine personalities. When asked their opinions on certain characters, they disliked the notion that some of them acted too “perfect”: “Can you not know how to do something so somebody can help out?!” They were not looking for another Superman or Sherlock Holmes, but rather someone that can feel like their friend. When creating the characters, we rewrote their personality to make them less “perfect”. We realized that they can give guidance and still struggle a bit more. Similar to their appreciation for someone who serves more as a friend than a superhero, they described qualities of people they like that were adapted to the characters of the story and their dialogues. These qualities included being reliant, passionate, and especially, someone that stays by your side in life.

It was incredibly inspiring to see how eager the students were to immerse themselves in the world we created - even though it was conveyed to them through simple pictures and audio files. They were taking their time to give long answers and think a lot about how they imagined a situation, how they feel about a story element, or which character they would additionally suggest for our story.

Students were also given a link with a mock version of the upcoming app, so they could actively engage in the story. All six students were ready to help one of the characters by solving a problem. They not only said so but also really engaged while entering possible passwords that satisfied certain rules to determine the meaning of an encrypted message. They not only tried once but repeatedly and over several minutes.

At the end of the test, they were told that this is a math learning app that we are developing. They were amazed and said they would definitely try it out. Here are the reactions that we captured:

“You are doing work, but it doesn’t feel like you’re doing work. It feels like you're playing the game and you’re just like solving the challenges.”

“I think it will make math more interesting and fun at the same time if there is a story behind it.”

“I’d probably be way more interested in doing it if this is my math homework.”

“It sounds more interesting than the homework we usually get.”

“If I could do that as part of my math homework? Yeah, I would do that. I would definitely do that.”

“I think that people my age would like it. Instead of doing problem and problem after problem, we’d have something to enjoy.”

Turning research into a success!

Conducting the interviews served as great knowledge, teaching us that students have an even more vivid imagination than we expected and that they enjoy getting involved in storyline learning. These insights strengthened Mastory’s plan to create scenarios where the student will have to choose which character to help and which mission to go on.

It was important to test out the characters since they carry the story. Additionally, a lot was gained from asking the students what they remembered after hearing about a character’s background. Knowing what catches the students’ attention will help us develop distinct character traits in future storylines.

The feedback we received was incredibly positive and inspiring. Most of the students using the app are children of working-class people who work jobs such as bakers, nurses, police officers, etc. These jobs require them to be away from home very often, leading our students to feel alone in their academic journey. This may also make school seem daunting and worthless since they don’t know many people who have achieved their dreams. With our characters, we will be creating people who not only accompany them in their academic journey but also inspire them to achieve their dreams and use all their inner potential.

Follow us

To learn more about the concept and practicalities of Mastory, please stay tuned for our next blog entries and read the ones we have already posted. Be a part of our journey!

This blog post is part of a series where we introduce the various components of our upcoming Algebra 1 course. If you have missed out on the first part of this series, make sure to have a look!