Math educators almost universally place high value on reasoning, sense-making, and logical thinking. The first two Common Core Standards for Mathematical Practice begin “Make sense of problems” and “Reason.” We promote math education as a means for students to learn how to analyze, understand, and improve the world around them. But what are we to conclude when students respond to a mathematical absurdity not with skepticism and reason, but with hapless and nonsensical attempts to impose classroom math upon it?
An online blog article by math educator and researcher Tracy Zager titled “Making Sense”raises this question with a fascinating example, a video filmed by Robert Kaplinsky, a district math specialist in California, who also wrote about it at his website. In the video, eighth graders were presented with a question popularized by Katherine Merseth: “There are 125 sheep and 5 dogs in a flock. How old is the shepherd?” Astonishingly, 75% of the students performed operations with the numbers in order to come up with a numerical answer. Robert added that a group of sixth graders had done even worse, universally coming up with numerical answers rather than pointing out that the problem does not make sense as written.
I’m sure I wasn’t the only person to watch this video and immediately question whether it was truly representative of American students. Maybe, I thought, Robert’s group of students was particularly intimidated, or unusually bad at math. But further reading showed his results were actually quite typical. The original researcher whom Merseth wrote about, Kurt Reusser, also found in 1986 that three fourths of schoolchildren did not make sense of the question. More recently, Robert’s comment section had comments from several teachers who tried the experiment with their students, and Tracy wrote about elementary school teacher colleagues who tried similar corresponding problems with their younger students. In many different groups, with varied teachers, a majority of students failed to make sense of the problem or object to it. Robert added in a comment on Tracy Zager’s post that when he reran the problem with a whole class, even after thorough group discussion of all the individual answers, many students still chose the nonsense answer. Tracy admitted that she had thought younger children would do better: “I had assumed kids went with nonsense by middle school because of what we’ve taught them about math. Now that I have preschoolers and 1st graders going with nonsense too, I have to revisit my assumptions.”
What is going on? “America has produced a generation of students who engage in problem solving without regard for common sense or the context of the problem,” concluded Merseth in 1993, before blaming societal attitudes about math, limited curricula in which “stress is on computation and procedure, not understanding and sense making,” and poor teacher preparation. Jo Boaler, author and professor of mathematics education at Stanford University, has also criticized our culture’s emphasis on computation and procedure. In her video “Why We Need Common Core Math,” she states, “If you ask most students what they think their role is in maths classrooms, they’ll tell you that it’s to answer questions correctly. They don’t even think that they’re in maths classrooms to learn or to explore the rich set of connections that make up mathematics. They think they’re there to get questions right.”
On Robert’s site and in Tracy’s blog’s comment section, teachers struggled to make sense of why students weren’t making sense. Here were some of their thoughts:
- Some believed the interaction between the teacher and student was key: “The student’s main objective is not to make sense of the question, or even to answer it correctly, but to give an answer that will satisfy the teacher,” said a commenter named Dave on Robert’s site. “Students believe that an honest answer (‘I don’t understand the question’) would not satisfy the teacher, so they make random guesses instead.” Commenter Pierre added, “Students are conditioned to assume that the teacher is teaching something sensible.” “It does appear that some students are overriding their common sense… and my hope for this video is that it will help us all reflect on why that is,” Robert Kaplinsky responded to Dave.
- Commenter MG on Robert’s post blamed overtesting, in part: “When we base schooling around testing, students learn to look like they’re smart in order to fool the test […] without understanding.”
- Annie Fetter of the Math Forum wondered in Tracy’s comment section if drawing a picture would help kids “step back and do the sense-making that they won’t do if they think it’s math.”
- Commenter Joshua (of 3jlearneng.blogspot.com ) on Tracy’s post theorized that “students seem to be triggered to think they are answering a ‘math-class’ question (MCQ). MCQs have […] exactly one single correct answer, […] can be answered using the tools you have been taught, […] require a mathematical object as an answer […], are asked of students, not by students, [and] they require entering into an idealized world where you only have the information given to you within the question, no more and no less.” Joshua’s ideas seemed to correspond with some shared by Dan Meyer in his well-known TED talk “Math class needs a makeover”: “[W]hat problem have you solved, ever, that was worth solving where you knew all of the given information in advance; where you didn't have a surplus of information and you had to filter it out, or you didn't have sufficient information and had to go find some.”
I Wonder What Would Happen In My Classroom?
If you’re like me, as you read about this problem and how students reacted to it, you started itching to go ask your own students how old the shepherd is! After all this online discussion, I decided to try to investigate my students’ sense-making with a nonsense question. These were my goals:
- Compare the performance of my sixth graders at an arts focus middle school (da Vinci, Portland Public Schools) with that of other groups of students. Although math is not generally the main interest or passion of my students, they often seem to me to be more willing to be original thinkers and to question authority than many sixth graders, so I thought perhaps they would be less likely to comply with a nonsense answer against their intuition.
- Give a nonsense prompt that seemed very likely to generate multiple attempted numerical answers, in order to help ensure that no individual felt exposed or stupid when the answer was revealed in a whole-class discussion. I decided to change the question to one that I felt was somewhat less obviously nonsensical, to be cautious on this point. I chose a fraction problem because we had recently finished a fraction unit.
- Generate a prompt which was close to one that could be solved, to see if students would make suggestions of what missing piece of information would help.
- Examine the role of drawing pictures, as suggested by Annie Fetter.
- Debrief in phases: sharing anonymous individual answers, discussing in pairs, then having a whole-class discussion. Ultimately, I wanted students to leave class understanding the actual answer (not enough information), knowing what motivated me to ask the question, and becoming aware that giving a numerical answer was not at all unusual. I also wanted to elicit their thoughts on why it is so common to try to answer a messed-up question with a specific answer, and how teachers can build a space where students feel safe and confident critiquing the question.
With these goals, I prepared this Google slide for one class (Group A) and another just like it except without the cue to draw a picture for two other classes (Groups B and C):
and provided it to my students to “solve” on individual whiteboards, after giving them two key verbal instructions: that it was essential they work silently and individually, and that if they had questions for me, they should write them down as part of their work, but that I would not answer questions while they worked individually. Each class (Groups A, B, and C) had 27, 23, and 21 students present, respectively, sitting at tables of three or four students.
A followup slide for a “pair share,” which happened after I collected the individual whiteboards, included the question (“It takes…”) preceded by the prompt: “Now discuss the problem with the person next to you (or the people across from you if you’re at a table of 3).”
Following the pair share, we had a group discussion, during which students shared out their table’s thinking, then I showed them the Robert Kaplinsky video and summarized the research results. After our class discussion, I made sure to obtain the first two groups’ agreement not to spoil the surprise by discussing the problem with other classes (and I believe they did not).
What Actually Did Happen In My Classroom
Initial Student Reaction
Our school has a large emphasis on performance arts, and my students tend to be extroverts, so I expected that watching their faces as they processed the question would be entertaining, and I was not disappointed. In Group A, many students looked worried, confused, and doubtful, and a few looked indignant, within a minute or so, before they picked up the pens to draw or write. Several insisted on calling me over to whisper to me (I responded as described above). To my surprise, the expressions on the students’ faces were very different for Group B as they absorbed the question: they looked much less worried and more confident, and many looked amused. Fewer called me over to question the problem, and most were quicker to write down their answers. Group C’s students’ confidence seemed between Group A’s and Group B’s, and the few students who called me over seemed, by their body language, more ready to tell me off.
Effect of the Pictures
For Group A, the students who were prompted to draw a picture, students were generally willing to do this, but I saw little evidence the pictures helped them realize the problem was nonsense: in fact, if anything, my impression was that my insistence on a picture made some think it must be a problem I thought they could solve.
Overall Results for Individual Answers
Unfortunately, these were not perfect research conditions: I don’t have a background as an educational researcher, and with back-to-back classes it was difficult to keep perfect records. Nevertheless, the results I was able to record were quite interesting and suggestive to me.
First of all, a significantly higher proportion of da Vinci students identified the “Cocoa at Joe’s” question as a nonsense question than students in previous formal and informal studies tended to do for the “How old is the shepherd” question. In Group A, more than half clearly stated in some form or another that it was a nonsense question, and why. (My notes seem to say it was 15 out of 27, they but are not completely clear.) In Group B, a few students looked worried about it, but virtually everyone ended up answering that the problem did not make sense or there was not enough information. In Group C, 12 out of 21 students present clearly indicated that the problem could not be answered and/or was nonsense. Here are some examples of ways students expressed this thought on their whiteboards:
(More photos of answers of this type are available here in the "Not Enough Info" folder.)
A few other students in Groups A and B, as well as three in Group C, were clearly uneasy with the question and refused to commit to a real answer, writing “IDK” (I don’t know) or similar statements.
There seemed to be no clear correlation between sense-making and general math achievement; if there was any trend, the students who seemed most confident it was a nonsense question tended to be from both the high and low ends of the class, grade-wise, more than from the middle.
Details of Individual Answers
As I expected, many of the students who gave a numerical answer answered 1 ¼ (the sum of 1 and ¼, the two numbers given) or 4 (the number of ¼’s in 1; the quotient, although they probably didn’t think of it that way). I was surprised by how many people answered 1, including one girl who said she thought it was a riddle: perhaps in their attempts to make sense of the question, they decided only one serving was described, so it must be one person. Here are some pictures of work from people who tried to explain their numerical answer:
(More photos of answers of this type are available here in the "Numerical or No Answers" folder.)
I suspect several of the people who avoided writing down a specific answer or line of reasoning (despite my encouragement) didn’t think the problem made sense, but were too confused and underconfident to write that down.
Results for Answers After Table Discussions
In all three groups, during table discussions, every single student became convinced it was a nonsense question. Group B seemed to spend the most time talking about why they thought I asked a nonsense question. In Group C, I was interested to hear several students loudly and confidently referring to algebra (this idea may have spread from one table to another as they overheard it, or maybe several people came up with it). “It’s algebra!”, “n,” and “random number” were some sentence fragments I wrote down.
After discussion at their tables, many students volunteered to summarize their tables’ thoughts. The people who spoke for the tables were mostly medium- to high-achievers in math class, but many others seemed intent on listening to their answers and clearly agreed. A few typically high-achieving speakers had begun with a numerical answer and seemed eager to share why they were now confident it was a nonsense question. Here was what the groups said when they reported back (comments recorded in chronological order):
Group A:
Our table agreed that if you knew how much they used in one day, then you could answer, but [that was missing].
You were trying to make us look at the question more than the other information.
It seems like they took out a sentence. It seems like algebra – looking for missing things, and how they fit together. Some of the things we need to answer the question aren’t there.
I changed my mind from the discussion. It says “how many people ordered,” and you only have information about one cup.
It doesn’t give you that little teaser! Any number could be right. There’s not enough information.
The question doesn’t make sense!
Group B:
You can’t do the problem. There’s no information about how much milk or cocoa. You wanted us to think of how equations can be wrong or [how there can be] not enough information.
You’d need to know how much was used. At first I thought it was a riddle.
I thought it was n. You have the information but not all of it, so it’s as many numbers as it is. [Another student from that table interjected: It could also be n x ¼ and n x 1 and add together.] It reminds me of [Dan Meyer’s] TED talk – you give them partial information and make them ask for more. [My mind was blown by this, since I wasn’t thinking of the TED talk at all, and wasn’t even certain which periods I’d shown it to. I asked, “Who thought of the TED talk while you were working on this?” and four people around the room raised their hands.]
There’s not enough information. At the end it doesn’t say who ordered.
We all agreed there was not enough information. [We thought] “Maybe she wanted to test us.”
We could not do it! Somebody got some numbers, but I said, how could you do it??
Group C:
I wrote one person, but I realized there’s no way to answer, so now I say x.
We all could have realized it wasn’t multiplying fractions or anything. [There’s] no answer, so we could call it x.
At first we looked for an answer but [… it] could be any number.
I think it’s also a letter because we’ve been studying algebra.
I asked Group C, “Why do you think I gave you this problem?” Four new students answered (this time across the achievement spectrum):
To show sometimes problems have algebra letters.
In algebra, there’s not always a specific number answer.
Maybe the problem was written wrong. [Interestingly, though, this answer was from the only student in this class who reported that an elementary school teacher had given his class a similar (nonsense) problem.]
Maybe whoever wrote this wanted us to think about algebra.
I believe I asked Group B a similar question, although unfortunately I didn’t write down my question(s), just their answers. One student stated firmly that people “lack common sense.” Another said that people “talking among themselves helps make sense.”
How They Felt
I asked students how they felt about this question when they were working on it on their own, and being middle schoolers, they were eager to share.
In Group A, a student reported feeling “annoyed,” and about three-fourths of the students agreed. Another student reported thinking it was funny; three or four people agreed. One added, “I thought it was one of the funniest things a math teacher’s ever done.”
In Group B, when asked to describe how they felt while working on this problem, one student replied, “That was amazing!” Students volunteered words to describe other ways they felt while they were working on the problem by themselves, and we got a count for each adjective of how many people felt that way. Out of 23 students, five felt “frustrated/GRRR,” nine felt confused, four felt content (which puzzles me!), six said “funny,” two felt surprised, and eight felt stupid (“because I couldn’t get the answer”).
In Group C, I solicited a list of words from students to describe ways they felt while they were working on the problem by themselves, and we got a count for each adjective of how many people felt that way (out of 21). Eleven people reported feeling stupid, seven felt confused, five felt indignant or annoyed, three felt determined, and two felt amused.
Take-Aways and Conclusion
I probably question and second-guess every instructional decision I make as a math teacher. Do problems that are challenging or that do not have a particular right answer help students develop as critical thinkers, or are they just discouraging “trick questions”? Does group work help students communicate ideas and come to a deeper understanding of math, or is it just pointless off-topic chatter or copying of work without comprehension? Do my students need more time for practice of procedures and less for conceptual understanding? Should they respect my authority more, or continue to be allowed to question me and the tasks they are given?
I’m going to continue asking those questions of myself, and I am sure the answers will fluctuate depending on students’ needs. Nevertheless, this activity and the results from it, especially compared to the results obtained in other settings with similar problems, gave me a renewed appreciation for the role of non-“math class questions,” group work, conceptual understanding, and student questioning in developing mathematical reasoning and sense-making. I work hard at promoting critical thinking and conceptual understanding, and believe elementary school teachers in my district do too. I think this emphasis contributed to how well the da Vinci students did at identifying the nonsense question. Students also clearly progressed in their thinking in the group discussion, partly because they were comfortable with this format, and they developed sophisticated ideas together of what the activity was about. Finally, I allow and sometimes even encourage students to criticize textbook problems that don’t seem to match real life (though often we solve them anyway). This used to feel indulgent, but now I feel it is important for my classroom culture and their critical thinking.
I also realized that our students need us to convey that mathematical reasoning and sense-making is not about getting procedural answers as fast as possible. When students use math outside of the classroom, they must make decisions about what types of problems they can solve and what information they will need. We do them a disservice if we only give them “plug-and-chug” problems. For Groups B and C, I recorded that 19 out of 44 students (43%) felt stupid when they could not solve the nonsense problem, including many who had correctly answered that there was not enough information. I hope they finished class realizing that the same critical thinking that made them so uneasy that they were “doing it wrong” was actually a strength.
As Dan Meyer says, “Math makes sense of the world. Math is the vocabulary for your own intuition.” Our students all deserve to have that experience.
