A shorter version of this article originally appeared in The Oregon Mathematics Teacher (TOMT) in May/June 2016, and is reprinted here by kind permission of TOMT and the Oregon Council of Teachers of Mathematics. The writing of this article was supported by the Writers’ Retreat facilitated by the editors of TOMT and funded by the Oregon Council of Teachers of Mathematics.
Math
educators almost universally place high value on reasoning, sense-making, and
logical thinking. The first two
Common Core Standards for Mathematical Practice begin “Make sense of problems” and “Reason.” We promote math education as a
means for students to learn how to analyze, understand, and improve the world
around them. But what are we to conclude when students respond to a
mathematical absurdity not with skepticism and reason, but with hapless and
nonsensical attempts to impose classroom math upon it?
An
online blog article by math educator and researcher
Tracy Zager titled “Making Sense”raises this question with a fascinating example, a video filmed by
Robert Kaplinsky, a district math specialist in California, who also
wrote about it at his website. In the video, eighth graders were presented with
a question popularized by Katherine Merseth: “There are 125 sheep and 5 dogs in a flock. How old is the shepherd?” Astonishingly,
75% of the students performed operations with the numbers in order to come up
with a numerical answer. Robert added that a group of sixth graders had done
even worse, universally coming up with numerical answers rather than pointing
out that the problem does not make sense as written.
I’m
sure I wasn’t the only person to watch this video and immediately question
whether it was truly representative of American students. Maybe, I thought,
Robert’s group of students was particularly intimidated, or unusually bad at
math. But further reading showed his results were actually quite typical. The
original researcher whom Merseth wrote about, Kurt Reusser, also found in 1986
that three fourths of schoolchildren did not make sense of the question. More
recently, Robert’s comment section had comments from several teachers who tried
the experiment with their students, and Tracy wrote about elementary school
teacher colleagues who tried similar corresponding problems with their younger students.
In many different groups, with varied teachers, a majority of students failed
to make sense of the problem or object to it.
Robert added in a comment on Tracy Zager’s post that when he reran the problem with a whole class, even after
thorough group discussion of all the individual answers, many students still
chose the nonsense answer.
Tracy admitted that she had thought younger children would do better: “I had assumed kids went with nonsense by middle school
because of what we’ve taught them about math. Now that I have preschoolers and
1st graders going with nonsense too, I have to revisit my assumptions.”
What is going on? “America has produced a generation of students who engage in
problem solving without regard for common sense or the context of the problem,”
concluded Merseth in 1993, before blaming societal attitudes about math,
limited curricula in which “stress is on computation and procedure, not
understanding and sense making,” and poor teacher preparation.
Jo Boaler, author
and professor of mathematics education at Stanford University, has also
criticized our culture’s emphasis on computation and procedure. In
her video “Why We Need Common Core Math,” she states, “If you ask most students what they think their role is in maths
classrooms, they’ll tell you that it’s to answer questions correctly. They
don’t even think that they’re in maths classrooms to learn or to explore the
rich set of connections that make up mathematics. They think they’re there to
get questions right.”
On Robert’s site and in Tracy’s blog’s comment section, teachers struggled to make
sense of why students weren’t making sense. Here were some of their thoughts:
- Some believed the interaction between the
teacher and student was key: “The student’s main objective is not to make sense
of the question, or even to answer it correctly, but to give an answer that
will satisfy the teacher,” said a commenter named Dave on Robert’s site. “Students
believe that an honest answer (‘I don’t understand the question’) would not
satisfy the teacher, so they make random guesses instead.” Commenter Pierre
added, “Students are conditioned to assume that the teacher is teaching
something sensible.” “It does appear that some students are overriding their
common sense… and my hope for this video is that it will help us all reflect on
why that is,” Robert Kaplinsky responded to Dave.
- Commenter MG on Robert’s post blamed
overtesting, in part: “When we base schooling around testing, students learn to
look like they’re smart in order to fool the test […] without understanding.”
- Annie Fetter of the Math Forum wondered in Tracy’s comment section if drawing a picture would help kids “step
back and do the sense-making that they won’t do if they think it’s math.”
- Commenter Joshua (of 3jlearneng.blogspot.com )
on Tracy’s post theorized that “students seem to be triggered to think they are
answering a ‘math-class’ question (MCQ). MCQs have […] exactly one single
correct answer, […] can be answered using the tools you have been taught, […] require
a mathematical object as an answer […], are asked of students, not by students,
[and] they require entering into an idealized world where you only have the
information given to you within the question, no more and no less.” Joshua’s
ideas seemed to correspond with some shared by Dan Meyer in his well-known TED talk “Math class needs a makeover”: “[W]hat problem have you solved, ever, that was worth
solving where you knew all of the given information in advance; where
you didn't have a surplus of information and you had to filter it out, or
you didn't have sufficient information and had to go find some.”
I Wonder What Would Happen In My Classroom?
If
you’re like me, as you read about this problem and how students reacted to it,
you started itching to go ask your own students how old the shepherd is! After
all this online discussion, I decided to try to investigate my students’
sense-making with a nonsense question. These were my goals:
- Compare the performance of my sixth graders at
an arts focus middle school (da Vinci, Portland Public Schools) with that of
other groups of students. Although math is not generally the main interest or
passion of my students, they often seem to me to be more willing to be original
thinkers and to question authority than many sixth graders, so I thought
perhaps they would be less likely to comply with a nonsense answer against
their intuition.
- Give a nonsense prompt that seemed very likely
to generate multiple attempted numerical answers, in order to help ensure that
no individual felt exposed or stupid when the answer was revealed in a
whole-class discussion. I decided to change the question to one that I felt was
somewhat less obviously nonsensical, to be cautious on this point. I chose a
fraction problem because we had recently finished a fraction unit.
- Generate a prompt which was close to one that
could be solved, to see if students would make suggestions of what missing
piece of information would help.
- Examine the role of drawing pictures, as
suggested by Annie Fetter.
- Debrief in phases: sharing anonymous individual
answers, discussing in pairs, then having a whole-class discussion. Ultimately,
I wanted students to leave class understanding the actual answer (not enough
information), knowing what motivated me to ask the question, and becoming aware
that giving a numerical answer was not at all unusual. I also wanted to elicit
their thoughts on why it is so common to try to answer a messed-up question
with a specific answer, and how teachers can build a space where students feel
safe and confident critiquing the question.
With
these goals, I prepared this Google slide for one class (Group A) and another
just like it except without the cue to draw a picture for two other classes
(Groups B and C):
and provided it to my students to “solve” on individual
whiteboards, after giving them two key verbal instructions: that it was
essential they work silently and individually, and that if they had questions
for me, they should write them down as part of their work, but that I would not
answer questions while they worked individually. Each class (Groups A, B, and
C) had 27, 23, and 21 students present, respectively, sitting at tables of
three or four students.
A followup slide for a “pair share,” which happened after I collected the
individual whiteboards, included the question (“It takes…”) preceded by the
prompt: “Now discuss the problem with the person next to you (or the people
across from you if you’re at a table of 3).”
Following the pair share, we had a group discussion, during which students shared out
their table’s thinking, then I showed them the Robert Kaplinsky video and
summarized the research results. After our class discussion, I made sure to
obtain the first two groups’ agreement not to spoil the surprise by discussing
the problem with other classes (and I believe they did not).
What Actually Did Happen In My Classroom
Initial Student Reaction
Our school has a large emphasis on performance arts, and my students tend to be
extroverts, so I expected that watching their faces as they processed the
question would be entertaining, and I was not disappointed. In Group A, many students
looked worried, confused, and doubtful, and a few looked indignant, within a
minute or so, before they picked up the pens to draw or write. Several insisted
on calling me over to whisper to me (I responded as described above). To my
surprise, the expressions on the students’ faces were very different for Group
B as they absorbed the question: they looked much less worried and more
confident, and many looked amused. Fewer called me over to question the
problem, and most were quicker to write down their answers. Group C’s students’
confidence seemed between Group A’s and Group B’s, and the few students who
called me over seemed, by their body language, more ready to tell me off.
Effect of the Pictures
For Group A, the students who were prompted to draw a picture, students were
generally willing to do this, but I saw little evidence the pictures helped
them realize the problem was nonsense: in fact, if anything, my impression was
that my insistence on a picture made some think it must be a problem I thought
they could solve.
Overall Results for Individual Answers
Unfortunately, these were not perfect research conditions: I don’t have a background as an
educational researcher, and with back-to-back classes it was difficult to keep
perfect records. Nevertheless, the results I was able to record were quite
interesting and suggestive to me.
First of all, a significantly higher proportion of da Vinci students identified the
“Cocoa at Joe’s” question as a nonsense question than students in previous
formal and informal studies tended to do for the “How old is the shepherd”
question. In Group A, more than half clearly stated in some form or another
that it was a nonsense question, and why. (My notes seem to say it was 15 out
of 27, they but are not completely clear.) In Group B, a few students looked
worried about it, but virtually everyone ended up answering that the problem
did not make sense or there was not enough information. In Group C, 12 out of
21 students present clearly indicated that the problem could not be answered
and/or was nonsense. Here are some examples of ways students expressed this
thought on their whiteboards:
(More photos of answers of this type are available here in the "Not Enough Info" folder.)
A few other students in Groups A and B, as well as three in Group C, were clearly
uneasy with the question and refused to commit to a real answer, writing “IDK”
(I don’t know) or similar statements.
There seemed to be no clear correlation between sense-making and general math
achievement; if there was any trend, the students who seemed most confident it
was a nonsense question tended to be from both the high and low ends of the
class, grade-wise, more than from the middle.
Details of Individual Answers
As I expected, many of the students who gave a numerical answer answered 1 ¼ (the
sum of 1 and ¼, the two numbers given) or 4 (the number of ¼’s in 1; the
quotient, although they probably didn’t think of it that way). I was surprised
by how many people answered 1, including one girl who said she thought it was a
riddle: perhaps in their attempts to make sense of the question, they decided
only one serving was described, so it must be one person. Here are some
pictures of work from people who tried to explain their numerical answer:
(More photos of answers of this type are available here in the "Numerical or No Answers" folder.)
I suspect several of the people who avoided writing down a specific answer or
line of reasoning (despite my encouragement) didn’t think the problem made
sense, but were too confused and underconfident to write that down.
Results for Answers After Table Discussions
In all three groups, during table discussions, every single student became
convinced it was a nonsense question. Group B seemed to spend the most time
talking about why they thought I asked a nonsense question. In Group C, I was
interested to hear several students loudly and confidently referring to algebra
(this idea may have spread from one table to another as they overheard it, or
maybe several people came up with it). “It’s algebra!”, “n,” and “random
number” were some sentence fragments I wrote down.
After discussion at their tables, many students volunteered to summarize their
tables’ thoughts. The people who spoke for the tables were mostly medium- to
high-achievers in math class, but many others seemed intent on listening to
their answers and clearly agreed. A few typically high-achieving speakers had
begun with a numerical answer and seemed eager to share why they were now
confident it was a nonsense question. Here was what the groups said when they
reported back (comments recorded in chronological order):
Group A:
Our table agreed that if you knew how much they used in one
day, then you could answer, but [that was missing].
You were trying to make us look at the question more than
the other information.
It seems like they took out a sentence. It seems like
algebra – looking for missing things, and how they fit together. Some of the
things we need to answer the question aren’t there.
I changed my mind from the discussion. It says “how many
people ordered,” and you only have information about one cup.
It doesn’t give you that little teaser! Any number could be
right. There’s not enough information.
The question doesn’t make sense!
Group B:
You can’t do the problem. There’s no information about how
much milk or cocoa. You wanted us to think of how equations can be wrong or
[how there can be] not enough information.
You’d need to know how much was used. At first I thought it
was a riddle.
I thought it was n. You have the information but not all of
it, so it’s as many numbers as it is. [Another student from that table
interjected: It could also be n x ¼ and n x 1 and add together.] It reminds me
of
[Dan Meyer’s] TED talk – you give them partial information and make them ask
for more. [My mind was blown by this, since I wasn’t thinking of the TED talk
at all, and wasn’t even certain which periods I’d shown it to. I asked, “Who
thought of the TED talk while you were working on this?” and four people around
the room raised their hands.]
There’s not enough information. At the end it doesn’t say
who ordered.
We all agreed there was not enough information. [We thought]
“Maybe she wanted to test us.”
We could not do it! Somebody got some numbers, but I said,
how could you do it??
Group C:
I wrote one person, but I realized there’s no way to answer,
so now I say x.
We all could have realized it wasn’t multiplying fractions
or anything. [There’s] no answer, so we could call it x.
At first we looked for an answer but [… it] could be any
number.
I think it’s also a letter because we’ve been studying algebra.
I asked Group C, “Why do you think I gave you this problem?” Four new students
answered (this time across the achievement spectrum):
To show sometimes problems have algebra letters.
In algebra, there’s not always a specific number answer.
Maybe the problem was written wrong. [Interestingly, though,
this answer was from the only student in this class who reported that an
elementary school teacher had given his class a similar (nonsense) problem.]
Maybe whoever wrote this wanted us to think about algebra.
I believe I asked Group B a similar question, although unfortunately I didn’t
write down my question(s), just their answers. One student stated firmly that
people “lack common sense.” Another said that people “talking among themselves
helps make sense.”
How They Felt
I asked students how they felt about this question when they were working on it
on their own, and being middle schoolers, they were eager to share.
In Group A, a student reported feeling “annoyed,” and about three-fourths of the
students agreed. Another student reported thinking it was funny; three or four
people agreed. One added, “I thought it was one of the funniest things a math
teacher’s ever done.”
In Group B, when asked to describe how they felt while working on this problem,
one student replied, “That was amazing!” Students volunteered words to describe
other ways they felt while they were working on the problem by themselves, and
we got a count for each adjective of how many people felt that way. Out of 23
students, five felt “frustrated/GRRR,” nine felt confused, four felt content
(which puzzles me!), six said “funny,” two felt surprised, and eight felt
stupid (“because I couldn’t get the answer”).
In Group C, I solicited a list of words from students to describe ways they felt
while they were working on the problem by themselves, and we got a count for
each adjective of how many people felt that way (out of 21). Eleven people
reported feeling stupid, seven felt confused, five felt indignant or annoyed,
three felt determined, and two felt amused.
Take-Aways and Conclusion
I probably question and second-guess every instructional decision I make as a math teacher. Do problems that are
challenging or that do not have a particular right answer help students develop
as critical thinkers, or are they just discouraging “trick questions”? Does
group work help students communicate ideas and come to a deeper understanding
of math, or is it just pointless off-topic chatter or copying of work without comprehension?
Do my students need more time for practice of procedures and less for
conceptual understanding? Should they respect my authority more, or continue to
be allowed to question me and the tasks they are given?
I’m going to continue asking those questions of myself, and I am sure the answers
will fluctuate depending on students’ needs. Nevertheless, this activity and
the results from it, especially compared to the results obtained in other
settings with similar problems, gave me a renewed appreciation for the role of
non-“math class questions,” group work, conceptual understanding, and student
questioning in developing mathematical reasoning and sense-making. I work hard
at promoting critical thinking and conceptual understanding, and believe elementary
school teachers in my district do too. I think this emphasis contributed to how
well the da Vinci students did at identifying the nonsense question. Students
also clearly progressed in their thinking in the group discussion, partly
because they were comfortable with this format, and they developed
sophisticated ideas together of what the activity was about. Finally, I allow
and sometimes even encourage students to criticize textbook problems that don’t
seem to match real life (though often we solve them anyway). This used to feel
indulgent, but now I feel it is important for my classroom culture and their
critical thinking.
I also realized that our students need us to convey that mathematical reasoning
and sense-making is
not about
getting procedural answers as fast as possible. When students use math outside
of the classroom, they must make decisions about what types of problems they
can solve and what information they will need. We do them a disservice if we
only give them “plug-and-chug” problems. For Groups B and C, I recorded that 19
out of 44 students (43%) felt stupid when they could not solve the nonsense
problem, including many who had correctly answered that there was not enough
information. I hope they finished class realizing that the same critical
thinking that made them so uneasy that they were “doing it wrong” was actually
a strength.
As
Dan Meyer says, “Math makes sense of the world. Math is the
vocabulary for your own intuition.” Our students all deserve to have that
experience.
