Showing posts with label Reading comprehension. Show all posts
Showing posts with label Reading comprehension. Show all posts

Wednesday, July 15, 2015

The Spirit is Willingham, but the Flesch is Weak

Teacher’s Question:
I have read a few articles and books by Daniel Willingham in the past, and I wonder if you are familiar with his work. I recently read an article (attached) about reading comprehension strategies and am curious to know what you think of his ideas. He says that focusing heavily on reading strategies isn’t really necessary.

(I often question the need for so many reading strategies, particularly when they take away from reading being a pleasurable activity. I can understand the importance of visualizing, using prior knowledge, and maintaining focus, but teaching the other “strategies”, in my opinion, is confusing the issue. I realize there are many studies to say otherwise, but, I just can’t be convinced.)

Anyway, again, just wondering what you think of Willingham’s paper.

Shanahan's Response 
Thanks. This is the second time in two weeks I’ve been asked about Daniel Willingham’s writing on comprehension strategies. I don’t know Dr. Willingham, but I’ve read his vita.

Daniel Willingham is a cognitive psychologist with a good research record—on topics other than reading education. Although I know of his book, it is written for lay audiences—and the short excerpts or off-shoots that have come to my attention, suggest to me that he hasn’t actually read much of the research that you are asking about. But he has read some appropriate summary pieces about the subject and/or talked to some respected experts).

In my opinion, he is kind of right. 

What is the good Doctor W right about? He is right that comprehension strategies (e.g., summarization, questioning, monitoring) are effective. There are a number of research reviews of this work, both focused on individual strategies and strategy teaching overall, and they are consistently positive. Teaching comprehension strategies appears to improve students’ reading comprehension, and it doesn’t matter if the review is somewhat comprehensive (NICHD, 2000) or highly selective, only including in the highest quality studies (Shanahan, et al., 2010); the answer is the same.

And, he is especially right to raise the issue of, “How much of this kind of teaching is needed?”

But that’s where my answer would deviate from his, and where reading the actual studies instead of the reviews can make a big difference. He claims students learn everything they need after 2 weeks of strategy instruction, and that we should limit such teaching to that extremely limited duration.

I think that claim is on very thin ice and it ignores a lot of issues and a lot of studies (remember the National Reading Panel reviewed more than 200 studies on the topic).

I say three cheers for Dan Willingham for questioning the amount of strategy instruction and I give him the raspberries for then answering his question that two weeks of strategy teaching is appropriate.

One thing that originally shocked me in reading the studies in that research literature was how brief the interventions were. Most studies focused on 6 weeks of instruction or less (though there were a few longer studies). That such brief interventions are potent enough to impact standardized reading tests is good. That we have no idea whether stronger doses have any added benefit is a serious problem. That’s why I agree with the notion that we are probably overdoing the strategy teaching. The only evidence we have on amount of strategy teaching is correlational and it is weak at best.

My conclusions:
(1)  Strategy instruction is effective when the instruction is concentrated. In all of the studies, students were given daily ongoing instruction of and practice with strategies. Programs that give occasional doses of instruction in various strategies may be effective, but there are no studies of that kind of practice.
(2)  Strategy instruction can be effective at improving reading comprehension scores at a variety of grade levels, including the primary grades. This surprised me, too. I was pretty sure that comprehension strategies made sense with older students, but not so much with younger ones. That’s not what the research has found, however.
(3)  Strategies are not all equal. There is a greater payoff to some strategies than to others, so I would definitely put my instructional nickel on the ones with the big learning outcomes. The most powerful strategies by far are summarization (stopping throughout a text to sum up) and questioning (asking and answering your own questions about the text). The weakest: teaching students to think about how to respond to different question types (effect sizes so small that I wouldn’t waste my time).
(4)  Strategy instruction can be effective with about 6 weeks of teaching and practice. Here I’m going with the modal length of strategy studies. Perhaps the effects would have been apparent with fewer weeks of instruction, per Willingham’s contention, and, yet, this hasn’t been studied. Weaker dosages may work, too, but with so little evidence I’d avoid such strong claims. 
(5)  Even more strategy instruction than this may be effective, but, again, with so little research no one knows. We do have studies showing that 3 years of phonics instruction are more effective than 2 years of phonics instruction, but we don’t have such studies of reading comprehension teaching, so let’s not pretend. 
(6)  You raise a question about the value of different strategies, Willingham does not. The research reviews show that the teaching of multiple strategies, either singly in sequence or altogether, is beneficial—with stronger results than from single strategies. Multiple strategy teaching may be better because of the possibility that different strategies provide students with different supports (one strategy might help readers to think about one aspect of the text, another might foster some additional insights or analysis). Teach multiple strategies.
(7)  The Willingham claims fails to consider the outcome measures. Strategies are good or bad, but he doesn’t focus on what they may be good at. His focus is on motivating readers, but the studies of strategy teaching do not focus on this outcome. I think we overdo the strategy thing, and yet, I’d be surprised if an overemphasis on strategies is why kids don’t like reading. The whole point of strategy teaching is to make students purposeful and powerful, focused on figuring out what a text says. Those kinds of inputs usually have positive motivational outcomes.
(8)  It is great that comprehension strategies improve performance on standardized reading tests, but their bigger impact has usually been on specially designed instruments made for the research. Thus, summarizing usually helps students to summarize a text more than it builds general reading comprehension. I think the best test of strategies would be to give two groups a really hard text—like a science textbook—and have them read it and see who would do the best with it (passing tests, writing papers, etc.). I suspect strategies would have a bigger impact on that kind of outcome than passing a test with fairly short easy passages, multiple-choice questions, in a brief amount of time. If I'm correct about that, then strategies would worth a more extensive emphasis. Willingham apparently hasn't read the studies so he is considering only what they have found, not what they haven't considered.
(9)  Most students don’t use strategies. Though we know strategies improve comprehension, they are not used much by students. I suspect the reason for this is our fixation on relatively easy texts in schools. The only reason to use a strategy is to get better purchase on a text than one would accomplish from just reading it. If texts are easy enough to allow 75-89% comprehension (the supposed instructional level that so many teachers aim at), there is simply no reason to use the strategies being taught. Teachers may be teaching kids to use strategies, but their text choices are telling the kids that the strategies have no value.
(10)   Willingham is trying to reduce the amount of comprehension strategy instruction so that kids will like school better. I doubt that he spends much time in schools. He hasn’t been a teacher of principal or even a teacher educator and his own research hasn’t focused on practical educational applications. I’ve been conducting an observational study of nearly 1000 classrooms for the past few years, and we aren’t seeing much strategy instruction at all. There definitely can be too much strategy teaching, but in most places any dosage, not overdosage, is the problem.


Sunday, February 1, 2015

Examples of Close Reading Questions

As a principal, I want my teachers to teach student how to read a text closely. After going through your Powerpoint, reading the questions you suggest and the responses, I think professional development in developing questions would be required to ensure they were actually asking the right kind of questions.

            One of the biggest implementation problems with Common Core that I see is that teachers (and curriculum designers) don’t understand close reading well enough to ask appropriate questions. The point of the questions is to guide students’ to think about the text in effective ways.

            To help this principal (and others) to provide the professional development noted above, I have provided what I hope will be a useful example. But let’s start off with a bit of explanation. 

            First, the questions should be about important issues raised by the text. Some people are taking close reading to mean “precise” reading or “thorough” reading. If you are asking about a story, you should ask about details that would be important in a summary of the story (e.g., character motivation, key plot details, theme). “Close” is not a synonym for “trivial.”

            Second, the questions should be text dependent. That just means that it shouldn’t be possible to answer the question without reading the text. The focus of close reading should be on what the author presents, and not on anything else. That’s the reason why it’s a good idea for students to explain or support their answers with text evidence (proof—from the text).

            Third, the questions should help the readers to accomplish three interpretive goals. Specifically, they should help the reader to think about what the text said (key ideas and details), how the text worked (craft and structure), and what it means (integration of knowledge and meaning). Unlike in other questioning schemes, these questions do not try to get kids to exercise particular thinking skills (e.g., inference, higher order reasoning, comparison); they focus on interpreting the text rather than on exercising particular cognitive muscles.

            I have attached an old basal reader story (from the 1955-1965 period), and below I have listed questions that I might ask a group of second- or third-graders about this story. I have separated my questions out into three sets—one set for each major interpretive goal, but they don’t have to be asked in that way; they can be interspersed with each other.

            I do think that it is a good idea to ask your questions in an order that helps students to follow through the text in an “orderly manner,” particularly with regard to a first read or the key ideas and details questions. It is not enough that kids get practice reading texts, but they should come away knowing more about their world. If the questions/discussion/task takes you through the content in a well-organized way, students will be more likely to come away with content knowledge. Thus, you could have the students read this story three times, and each time use a different set of questions; or you could simply intersperse the second two sets of questions into the first wherever you think they fit best. In each list here, I have gone through the story in the same order that the author presented the pertinent information.

            Of the three sets of questions, the “craft and structure” questions are the most characteristic of close reading (the other interpretive goals are important too, but they are not unique to close reading). That means that most of us need more practice with “craft and structure”—something largely or entirely neglected in the video that I recently critiqued in this space.

            You may notice that I did not go through and try to have a balance of “right there” and “think and search questions” or that I didn’t fool with Bloom’s taxonomy. The reason is quite simple: my focus is on—and should be on—the text during close reading. If a text is very explicit, then I’ll ask a lot more comprehension or “right there” questions. If the text is more oblique, then we’ll end up with more inferencing practice. The point isn’t the inferencing practice, however, it is to get students to think closely about the meaning of the particular text we are reading now (that's one of the reasons close reading questions are hard--because they follow each text, not some questioning scheme).

Questions about key ideas and details—What did the text say?
What was special about Tom?                                                        
What did Tom do when the men were loading the train?                      
Why did he pretend to sleep?                                            
What did Tom do that got his picture in the newspaper?                      
How did his life change after he got in the newspaper?             
What happened when the chipmunk showed up?                                  
Why did Tom follow the chipmunk?                                              
When he was in the railroad car what was Tom’s problem?      
When Tom got out of the car where was he?                               
Who found Tom?                                                                             
How did the engineer know that Tom wanted to go with him? 
Why did the engineer take him?                                                    
When the engineer and Tom left what was their problem?       
When Tom yowled, what did the engineer think he wanted?    
What changed the engineer’s mind?                                             
What did the fireman think Tom meant?                         
So what did the engineer do?                                                         
According to the engineer, why was it so important Tom yowled?
How did Tom know the bridge was out?                                       
What happened after Tom saved the train?                                

Questions about craft and structure—How did the text work?
What does the author mean when he writes that Tom “had never seen a kitchen nor climbed a back yard fence”?
What is a “conveyor belt”?                                                                                      
What’s “freight”?                                                                 
On page 1, the story says that Tom was a "hero." What does that mean? (What made him a hero?)
On page 2, the author puts some words in quotation marks (“Oh, boy!,” “Fish at last!” “thank you”). What is he trying to show by doing that? Can Tom talk?
What kind of story is this (fantasy or realistic fiction)?   
On page 4, it again calls Tom a “hero.” How is the meaning of “hero” different here than on page 1?
Why does the author tell us about the chipmunk again at the end?

Questions about integration of knowledge and meaning—What did the text mean?
The author used the word “hero” in two different ways. Which meaning is the right one?
What’s the difference between being a hero and being famous?
Is it better to be a hero or to be famous?

What was the point of the story? What did the author want you to learn from Tom?

Railroad Cat Story

Sunday, January 11, 2015

Close Reading: A Video Replay

Last week, I provided a link to a video that a reader sent me Close Reading Video . The link purported to present a model “close reading” lesson.

Although, there was much to like about the lesson, I complained that it wasn't close reading. Close reading is not a synonym for reading comprehension (or even "really good reading comprehension"). 

This is happening a lot. A company says their anthologies include “complex text,” but it isn’t clear what teachers are supposed to do with it, or why it's there at all since the instructional procedures still seem to favor the idea of protecting kids from complex text.

Last week I dinged that video for claiming that close reading is a teaching technique (it's an approach to reading). I was critical of the idea that close reading helps students “conquer complex text,” if that includes language complexity as measured by Lexiles. I didn’t like the idea of reading the book to the kids; I’m a fan of reading texts to kids (see recent NewYork Times article on this), but not the texts the kids are supposed to be reading. Finally, I didn’t like how rereading was being approached. 

Here is the rest of my thinking about this lesson. Hope it’s useful to you.

1.   Confusion of story and exposition.
A big issue with the standards is the shift to informational text. Unfortunately, teachers lack experience teaching informational text, and they haven’t developed a language for it yet. In the video the teacher repeatedly refers to the “story” that the students are reading. Better choices: “informational text,” “book,” “article,” “science selection,” and so on. 

Our language cues kids as to which strategies to use and what text features to rely on. Stories have different characteristics than science articles do. They are organized differently and use language in different ways.

1    2.     The terrific teaching strategies are irrelevant to close reading.
Many teachers who watch the video are going to be impressed with the clever way the teacher had kids sharing information (the back-to-back arrangement, the whip around). Those are clever techniques and I’m all for them. They're the kind of thing that allows effective teachers to reap the benefits of small group instruction even when teaching a whole class. As a teacher educator, I’d be very pleased if my students walked away from this viewing with those techniques.

However, those techniques have nothing to do with close reading. A lesson will involve students in close reading whether or not those techniques are used. (That's why this can be a "good lesson"--because of the high engagement level of the students--but a poor lesson, if the goal was to engage them in close reading.  

2    3.     Close reading focuses on the text, not the reading strategies.
A major purpose of close reading was to shift readers' attention from authors’ biographies, the historical period from which the text emerged, or from past critical response. It aimed to shift this attention to the text itself.  
          
One of the biggest problems with the presentation is its heavy emphasis on main idea and key detail detection, annotation techniques, rereading procedures. What the author had to say and how the author said it is getting lost here. That’s why I see this lesson as no different from what was common in schools in the 1960s, 70s, 80s, 90s, and the Oughts. This isn’t an advance; it is just a new set of labels for what we were doing before.

I do believe that, as teachers, we need to teach the reading process to kids, and having some lessons that focus on how to summarize or question a text makes great sense. Similarly, I’m all for explicitly teaching kids some of the common ways that texts are organized and to have them practice reading texts to use those strategies or to figure out a text’s structure. But, as useful as such lessons can be, they are different than the lessons in which the emphasis should be entirely upon the content and approach of a particular text.

One can’t really tell from the video when certain things happened (is this what the teacher started with or did she tell the kids this after they had read the text once or twice?). One example is purpose. She stresses that the purpose is to get the main idea and details and then tells students to look for the main ideas (she even helps this along by asking them what they know about adaptation). The problem is that her purposes are more about the reading process than the text. 
            
A model lesson on close reading should stress the text, not the reading strategies. And, it should focus attention on not just what the text said, but how the author expressed, reinforced, or extended the meaning through his/her choices of language and structure. This lesson ignored tone, the role of illustrations, why the author chose particular words, or why information was sequenced in particular ways. Kids will likely come away with some of the facts (and that is good), but there is more to it.


Sunday, January 4, 2015

A Close Read of a Close Reading Video

            My daughters are Erin and Meagan. When they were little, Meagan would get upset because we always “ran Erins,” but never “ran Meagans.”

            That’s cute when a little one doesn’t know the meaning of a word. But such miscommunication can be a real problem in Common Core State Standards implementation.

            It’s getting so that I hate to hear the term “close reading” because it is misused so often these days. 

            A comment from a reader of last week’s blog entry challenged me to evaluate an online video of a close reading lesson. I gave it a quick review and replied. 

            It’s been bugging me ever since, and I decided to give this 8-minute video a close read of my own. I’m going to be pretty critical, but please don’t take that as an attack on this teacher (these video minutes are all I know or her). She looks to be pretty good teacher. But the close reading espoused here is not especially well connected to the concepts of close reading or Common Core. 

            Because of length of my critique, I'll spread the analysis over two blog entries. Here's the first:

1          1.     The video says close reading is an “instructional strategy.”
It is not. More properly, it is a way of reading text. Viewers should not watch this with the idea that this is how you teach close reading. There are some great teaching techniques here, but a teacher who followed these steps scrupulously would not be teaching kids to be close readers.   

2.  The video indicates close reading helps students “conquer complex text.”
That’s sort of true, but not as demonstrated in this video. Texts are complex in multiple ways, and all approaches to reading can be expected to address some of that complexity. For example, I don’t know of any reading approach that doesn’t require readers to come away with a text’s main points and key details. All past reading standards in the U.S. trumpeted those particular skills already, so a shift to close reading would change nothing in that regard.   No wonder some teachers tell me that they have always taught “close reading.”
The teacher in the video is correct that close reading is useful for dealing with texts that have “layers of meaning.” But she doesn’t demonstrate that in any way in the video (main ideas and key details are not layers of meaning).
In this kind of text, “layers of meaning” might require a consideration of the effects of how the text conveyed the information (how the telling extended or reinforced those main ideas and key details). For example, in his explanation of natural selection, Darwin writes: “The tail of the giraffe looks like a fly-trapper; and it seems at first incredible that this could have been adapted by successive modifications for so trifling an object as to drive away flies.” A close reader should wonder why Darwin focuses on such a “trifling object” in this magnificent argument.
But that, of course, was Darwin’s point. He wanted to show that even the tiniest organs of little apparent importance were affected by natural selection in ways that we could only guess at. Asking students what the giraffe does with its tail or toward what end the adaptation of the tail progressed are fair questions, but they aren’t close reading questions, per se because they don’t include an analysis of those rhetorical considerations.

3          3.     The teacher reads the text to the students.
If this is the “close reading instructional strategy” and its purpose is to teach students to “conquer complex text,” then reading the complex text to the students is going to be many teachers’ takeaway. And it would be a bad one. The kids need to do the reading if they are going to become better readers.
Close reading has nothing to do with whether a text is read aloud to students or whether they read it themselves. Doing the reading for kids will not make them stronger readers. The point of having kids read texts with higher Lexiles estimates is not so teachers can practice their reading skills, it is so kids can do so. I think this teacher makes a big mistake reading the text to the kids instead of giving them a chance to make sense of what it says.
This is not an issue of close reading, but of complex text. Those are two separate, but overlapping, issues in Common Core. Students need to learn to deal with text complexity, including learning to read complex language and dealing with the complex ideas. The teacher here seems to recognize that close reading won’t help the kids to read the challenging language of this text, so she does that part of the work for them (she takes challenging language out of the equation by making sure that no one actually has to deal with it). 

4          4.     Close reading requires multiple readings of a text.
This idea is correct. Going through a complex text more than once is often necessary to figure out what the text says and how it works, or to develop a deeper understanding of it. But, again, there are two ideas operating here. One of them is that reading and rereading is a kind of “try and try again” or “practice makes perfect” idea; if you didn’t get it the first time, maybe you will on a second read. Repeated reading in fluency is kind of like that: a student reads a text aloud making fewer miscues on each rereading.
That’s not a bad thing, and I have no doubt these third-graders will benefit from this kind of thorough attention to the content of this book. This teacher definitely is not just rushing through the text to get it done; it looks to me like these students will come away knowing something about adaptation and that’s a real plus.
However, the rereading that is inherent in close reading requires a bit more than that. It isn’t about doing a better job each time. It’s about doing a different one. Yes, it might take 8-year-olds two or three readings just to come to terms with what a text has to say. But that isn’t the rereading that is central to close reading.
In close reading, now that you understand what a text has to say, you can reread it to determine how it works. For example, how did the illustrations help you to understand what the author meant by adaptation? Or, why do scientists use the term “adaptation” instead of “change”?
The video shows kids rereading to figure out what the main idea and key details of the text were. That’s terrific and this teacher did that well. But that isn’t what we mean by close reading alone isn’t what is meant by close reading, and kids who can only do that with a text will not accomplish the standards.

Disclaimer: Publicly critiquing a video lesson is inherently risky. It's possible that the instructional segment is just part of a lesson, and that had the viewer seen the whole thing, the analysis would be quite different. Or, perhaps it is one lesson in a developmental sequence, and in future lessons the teacher would move the reading over to the kids, and would have them dealing with the more analytical and evaluative aspects of close reading as they read additional texts. The point of this critique is not that this is a bad teacher, or even that this is a bad lesson (neither of those conclusions are mine), but that this is not a particularly apt illustration of close reading or close reading preparation.

Monday, December 29, 2014

Why Reading Strategies Usually Don't Help the Better Readers

Last week, I explained why disciplinary reading strategies are superior to the more general strategies taught in schools. That generated a lot of surprised responses.

Some readers thought I’d mis-worded my message. Let me reiterate it here: strategies like summarization, questioning (the readers asking questions), monitoring, and visualizing don’t help average or better readers. They do help poor readers and younger readers.

I didn’t explain better readers don’t benefit, so let me do that here.

Readers read strategically only when they have difficulty making sense of a text.
Recently, I was took a second shot at reading the novel, Gilead. I tried to read it a few months ago, but couldn’t follow the plot. I often read just before sleep and especially subtle or deep texts are not usually best read a few pages at a time like that.

In the meantime, Cyndie read it with great enjoyment, so now my self-image as a sophisticated reader was on the line. For my second reading, I carved out bigger chunks of time, and marked the text up quite a bit (even writing a summaries of the first several chapters). This time, I read with great understanding. Whew!

If the book had been easy for me, I never would have gone to that kind of trouble.

Let’s face it: school texts are not particularly hard for average readers and above. We teach strategies to them, but they don’t really need them—at least not with the texts we use to teach reading.

It may not even matter much if a student understands a text. Students can often hide out, letting the others answer the hard questions, and gaining sufficient info from the discussions and illustrations. No need for strategies under such circumstances.

The new emphasis on teaching students with more challenging texts—texts not as likely to be understood from reading alone—should increase the value of general reading strategies.

Of course, even good readers sometimes confront challenging texts at school (like ninth grade biology textbooks). Unfortunately, they often don’t use reading strategies even with such texts.

My guess as to what is going on is two-fold: students who usually get by on the basis of language proficiency alone, have no idea what to do when confronted with such demands. They go into default mode, not using the strategies at all—even though in this context such strategies would probably be helpful.

But let’s face it. Too often, meaning just doesn’t matter at school. Students can often get by with a superficial purchase on the content. I once got half credit on an astronomy exam question that asked how to measure the distance to the Northern Lights (my answer: use the same method that you’d use to measure the distance to the moon—a correct answer, and yet one that doesn’t require any grasp of the content).

Superficial understanding is often enough in school. Low readers may not be able to gain this successfully by applying their language skills alone, so strategies increase their chances. Good readers can, but when the stakes are raised they don’t necessarily adjust and start using the general reading strategies. But no matter how challenging the texts are, if “acceptable levels” of performance are low enough, strategies again won’t be necessary.

Yes, we should teach reading comprehension strategies, even to good readers. But we should do so in an environment that emphasizes the value of knowledge and understanding, and that requires students to confront genuine intellectual challenges. Those disciplinary literacy strategies touted in my last entry seem to have motivation built in: trying to connect the graphics and the prose in science to figure out how a process works; or judging the veracity of multiple documents in history; or determining which protagonist an author is most sympathetic to in literature tend to be more purposeful and intellectually engaging than turning headers into questions or summarizing the author’s message.