Misinformation Marches On

The winter issue of The California Reader includes a spirited response by Glenn DeVoogd to an article that I published in that outlet this fall. Nothing wrong with differences of opinion, so I’ll not use this space to try to argue that I’m wrong and he’s right on those issues. However, I will address some egregious errors in his claims.

1. Glenn says the National Reading Panel (NRP) promoted “a more skills-based approach to reading rather than a meaning-based approach focusing on comprehension” (p. 5). In fact, NRP looked at 205 studies on the teaching of reading comprehension, all with reading comprehension as the outcome. NRP considered 45 studies on vocabulary teaching and 16 on oral reading fluency, all with comprehension outcomes. Even 18 of the 52 phonemic awareness studies and 35 of the 38 phonics studies focused on reading comprehension. Maybe the complaint isn’t that NRP failed to focus on comprehension outcomes, but that we dared to consider a broader set of outcomes (like spelling, fluency, and word recognition).
2. He also claims NRP “missed some well-designed studies supporting the use of” sustained silent reading (SSR), the Book Flood studies of Warwick Elley. NRP did not ignore those studies. It searched for them systematically as described in the report, examined them, and set them aside because they only included second-language learners (beyond NRP’s scope). We were concerned about differences between first- and second-language learners, and, we were not willing to generalize from one group to the other given that their learning situations are so different. Later the National Literacy Panel for Language Minority Children and Youth (NLP), a panel devoted to synthesizing research on second-language learners, examined the Elley studies. Glenn claims the Elley studies were well-designed, but the NLP scientists were troubled by the lack of either random assignment or any kind of pretesting. The supposed “gains” from Book Flood may have been pre-existing differences. One book flood study had a sounder design and a positive result, and it was included in NLP. A provocative pattern emerged from that analysis. Three studies, including book flood, showed positive benefits for encouraging reading and three did not. The three that did had second-language learners reading independently in English, and the three negatives had the kids reading in their home languages. The English learners in the positive studies were very isolated from English and had little opportunity to hear it, see it, or use it beyond their school lessons, and this might have been why this treatment was successful. It’s funny that having kids read in their home language had no impact on their reading skills, sort of like the SSR studies with native English speakers.
3. Glenn repeats the incorrect claim that the NRP set aside studies of SSR that did not include oral reading fluency outcomes. That is not the case. That was claimed many years ago by Jim Cunningham whose critique was rife with that kind of misinformation. Glenn apparently believed the critic but failed to check this out himself. Nope, NRP did not miss some big group of SSR studies that focused on comprehension. Didn’t happen. Those studies were ALL included.
4. Glenn confuses the effects of independent reading with the effectiveness of the methods used to get kids to read more. That is a huge interpretive problem. That reading CAN have positive effects is not proof that particular ways of encouraging kids to read more will be effective (maybe not all approaches for encouraging kids to read work). I remember when Newt Gingrich set up a program to pay kids to read during the summer. Lots of school people set up a howl that claimed to pay kids to read would be ineffective. The assumption behind such complaints is that the Gingrich approach is a bad one, not that reading is bad for kids. The fact that SSR has almost no impact on kids learning (average effect sizes are a negligible .05 to .10) should bother people who want to encourage kids to read since if it doesn’t work, something else should be tried. (Since NRP various researchers, such as James Kim, have been conducting studies where they try to get a learning effect from encouraging reading. They are having a heck of a time of it, because it turns out it is not that easy to get kids to increase their reading enough to make a difference, but we are certainly learning important things from their efforts—more than we are the folks who are clinging to the failed SSR methodology).
5. Glenn attributed causality to studies that show a correlation between the amount of reading and reading quality. Doing that opens the door to lots of quack remedies to reading problems like eye movement training, learning styles, balance beam exercises, etc.—all of which claim effectiveness on the basis of such correlations. A bigger problem with correlations is the fact that the relationship between two variables can be due to their relationship with an intervening variable. I’m surprised those who push these correlations as evidence don’t bother to control for the effects of parent’s socioeconomic status. When you do that, the correlation between the amount of reading and reading ability drops dramatically. Kids whose parents have high incomes and high education read more than kids who don’t. (Shhh! Don’t tell the teachers: they might not use SSR if they knew that was the evidence on which it was based). Glenn expresses concern that teachers have stopped using SSR because of the NRP finding that it had insufficient evidence showing it works. He apparently thinks it is bad that teachers have dropped this ineffective approach. Interestingly, Glenn suggests some ways to improve SSR—and all of his recommendations make it more like instruction, very different from the SSR designs recommended in textbooks or evaluated by research or that one commonly sees practiced in actual classrooms (but more like the reading comprehension interventions that have been found to be so effective). That’s good advice in my opinion, but it makes me wonder why such a smart man is insistent that teachers continue to use such problematic approaches instead of pushing hard for alternative procedures like the ones he notes.
6. Another area I talked of in my article was the findings being reported for studies of reading to children. It turns out that almost none of those studies have reading outcomes, and that the oral language measure used to evaluate the effectiveness of these procedures (simple receptive vocabulary) has a very low relationship with later reading achievement. Glenn’s response is that reading to children has been shown to have a close connection to pre-reading skills… in other words, he takes a “skills-based approach to reading rather than a meaning-based approach focused on comprehension.” Wow, that is a very different standard than the one he had a page or two earlier for the NRP. It is those inconsistencies that undermine his arguments: he wants to be able to cherry-pick the evidence that supports his case, no matter what measures were used or how badly the studies were executed, and he wants to be able to ignore the evidence that doesn’t fit with what he wants teachers to do.

  Ultimately, that’s why these large public syntheses of research studies by scientists are so important. They are an antidote to the priesthood of professors who claim to be the ones who know best what needs to be done in schools, even as they obscure their claims in mysterious evidentiary standards and inconsistent logic.