Showing posts with label amount of instruction. Show all posts
Showing posts with label amount of instruction. Show all posts

Sunday, February 12, 2017

How Much Reading Gain Should be Expected from Reading Interventions?

This week’s challenging question:
I had a question from some schools people that I’m not sure how to answer. I wonder if anyone has data on what progress can be expected of students in the primary grades getting extra help in reading. 

Let’s assume that the students are getting good/appropriate instruction, and the data were showing that 44% of students (originally assessed as “far below”) across grades 1-3 were on pace to be on grade level after 2 years of this extra help.

Is this expected progress for such students or less than what has been shown for effective early reading interventions?

Shanahan’s answer:
            This is a very complicated question. No wonder the field has largely ducked it. Research is very clear that amount of instruction matters in achievement (e.g., Sonnenschein, Stapleton, & Benson, 2010), and there are scads of studies showing that various ways of increasing the amount of teaching can have a positive impact on learning (e.g., preschool, full-day kindergarten, afterschool programs, summer school programs).

            Although many think that within-the-school-day interventions are effective because the intervention teachers are better or the methodology is different, but there is good reason to think that the effects are mediated by the amount of additional teaching that the interventions represent. (Title I programs have been effective when delivered after school and summer, but not so much with the daytime within school (Weiss, Little, Bouffard, Deschenes, & Malone, 2009); there are concerns about RtI programs providing interventions during reading instruction instead of in addition to it (Balu, Zhu Doolittle, Schiller, Jenkins, & Gersten, 2015)).

            Research overwhelmingly has found that a wide-range of reading interventions work—that is the kids taught by them outperform similar control group kids on some measure or other—but such research has been silent about the size of gains that teachers can expect from them (e.g., Johnson & Allington, 1991). There are many reasons for such neglect:

(1)  Even though various interventions “work” there is a great deal of variation in effectiveness from study to study.

(2)  There is a great deal of variation within studies too—just because an intervention works over all, doesn’t mean it works with everybody who gets it, just that it did better on average.

(3)  There is a great deal of variation in the measures used to evaluate learning in these studies—for example, if an early intervention does a good job improving decoding ability or fluency, should that be given as much credibility as one that evaluated success with a full-scale standardized test that included comprehension, like the accountability tests schools are evaluated on?

(4)  Studies have been very careful to document learning by some measure or other, but they have not been quite as rigorous when it comes to estimating the dosages provided. In my own syntheses of research, I have often had to provide rough guestimates as to the amounts of extra teaching that were actually provided to students (that is, how much intervention was delivered).

(5)  Even when researchers have done a good job of documenting numbers and lengths of lessons delivered, it has been the rare intervention that was evaluated across an entire school year—and, I can’t think of any examples, off hand, of any such studies longer than that. That matters because it raises the possibility of diminishing returns. What I mean is that a program with a particular average effect size over a 3-month period may have a lower size of effect when carried out for six or 12 months. (Such a program may continue to increase the learning advantage over those longer periods, but the average size of the advantage might be smaller).

            Put simply? This is a hell of a thing to try to estimate—as useful as it would be for schools. 

            One interesting approach to this problem is the one put forth by Fielding, Kerr, & Rosier, 2007. They estimated that the primary grade students in their schools were making an average year’s gain of one year for 60-80 minutes per day of reading instruction. Given this, they figured that students who were behind and were given additional reading instruction through pullout interventions, etc. would require about that many extra minutes of teaching to catch up. So, they monitored kids’ learning and provided interventions, and over a couple of years of that effort, managed to pull their schools up from about 70% of third graders meeting or exceeding standards to about 95%—and then they maintained that for several years.

            Fielding and company’s general claim is that the effects of an intervention should be in proportion to the effects of regular teaching… thus, if most kids get 90 minutes per day teaching and, on average, they gain a year’s worth on a standardized measure, then giving some of the kids an extra 30 minutes teaching per day, should move those kids an additional 3-4 months. That would mean that they would pick up an extra grade level for every 2-3 years of intervention. I’m skeptical about the accuracy of that, but it is an interesting theory.  

            Meta-analyses have usually reported the average effect sizes for various reading interventions to be about .40 (e.g., Hattie, 2009). For example, one-to-one tutoring has a .41 effect (Elbaum, Vaughn, Tejero Hughes, & Watson Moody, 2000.

            However, those effects estimates can vary a great deal, depending on when the studies were done (older studies tend to have less rigorous procedures and higher effects, etc.), by the kind of measures used (comprehension outcomes tend to be lower than those obtained for foundational skills, and standardized tests tend to result in lower effects than experimenter-made ones), etc.

            For example, in a review of such studies with students in grades 4-12, the average effect size with standardized tests was only .21 (Scammacca, Roberts, Vaughn, & Stuebing, 2015); and in another sample of studies, the impact on standardized comprehension tests was .36 (Wanzek, Vaughn, Scammacca, Gatlin, Walker, & Capin, 2016).

            You can see how rough these estimates are, but let’s just shoot in the middle someplace… .25-.30 (a statistic I obviously just made up, but you can see the basis on which I made it up—relying most heavily on the best studies, the best and most appropriate measures).

            What does that mean? As long as we are talking about primary grade kids and typical standardized reading tests, the usual size of a standard deviation is about 1 year. In other words, if you took a 3rd grade Gates-MacGinitie and tested an average group of second and third graders with it, you’d find about 1 standard deviation difference in scores between the grade level groups. (Those connections between amount of time and standard deviation change as you move up the grades, so you can’t easily generalize up the grades what I am claiming here).

            Thus, if you have a second-grader who is one full year behind at the beginning of the year (that is the class gets a 2.0 grade equivalent score in reading, but this child gets a 1.0), and the student is in a good classroom program and an effective intervention, we should see the class accomplishing a 3.0 (that would be the year’s gain for the year’s instruction), and the laggard student should score at a 2.25-2.30.

            All things equal, if we kept up this routine for 3-4 years, this child would be expected to close the gap. That sounds great, but think of all the assumptions behind it: (1) the student will make the same gain from classroom teaching that everyone else does; (2) the intervention will be effective; (3) the intervention will be equally effective each year—no one will back off on their diligence just because the gap is being closed, and what was helpful to a second-grader will be equally helpful with a third-grader; (4) the intervention will continue to be offered year-to-year; and (5) that the tests will be equally representative of the learning elicited each year.

            That tells you how much gain the group should make. Your question doesn’t tell how far behind the kids were when they started, nor does it tell how much gain was made by the 56% who didn’t reach grade level… so moving 44% of them to grade level in 2 years may or may not be very good. I could set up the problem—plugging in some made up numbers that would make the above estimates come out perfectly, which would suggest that their intervention is having average effectiveness… or I could plug in numbers that might lead you to think that this isn’t an especially effective intervention.

            I have to admit, from all of this, I don’t know whether their intervention is a good one or not. However, this exercise suggests to me that I’d be seeking an intervention that provides at least, on average, a quarter to a third of a standard deviation in extra annual gain for students. And, that has some value.

References
Balu, R., Zhu, P., Doolittle, F., Schiller, E., Jenkins, J., & Gersten, R. (2015). Evaluation of response to intervention practices for elementary school reading. Washington, DC: U.S. Department of Education.
Elbaum, B., Vaughn, S., Tejero Hughes, M., & Watson Moody, S. (2000). How effective are one-to-one tutoring programs in reading for elementary students at risk for reading failure? A meta-analysis of the intervention research. Journal of Educational Psychology, 92, 605-619.
Fielding, L., Kerr, N., & Rosier, P. (2007). Annual growth for all students… Catch up growth for those who are behind. Kennewick, WA: New Foundation Press.
Hattie, J. (2009). Visible learning. New York: Routledge.
Johnson, P., & Allington, R. (1991). Remediation. In R. Barr, M. L. Kamil, P. B. Mosenthal, & P.D. Pearson (Eds.), Handbook of reading research (vol. 3, pp. 1013-1046). New York: Longman.
Scammacca. N.K., Roberts, G., Vaughn, S., & Stuebing, K.K. (2015). A meta-analysis of interventions for struggling readers in grades 4-12: 1980-2011. Journal of Learning Disabilities, 48, 369-390.
Sonnenschein, S., Stapleton, L. M., & Besnon, A. (2010). The relation between the type and of instruction and growth in children’s reading competencies. American Educational Research Journal, 47, 358-389.
Weiss, H.B., Little, P.M.D., Bouffard, S.M., Deschenes, S.N., & Malone, H.J. (2009). The federal role in out-of-school learning: After-school, summer school learning, and family instruction as critical learning supports. Washington, DC: Center on Education Policy.
Wanzek, J., Vaughn, S., Scammacca, N., Gatlin, B., Walker, M.A., & Capin, P. (2016). Meta-analyses of 
      the effects of tier 2 type reading interventions in grades K-3. Educational Psychology Review, 28, 
      551-5

Sunday, August 7, 2016

Why An Overemphasis on Foundational Reading Skills Makes Kids Sick

Principal’s question:

District leadership has advised primary teachers to focus on the Foundational Skills Strand, and de-emphasize the other strands. The belief is that if students go into Grade 3 having mastered foundational skills, they will be prepared to master the rigor of the other strands.

As the principal, the message I'm considering sending is to teach all strands, closely monitoring foundational skills with DIBELS, immediately addressing gaps. Students who are meeting foundational skills standards may spend more time in other strands while those struggling get focused support in assessed areas of foundational skills difficulty. Does that sound reasonable?

I'm concerned that de-emphasizing the other strands will make it hard for students to catch up in third grade, and many students may lose interest if not exposed to a variety of thought-provoking work. On the other hand, I understand the immense importance of systematic, explicit instruction in the foundational skills- and know they must be a focus in early years.

All that said, can you give a guideline as to the percent of the E/LA time that should be spent on foundational skills for the "typical" primary student? Our district adopted Benchmark Advance, which looks to me as though it does NOT emphasize the foundational skills. I would like to give teachers a time guideline for initial whole-group instruction in foundational skills so we know how much we may need to supplement with other curriculum.

Shanahan’s response:

     Imagine if district leadership advised the cafeteria crew to focus on calcium only, and to de-emphasize the other nutrients? Their belief might be that if students reached the age of 8 without strong teeth and bones, they would not be prepared for the later rigors of eating grains, meats, and vegetables.

     You’d be writing to me to find out if it’s okay to serve cereals with the morning milk and green beans at lunch. And, let's face it, these kid's autopsies would likely reveal strong teeth and bones.

     Sadly, this analogy is apt.

     Of course, one can put all the primary grade focus on some skills to try to advance progress in those skills, just as one could put all the emphasis on some nutrients to promote some health needs over others. Doing so won't accomplish the real goal, but it might fool some observers into thinking it has been reached.

     Here are some facts worth knowing:

1.  In the 1960s, the National Institute of Child Health and Human Development (NICHD) began a rigorous analysis of beginning reading in an effort to identify how effectively to avoid or to address learning problems. This coordinated effort is generally credited with much of the progress that has been made in understanding the role that skills like phonological awareness play in reading and the value of explicit phonics instruction. One important finding of that effort: addressing only students’ phonological/orthographic needs during the primary grade years leaves those students vulnerable to continued reading disability (due to a lack attention to their language development). There either are usually undiagnosed language deficits early on, that become more evident later, or the inattention to non-foundational skills limits their growth during these years. I don’t think anyone can read that body of research without concluding both that kids need substantial attention to foundational skills early on, AND that solely focusing on such skills would be harmful.

2.  The National Reading Panel was pressed into service to review research on what works in reading at the request of the U.S. Congress, under the auspices of NICHD and the U.S. Department of Education. Unlike many of the critics at the time, panel members, who were unpaid volunteers, were not allowed to have any potential conflicting commercial interests. That panel reviewed 51 studies of the teaching of phonemic awareness, 38 studies of phonics, and 32 studies of oral reading fluency. The panel concluded that students would benefit from explicit, systematic instruction in each of those foundational skills during the primary grades. However, it should be noted that in no case within those studies did anyone consider those skills as separable from the rest of reading. For example, when studying phonics, the students in the control groups and the phonics groups were receiving instruction in vocabulary, comprehension, writing and the like. The only difference was that the experimental group would be getting phonics or some more ambitious version of phonics. Thus, the panel’s conclusion that these skills need to be taught was determined in the context of these skills being taught along with other reading skills. Such a heavy focus on any of these skills to the omission of the others likely would have led to very different conclusions.

3.  Over my career, I have worked with some of the biggest proponents of foundational skills teaching: Patricia Cunninghma, Linnea Ehri, Jack Fletcher, Barbara Foorman, David Francis, Douglas Fuchs, Lynn Fuchs, Christopher Lonigan, Louisa Moats, Michael Pressley, Christopher Schatschneider, Sally Shaywitz, Steve Stahl, Keith Stanovich, Joseph Torgesen, Sharon Vaughn, etc. These brilliant men and women disagree—with me and with each other--on many issues, but they seem to all be in agreement that the foundational skills are NECESSARY for learning to read (so you'd better make sure kids are instructed in them), BUT THAT THEY ALONE ARE NOT SUFFICIENT for learning to read (so you'd better do more for kids' reading than teach them foundational skills).

      I have long been an advocate for providing children with 120-180 minutes per day of literacy instruction. I divide that time roughly in quarters: 25% devoted to words and word parts (e.g., letters, sounds, decoding, PA); 25% to oral reading fluency; 25% to reading comprehension; and 25% to writing. That means that primary grade kids would receive about 60 to 90 minutes per day of foundational skills instruction (combining the word work with the fluency work).

     There are variants on this scheme. For example, Joe Torgesen touched it up by advocating 2 hours of daily literacy instruction, with up to a third hour dedicated to remediation in those foundational skills. Thus, your idea of giving some kids more foundational work beyond the amount that everyone receives in class makes great sense and can easily be accommodated in this plan. However, ignoring essential skills that can't easily be tested to focus on ones that can be, won't help kids much.

      I sympathize with your administrators. They want a quick fix. Sadly, the positive third-grade reading data that they are imagining would at best be briefly hiding their failure. Sort of like painting over the rot in a wooden porch; the paint will make it look nice, but it won't keep the steps from soon collapsing. In addressing a problem, you must recognize what is necessary, as well as what is insufficient.


     Pass the green beans, please!

Sunday, May 22, 2016

How Can Reading Coaches Raise Reading Achievement?

Teachers question:

I have just been hired as a reading coach in a school where I have been a third-grade teacher. My principal wants me to raise reading achievement and he says that he’ll follow my lead. I think I’m a good teacher, but what does it take to raise reading achievement in a whole school (K-5) with 24 teachers?


Shanahan's answer:

            It’s easy J. Just do the following 9 things:

1.    Improve leadership.
                  
            Literacy leadership matters. You and your principal will need to be a team. The more the two of you know and agree upon the better. Over the next few years, your principal will be hiring and evaluating teachers, making placement and purchasing decisions, and communicating with the community. You need to be in on some of those things and you need to influence all of them. Your principal should tell the faculty that you speak for him on literacy matters and you both need to devote some time to increasing his literacy knowledge so he can understand and support your recommendations. I’d get on his calendar at least a couple of times per week to discuss strategy and debrief on what you are both doing, but also for professional development time for him.

2.    Increase the amount of literacy instruction.

            How much reading and writing instruction and practice kids get is critical.  Take a close look at how much of this kids are getting. Observe, talk to teachers, survey… find out how much teaching is being provided and how much reading the kids do within this teaching. Be on the look out for lost time. Mrs. Smith may schedule two hours of ELA, but she doesn’t start class until 9:12 most mornings due to late bus drop offs, milk money collection, Pledge of Allegiance, morning announcements and so on. And, her class takes a 7-minute bathroom break at about 10 each morning. She isn’t trying to teach for 2 hours, but only 1 hour 41 minutes (and the actual amount of instruction may be even less). That’s a whopping 60 hours less instruction per year than what she schedules! Try to get everyone up to 2-3 hours per day of reading and writing instruction, with a large percentage of that devoted to kids reading and writing within instruction (and, yes, a student reading aloud to the group, only counts as one student reading).

3.    Focus instruction on essential curriculum elements.

            ELA often is used for wonderful things that don’t make much difference in kids learning. I watched a “phonics lesson” recently in which most of the time was spent on cutting out pictures and pasting them to a page. The amount of sounding and matching letters to sounds could have been accomplished within about 30 seconds of this 20-minute diversion. You definitely can send kids off to read on their own, but not much learning is usually derived from this. Instead, make ia commitment to obtaining substantial instruction in each of the following research-proven components for every child.

(a) Teach students to read and understand the meanings of words and parts of words (decoding and word meaning): Dedicate time to teaching students phonological awareness (K-1, and strugglers low in those skills); phonics or decoding (K-2, or again the strugglers); sight vocabulary (high frequency words, K-2); spelling (usually linked to the decoding or word meanings); word meanings; and morphology (meaningful parts of words).   

(b) Teach students to read text aloud with fluency so that it sounds like language (accuracy—reading the author’s words as written; appropriate speed—about the speed one talks normally; and proper prosody or expression—pausing appropriately, etc.).

(c) Teach students to read with understanding and the ability to learn from text. With beginning readers this, like fluency practice, needs to be oral reading. However, by the end of Grade 1 and from then on, most reading for comprehension should be silent reading. Such instruction should teach students about text (like how it is organized, how author’s put themes in stories, or how history books differ from science books), about the kinds of information that is important (like main ideas or inferences), and ways to think about texts that will increase understanding (like summarizing along the way, or how to ask oneself questions about a text).

(d). Teach students to write effectively.  This would include training students in various means of getting their ideas onto paper—printing, handwriting, and keyboarding, but it also teaching them to write for various purposes (narration, exposition, argument), to negotiate the writing process effectively (planning, drafting, revising, editing), to write for a range of audiences, and to write powerful pieces (with interesting introductions, strong organizations, sufficient amounts of accurate information, etc.).

            All four of those are detailed in your state standards, no matter where you live, but make sure that kids get lots of teaching in each. (I’d strive for roughly 25% of the instructional time into each of those baskets—that comes out to approximately 90-135 hours per year of instruction in each of those 4 things).

4.    Provide focused professional development.

            I suspect this will be where much of your time is focused; making sure your teachers know how to teach those four essentials well. This might take the form of professional development workshops on particular topics, organizing teacher reading groups to pursue particular instructional issues, observing teachers and giving them feedback on their lessons, co-planning lessons with one or more teachers, providing demonstration lessons, and so on. You need to make sure that every one of your teachers knows what needs to be taught and how to teach it well.


5.    Make sure sound instructional programs are in place.

            It is possible to teach reading effectively without a commercial program, but there are serious drawbacks to that approach. First, there’s the fairness issue. Programs that are shared by school staff will not make all teachers equal in their ability to teach reading, but they sure can reduce the amount of difference that exists (especially when there is adequate supervision and professional development—see numbers 1 and 4 above). Second, programs can ensure that kids get instruction in key areas of reading, even when teachers aren’t comfortable providing such teaching. Basically, we want to ensure that every teacher has an adequate set of lessons for productive instruction in those four key components for sufficient amounts of time. If your teachers are skilled enough to improve upon the lessons in the shared core program, then by all means support these improvements and make sure they’re shared widely.

6.    Align assessments.

            It can be helpful to monitor kids learning, at least in basic skills areas that are amenable to easy assessment. It is reasonable, depending on the tests and the skills, to evaluate decoding skills or fluency ability formally 2-4 times per year. Of course, teachers can collect such information within instruction much more often than that. For instance, if a teacher is going to teach fluency for several minutes per day, why not take notes on how well individuals do with this practice and keep track of that over weeks. In any event, if we recognize that some students are not making adequate progress in these basic skills, then increasing the amount of teaching they get within class or beyond class can be sensible. The amount of testing needs to be kept to an absolute minimum, so this time can be used to improve reading.

7.    Target needs of special populations.

            Often there are particular groups of kids who struggle more than others within your ELA program. Two obvious groups are second-language learners (who may struggle with academics because they are still learning English) or kids with disabilities (who struggle to learn written language). Making sure that they get extra assistance within class when possible, and beyond class (through special classes, afterschool and summer programs, etc.) would make great sense. If you are making sure that everyone in the school benefits from 2 hours per day of real reading and writing instruction, then why not try to build programs that would ensure that these strugglers and stragglers get even more? I know one coach who runs an afterschool fluency program, for instance.

8.    Get parent support and help.

            Research says parents can help and that they often do. I suggest trying to enlist their help from the beginning. Many coaches do hold parent workshops about how to read to their kids, how to listen effectively to their children’s reading, how to help with homework, etc. Lots of times teachers tell me that those workshops are great, but that the parents they most wish would attend don’t show up. Don’t be discouraged. Sometimes those parents don’t get the notices (perhaps you could call them), or they work odd schedules (sometimes meetings during the school day are best for them—perhaps close to the time they have to pick their kids up from school), or they need babysitting support or translation (those one can be worked out, too).

9.    Motivate everybody.

            Just like leadership (#1 above) is necessary to get any of these points accomplished, so is motivation. You have to be the number one cheerleader for every teacher’s reading instruction, for every parent’s involvement, and for every student’s learning gains. Information about what your school is up to has to be communicated to the community so that everyone can take part. Some coaches hold reading parades in their neighborhoods, others have regular reading nights where kids in pajamas come to school with mom and dad to participate in reading activities, there are young author events, lunchtime book clubs, and million minute reading challenges, etc. You know, whatever takes to keep everyone’s head in the game. 


            Like I said, raising reading achievement is easy. You just have to know everything, get along with everybody, work like a horse, and keep smiling.