Showing posts with label Testing. Show all posts
Showing posts with label Testing. Show all posts

Sunday, January 8, 2017

Further Arguments about Too Much Testing

I hear you.

            Last week I posted a blog challenging the amount of testing and test preparation in American reading classes. I got smacked, metaphorically, by friend and foe alike. Some posted their concerns, many more sent them to me directly.

            The grumbles from past foes are the easiest to reply to. They often expressed—in passive aggressive tones—exasperation that I have “finally” woken up to the idea that testing companies are evil and that testing is a conspiracy against kids and teachers. They know because they follow Diane Ravitch’s “research.”

            The thing is—and I’m sure this is true since I’ve reread last week’s posting—I didn’t really come out against testing. Just against over-testing and test prep generally. The politicians have imposed some testing—and I think they have overdone it—but teachers and principals are also devoting too much time to testing, and that's on us.

            Dr. Ravitch seems to be quite upset about accountability testing, which she herself helped impose on educators overriding the critics who depended upon research in their arguments. (Ravitch is an educational historian, and quite a good one, but ignrores—then and now—psychological and educational research).

            I’m not even against accountability testing, as long as the amount of testing is commensurate with the information that one is collecting. To find out how well a school or district is doing, do we really need to test every year? Do they change that fast? Do we really need to test everyone? Anyone ever hear of random sampling? Come onnnnnn!

            If Dr. Ravitch’s minions spent more time in schools, they’d know the heaviest testing commitments are the ones the districts (and, sometimes, even individual principals and teachers) have taken on themselves. We may blame those misguided efforts on the accountability testing—we all want to look good for the picture—but, it is a bad choice, nevertheless. And, it is a choice.

            I do find the critics’ vexation with me a little surprising. For example, when I was director of reading in the Chicago Public Schools (15 years ago), I was ordered, by then Mayor Daley—to emphasize test prep in my teacher education efforts in the city. Unlike some of the critics who these days are so noisy about over-testing, I had skin in the game and I refused.

            It might be worth noting that my refusal led to two outcomes that matter: (1) the Chicago Public Schools engaged in the least test prep—before or since; and (2) Chicago kids made their biggest measured gains in reading. Not a research study, but a policy dispute affecting nearly a half million kids.

            Of course, those who appreciated my past candor were now chagrined at my remarks. They weren’t necessarily upset by what I had to say about accountability testing (many of them concur that it is over the top), but they were scared to death by my comments on the various screening, monitoring, and diagnostic tests that are so much of the daily lives of primary grade classrooms.

            Again, I think I was clear, despite the concerns. The typical complaint: “I understand you, but no one else will.” That is, they get that I am not opposed to all classroom assessment, but they are sure no one else will appreciate the subtlety of what they see as a complex position.

            For example, one dear friend, a grandmother, pointed out her appreciation that her grandkids are given annually a standardized test in reading and math. The reason? She doesn’t trust teachers or schools to actually tell how kids are doing.

            The fact is too often teachers don’t tell parents how their kids are doing. For all kinds of reasons: What if a child isn’t doing well and I don’t know what to tell the parent—why raise a question I can’t answer? What if I don’t think there is anything that can be done—it’s a minority child without economic resources whose family is a wreck? What if I only notice effort and not achievement? What if I just don’t want the argument (often parents don’t like to hear that junior isn’t succeeding)?

            An annual test isn’t perfect, but it doubles the amount of information that most parents have and that isn’t a bad thing. I’m not against that kind of testing.

            One reader thought I was smacking DIBELS, but I wasn’t. I was tough on the notion that tests like DIBELS can profitably be given to ANYBODY every week or two through a school year. But not because I was anti-DIBELS.

            Twice a year I go to my dentist. She takes x-rays every fourth visit. Why doesn’t she do it every time? For two reasons: first, dental health doesn’t change that fast, so they try not to test more than would help; and, second, because x-rays can cause damage, so the balance is best struck between help and hindrance, by testing once every four checkups instead of the seemingly more rigorous testing every time.

            DIBELS-like instruments won’t do physical damage, like x-rays, but they do reduce the amount of teaching and they might shape that teaching in bizarre ways. That is harmful.

            My advice:
1.     Reduce accountability testing to the minimum amounts required to accomplish the goal. Research is clear that we can test much less to find out how states, districts, and schools are doing. Without a loss of information.

2.     Test individual kids annually to ensure parents have alternative information to that provided by teachers.

3.     Limit diagnostic testing in reading to no more than 2-3 times per school year. Studies do not find that any more testing than that is beneficial, and no research supports reducing the amount of teaching to enable such over-testing.

4.     Give most test prep a pass. It doesn’t really help and it reduces the amount of essential instruction that kids should be getting. One practice test given once one or two weeks ahead so kids will feel comfortable with the testing should be plenty.




Tuesday, January 3, 2017

Welcome 2017: Let’s Teach, Not Test

             I believe in being upfront with my readers, so let me start with a confession:  I don’t hate testing.

            I know it is a horrible thing for a so-called “educator” to admit. It’s sort of a social disease.

            Perhaps someone has a 12-step program that could help me… Assessment Anonymous. Perhaps.

            When I was a practicing teacher working on my Master’s degree, I loved collecting tests in a big notebook. Sight word lists, multiple-choice phonics quizzes, informal reading inventories, motivation questionnaires. 3-holes punched in their left margins. Organized by purpose. I loved them all.

            In one of my jobs I even did school entry testing, putting prospective kindergartners through their paces.

            Then, later, as my habit worsened, I started working on tests… the ACT, the SAT, the National Assessment, eventually even co-authoring a state test in Illinois.

            You probably know how this story comes out… everyone hooked on testing eventually hits bottom, the dark night of the soul when you know you have to change or it will be all over. I reached my nadir when I found myself writing a positive review of DIBELS for the Buros Mental Measurements Yearbook.

            Okay, now that I have that off my chest, let’s get real.

            Over the past 15 years or so, we have so overdone the testing thing. Not just the tests that educators usually don’t like—the high stakes accountability tests—but even the instructionally relevant ones that we believe can be beneficial… the running records, informal reading inventories, DIBELS-style screeners and monitors, and a slew of acronym-titled diagnostic measures. All of them. Too much. Too damn much.

Accountability Testing
            Accountability testing was not a bad idea… it just hasn’t worked the way its proponents thought it would. Nothing wrong with that: You have an idea; you try it out on millions of kids without any empirical evidence that it will work; then after a couple of decades of doing that with few victories… you keep doing it?

            The basic idea was this: schools should be run more like businesses. Business figured out how to improve quality by measuring quality. By carefully monitoring their products and services—by testing them, they could ensure higher quality. It’s why your car starts in morning, every morning.  

            By analogy, the idea was that if we tested kids, we’d see which districts, schools, and teachers weren’t getting the job done, then resources and efforts would be focused and kids’ learning would improve. That movement started back in the 1970s, but really got going full-bore in the 1990s… more than 20 years ago. Needless to say, we are still waiting with baited-breath for the uptick.

            I still like the idea of the public knowing how well schools are doing, even if that has no direct impact on kids’ learning. However, we don’t need to test as much as we do to find out how schools are doing. Such tests need to be as brief as possible, and they only should be administered to samples of children, not all children (the National Assessment does a very good job of this on a national basis, testing fewer than 100,000 kids every two years).

            But whether or not we adopt an accountability-testing plan that makes sense, there is NO excuse for teachers to spend inordinate amounts of time getting kids ready for these exams. So-called “test prep” should be banned if it goes more than a couple of hours a year; like having kids take a practice test the week before testing. Almost all of the time currently devoted to prepping kids for the PARCC, SBAC, STAAR, Aspire, and the other state tests should be devoted to…wait for it… teaching! That time could be profitably spent teaching reading, writing, math, science, social studies, and the rest of the curriculum.

            Why would I recommend such a crazy thing? Because the surest way to raise reading achievement is not through test prep, but through teaching kids to read.

Instructional Testing
            But the testing glut is not just due to the politicians and their accountability schemes. A good deal of the over-testing we have brought on ourselves. Again, the theory has seemed reasonable.

            If we know which kids are lagging in which skills, then we can be sure to teach those skills to the right kids, and voila, higher reading achievement. This idea is especially prevalent among those responsible for kids with learning problems; often it is proposed that those children be tested weekly! The claim is that such testing represents a more rigorous effort on behalf of the strugglers.

            But that claim has no basis in research at least as far as reading achievement goes. I’m not arguing against occasionally testing certain skills to see what kind of progress is being made, and if anyone is falling through the cracks… but that can be accomplished well by testing 2-3 times per school year. I’m also not talking about the teachers who observe kids’ performance within daily instruction and who look carefully at kids’ written work (in fact, they’re my heroes).

            But interrupting instruction frequently to have kids take tests—even tests aimed at focusing instruction—is a big time waster. There is no evidence that such testing regimens actually improve learning, but there is plenty of evidence supporting the teaching of reading. Our New Year’s resolution should be, “Let’s teach, not test!” Let’s devote our instructional time to teaching kids to read—not to preparing them for tests, not for administering tests. 

Tuesday, September 22, 2015

Does Formative Assessment Improve Reading Achievement?

                        Today I was talking to a group of educators from several states. The focus was on adolescent literacy. We were discussing the fact that various programs, initiatives, and documents—all supposedly research-based efforts—were promoting the idea that teachers should collect formative assessment data.

            I pointed out that there wasn’t any evidence that it actually works at improving reading achievement with older students.

            I see the benefit of such assessment or “pretesting” when dealing with the learning of a particular topic or curriculum content. Testing kids about what they know about a topic, may allow a teacher to skip some topics or to identify topics that may require more extensive classroom coverage than originally assumed.

            It even seems to make sense with certain beginning reading skills (e.g., letters names, phonological awareness, decoding, oral reading fluency). Various tests of these skills can help teachers to target instruction so no one slips by without mastering these essential skills. I can’t find any research studies showing that this actually works, but I myself have seen the success of such practices in many schools. (Sad to say, I’ve also seen teachers reduce the amount of teaching they provide in skills that aren’t so easily tested—like comprehension and writing—in lieu of these more easily assessed topics.)

            However, “reading” and “writing” are more than those specific skills—especially as students advance up the grades. Reading Next (2004), for example, encourages the idea of formative assessment with adolescents to promote higher literacy. I can’t find any studies that support (or refute) the idea of using formative assessment to advance literacy learning at these levels, and unlike with the specific skills, I’m skeptical about this recommendation.

            I’m not arguing against teachers paying attention… “I’m teaching a lesson and I notice that my many of my students are struggling to make sense of the Chemistry book, so I change my up my upcoming lessons, providing a greater amount of scaffolding to ensure that they are successful.” Or, even more likely… I’m delivering a lesson and can see that the kids aren’t getting it, so tomorrow we revisit the lesson.

            Those kinds of observations and on-the-fly adjustments may be what all that is implied by the idea of “formative assessment.” If so, it is obviously sensible, and it isn’t likely to garner much research evidence.

            However, I suspect the idea is meant to be more sophisticated and elaborate than that. If so, I wouldn’t encourage it. It is hard for me to imagine what kinds of assessment data would be collected about reading in these upper grades, and how content teachers would ever use that information productively in a 42-minute period with a daily case load of 150 students.

            A lot of what seems to be promoted these days as formative assessment is getting a snapshot or level of a school’s reading performance, so that teachers and principals can see how much gain the students make in the course of the school year (in fact, I heard several of these examples today). That isn’t really formative assessment by any definition that I’m aware of. That is just a kind of benchmarking to keep the teachers focused. Nothing wrong with that… but you certainly don’t need to test 800 kids to get such a number (a randomized sample would provide the same information a lot more efficiently).

            Of course, many of the computer instruction programs provide a formative assessment placement test that supposedly identifies the skills that students lack so they can be guided through the program lessons. Thus, a test might have students engaged in a timed task of filling out a cloze passage. Then the instruction has kids practicing this kind of task. Makes sense to align the assessment and the instruction, right? But cloze has a rather shaky relationship with general reading comprehension, so improving student performance on that kind of task doesn’t necessarily mean that these students are becoming more college and career ready. Few secondary teachers and principals are savvy about the nature of reading instruction, so they get mesmerized by the fact that “formative assessment”—a key feature of quality reading instruction—is being provided, and the “gains” that they may see are encouraging. That these gains may reflect nothing that matters would likely never occur to them; it looks like reading instruction, it must be reading instruction.

            One could determine the value of such lessons by using other outcome measures that are more in line with the kinds of literacy one sees in college, as well as in civic, familial, and economic lives of adults. And, one could determine the value of the formative assessments included in such programs if one were to have groups use the program, following the diagnostic guidance based on the testing, and having other groups just use the program by following a set grade level sequence of practice. I haven’t been able to find any such studies on reading so we have to take the value of this pretesting on the basis of faith I guess.

            Testing less—even for formative purposes—and teaching more seems to me to be the best way forward in most situations. 

Saturday, June 20, 2015

Making Whole Class Work More Effective

          Recently, I wrote about the quandary of grouping. Small group instruction supports greater student engagement, higher amounts of interaction, greater opportunity for teacher observation, and more student learning. However, the benefits of small group are balanced by the relative ineffectiveness of most seatwork activities. Subtracting the downside of working on one's own away from the teacher from the clear benefits of small group teaching, one ends up with little advantage to all of the effort of orchestrating the small-group oriented classroom.
          Despite this, the benefits of small group teaching is so obvious, it is not uncommon for coaches and supervisors to promote a lot of small group work in spite of its ultimate lack of benefit.

          While arguing to keep the small group-teaching arrow in my quiver, I suggested that one of the best things we could do as teachers was to work on our large-group teaching skills. The focus of this has to be, not on organizing our classes in particular ways, but in ensuring that all of our students learn as much as possible. 

So what kinds of things can one do to make large group or whole class teaching more effective? In other words, how can you maintain the efficiency of whole-class teaching, while grabbing the same benefits one gets from small-group work?

1.     Get close to the kids
            In small-group work, teachers command greater attention and involvement partly by being so close. Small groups are often arrayed around the teacher or pulled together at a single table. But with whole-class work, the teacher may as well be on the Moon. Perching yourself at the desk or whiteboard puts you in a different orbit than the kids. No eye contact with the individual students, or no chance that you’ll reach out and touch them; no wonder we lose attention. Set up your classroom so that you can move easily among the students and can reach them without a lot of rigmarole. Place students where you want them to be to support high attention (no Billy cannot sit where he wants).

2.     Ask questions first and assign them to students later
          One way of maximizing attention is to ask your questions first, and then call on the student who is to answer. Even put a bit of pause in between the question and the assignment. The point of the question is rarely to get one student thinking, but to get the whole class to reflect on the problem. When a teacher says, “Johnny, why was Baby Bear so upset with Goldilocks?,” Johnny will think about it, but most of the other kids will take a pass. When she says, “Why was Baby Bear so upset with Goldilocks?.... Johnny?” everybody has to think about it because they can’t be sure who'll get called. 

3.     Focus on teaching, not putting on a show
          Many of us grew up watching Phil Donahue and Oprah. We know how to run a Q&A discussion with a studio audience because we have seen it so often. The tempo moves along, there aren’t long pauses or digressions, and at the end the pertinent info has been covered. But what’s good TV would be lousy teaching. The idea that you’re the emcee presenting information—even with some audience participation, is the wrong mindset. You may be teaching a group of 30 students in a whole class setting, but you have to think of them as 30 individuals, not one group. Your job is to maximize participation for the students while increasing your opportunity to monitor individual progress.

4    Maximize student response.
                 Too often in whole-class work the teacher asks a question, then calls on a child to answer. There are many better schemes for this that allow more student thinking and response, such as “think-pair-share.” Here the teacher asks a question, but has the kids talking it over with each other before answering (the smallest configuration for this can be pairs, but the pairs can then talk to other pairs, and other schemes make sense as well). This increases the degree to which everyone thinks about the question and tries to figure out an answer.

               Another popular approach is the multiple-response card. With simple yes-no tasks, thumbs up-thumbs down may be sufficient. Thus, if the teacher is doing a phonological awareness activity, she may have the students respond with thumbs up if a pair of words start or end with the same sound, and a thumbs down otherwise. For more complex responses, cards may be better. For example, the students might have a card for each character in a story, and the teacher can then ask questions like, Who packed the picnic basket? Who was supposed to take the basket to grandmother? Who was lurking in the woods? And, all the students then hold up the cards that reveal the answer.

                A third way, not used enough in my opinion, is the written answer. Teachers can ask any kind of question, and have everyone write an answer to the question. The oral responses that follow tend to be longer and more involved than what kids come up with orally. The written record is useful here because it allows teachers to check to see who answered the question well, the quality of the reasoning, and can take them back into the text to figure out the discrepancies.

5    Teach groups in whole class—teaching in a fishbowl
            Sometimes you can increase the involvement of particular students even though you are working in whole class. Let’s say everyone has been asked to read Chapter 6 of the social studies book, and now the class is going to discuss. The teacher might select 5-8 students who she wants to be the primary discussants this time. These students may sit in a circle in the middle of the classroom and everyone else will be arrayed around them. The teacher leads the discussion with her questions and challenges, and the students in the inner circle answer and talk about the ideas. The students on the outside observe, participate in the discussion if the inner group is stuck, and perhaps write answers to the same questions. Through careful selection, the teacher is able to maximize the amount of participation of quiet students or those who usually get shut out of the discussions by being too slow.

6    Be strategic in calling on students 
          It can be difficult to manage the calling on students. Certain students always seem to have an answer, and are quick to respond. This shuts out others who need to explore their thinking and who would benefit from teacher follow up. Teachers can do what football coaches do, which is plan their plays ahead of time, changing up the routine only if the situation changes. Thus, a teacher might, during planning, decide not just what to ask, but who she wants to hear from. That means if certain students are struggling to give longer answers or sufficient explanation, the teacher can be ready to initiate and guide them through some scaffolded work within the context of the whole class lesson. In other cases, more randomized calling (in which everyone has an equal chance) might make sense; this is easily accomplished with the tongue-depressor routine, in which all the student names are on tongue depressors and the teacher just pulls sticks out of the can as she needs a response or explanation.

7    Whole class can be more than lecture or Q&A
          Instead of using worksheets as “shut up sheets” (thanks, Vicki Gibson), use these tasks to engage everyone within the class in an interactive activity. For example, let’s say the task is finding text evidence. The worksheet includes assertions based on the text, and the students have to locate information from the text that supports the assertion. Kids could go off and do that on their own or they could do it in separate small group activities with teacher scaffolding, but that kind of task could be done most efficiently with teacher participation in the whole class. The teacher needs to observe how the students go about the task—maybe even taking notes on who just started reading and who went to particular parts of the text, who's copying, who's paraphrasing, and so on. At any point, the teacher might stop the class and ask about the strategies being used and might provide some guidance for proceeding more effectively.

            Remember, even in whole class teaching, you want students to pay attention; you want to get as many students to respond and participate as possible (without losing everyone else’s attention); you want maximum possibility of identifying when problems and misunderstandings occur so that you can scaffold, explain, and guide students to solve the problem. Structure whole group lessons in those ways, and then follow up in smaller groups (and even individually) to ensure success with what is being taught.

My recent presentation on improving test performance:

My recent presentation on teaching with challenging text: