We're Getting Odd Reading Results from Our Progress Monitoring Tests

I was looking at these roller coaster trends as well as "drops" in scores from fall to spring on our CBM measures, especially in our 5th grade class district-wide. About 90% of the students did worse if spring than fall. I figured 90% of our 5th graders were unlikely to perform worse on a measure than the last time after a year of instruction, so I took the test myself and asked 3 other teachers to take it as well. We found that the test was incredibly poorly written. Some questions had multiple correct answers, with direct textual evidence, yet only one was deemed correct. One CBM's correct answer for one question was completely wrong. Finally, the test passage itself was incredibly prejudicial in its content. We were appalled at the "moral" of the story, which was that you should be grateful for your food and obey your parents or you might end up like the homeless people in the shelter where the kid was forced to volunteer.

My concern with some of these adaptive and computer based measures is that we're not allowed to see them in full. We've been using this particular measure for years, yet no teacher had every analyzed it and it's readily available to us in full. Yet we send kids off to take tests we can't see or analyze and assume that the results are valid or worthy. It's made very clear to us during state testing (which I know is different than the topic here) that we can lose our licenses for reading questions on the kids' screens, making note of the type of questions being asked, or discussing the material on the measures. It defies logic, in my opinion.

Note that I am a consultant for Pearson's aimsweb product. I'd like to think it's because I know something about assessment, including basic skill progress monitoring and screening. Note that this question is about NWF. I have had long standing concerns about it largely because like other tests in Campbell's law, it is subject to "corruption pressure." That is, what gets measured and what takes on importance has a tendency to get corrupted." With respect to NWF, too often I've seen flashcard drill with nonsense words--in the absence of phonic instruction--and that bothers me. They get treated as another sort of sight word. Additionally, I've found that Letter Sounds is correlated with other reading measures as well or better than NWF--and in the absence of phonics instruction teachers teach Letter Sounds, I'm not as Uncomfortable with that corruption.

My hypothesis about declines in NWF are due to the fact that there is little utility in repeatedly administering NWF to Grade 1 students. It is plausible that NWF ceilings out--and students should obtain fairly high NWF scores in the FALL of Grade 1. Me, at the least, I'd like them to be reading authentic highly decodable text pretty well if they've had any reading instruction in K. But IF I use NWF, it is as a Fall Grade 1 screener. That's it. I wouldn't repeat it G1 w and certainly not G1 Spring. Shouldn't students be showing growth--and dramatic growth in reading "real words" in connected text. Another plausible hypothesis is motivation. I would hope that gets might be more interested in reading pseudo words when they can't read real words and when they can read real words, what's the point of reading nonsense words.

One of my pet peeves--and especially with young students--is that too many educators think a screener needs to be repeated as a progress monitoring test. We know the most about a simple, short, scientifically sound measure of oral reading. THAT should be assessed regularly--but not frequently--for typically developing kids, and more frequently for students who are receiving intensive intervention. But like Letter Names, PSF--which I find more useful as a longer diagnostic test than as a screener or PM test--I don't recommend using NWF as a progress monitoring test. Falling in love with 3 times per year without thinking only diverts time away from instruction and inadvertently turns some educators off on data.

Nonsense word fluency assesses a student's use of letter-sound sequencing when decoding words. The most obvious reason for a drop in scores on AIMSweb's Nonsense Word Fluency (NWF) is that students are likely processing printed text differently at the end of the year than they did at the beginning of the year. On AIMSweb's NWF, students can earn one point for each correct letter-sound they recall in a CVC nonsense word (e.g., three points if all correct) or can earn all three points for correctly pronouncing the whole nonsense word. Struggling readers start out the year by pronouncing each letter-sound. As the year progresses, they sound out the sequence of letters, and later (often with a bit of a pause), slide the sounds together to correctly pronounce the word. Later, when presented with a NWF word, they pause briefly as they process the information and then correctly pronounce the word. This difference in processing can result in declining, flat, or bumpy data slopes, such as described in the teacher's question to this site. The NWF graphs of more successful readers often show a more linear progression, but will still often include a dip at the point in time when the student starts to process the letter-sequences as whole words. When administered on a weekly basis, this assessment can provide a teacher with information regarding this qualitative change in the way a student processes print. Frequent progress monitoring provides teachers with information needed to design timely and effective interventions for a struggling reader. Still, the interpretation and critique of AIMSweb/NWF data requires an understanding of the measure and of how the processing of print information changes in an early reader. Do not expect to see a linear progression of scores for all students on this particular AIMSweb measure. However, the resulting data graph is a rich source of information regarding qualitative changes in student's development of decoding skills.
DIBELS also has a NWF measure, but has separate scoring for sounding-out letters (Correct Letter Sound/CLS) and for decoding the entire word (Whole Word Read/WWR). However, students do not receive a point on WWR if they had initially sounded out aloud the letters of the word (e.g., "N-I-F, NIF"). This has always seemed odd to me in that students are often taught to sound-out unfamiliar words before attempting to pronounce them. Separating WWR from CLS scores in DIBELS/NWF avoids confounding data regarding these two different ways of processing print.

The reason why your first grade students don’t do as well on the NWF is that by the time you do the spring testing they have the learned the long vowel sounds so now they use that skill. I would see this every year with our students so we would stop giving NWF to first graders in the spring .