Which Should We Use, Nonsense Word Tests or Word ID Tests?

  • nonsense words
  • 26 August, 2023
  • 29 Comments

Teacher question:

I am an Assistant School Superintendent. We are moving toward explicit phonics instruction this year and are debating between using the nonsense words assessment or the decodable words assessment. Do you have thoughts about this? I have consulted with several people who I respect, and opinions are varied and passionate. 

RELATED: Is digital text a good idea for reading instruction?

Shanahan response:

I feel your pain.

Recently, a colleague asked me to make a similar recommendation to help figure out something about a grandchild’s reading. I suggested the use of DIBELS Nonsense Word test, given the specific purpose and its easy availability.

You’d have thought I’d recommended drowning kittens or banning the Barbie movie!

People do get passionate about the strangest things.

I try to save my passion for non-empirical questions (Go Cubs, go!). If we have data that will allow us to make a sound determination, I’d turn the heat down and try to follow the numbers. Remember this is about trying to do what’s best for kids. It is not an opportunity to vent your spleen or espouse your philosophy.

There are two different kinds of tests used to determine student progress in decoding. Both kinds have a proven ability to evaluate how well students are learning their phonics and both can predict later success with oral/text reading fluency and reading comprehension.

Word identification tests have been around for a long time – more than 100 years. Nonsense word or pseudoword tests are a newer development.

Researchers were concerned about the validity of word identification tests for determining the effectiveness of decoding instruction. Word identification tests often focus on irregular spellings (e.g., the, of, done), the kinds of words that are inconsistent with the spelling patterns usually stressed in phonics. Such tests couldn’t tell much about the effectiveness of phonics instruction. Even word tests with more common spellings were suspect. With such tests it was impossible to know if a student decoded a word or just remembered it from previous exposures.

The solution to the problem was the creation of nonsense word or pseudoword tests. Because the researcher (and, later, the test designer) constructs the words by mimicking English spelling patterns, there are no exceptional spellings, one offs, accidents of morphological history, and the like. Whether teachers are leading the kids to memorize Dolch or Fry list words or are just providing them with repeated exposure to certain words through phonics instruction, it was certain that the students wouldn’t have previously seen before letter combinations like dop, lan, or sepe. 

The idea was that a nonsense word measure would provide a purer look at how well students can decode, and their performance on such a test should reveal their decoding progress.

As is often the case, scientists often may identify a real problem, but solving it may not be so easy.  

At first blush, the nonsense test appeared to do a terrific job of assessing decoding ability, perhaps, more valid than the traditional word identification test.

Over time, their faults became evident.

Often, if teachers know that their students are to be evaluated with nonsense words, they start teaching them to the students. This teaching is a waste of time for producing readers and renders useless the intended improvement in test design. Researchers and school district administrators must be vigilant in discouraging teachers from fraudulently enhancing their students’ test performance. (I don’t think most teachers are intentionally trying to defraud – they just want to make sure their kids do well on the test, and teaching the specific test items seems logically to be the most direct route to that outcome.) well meaning but unfortunate

A more important issue has to do with the nature of decoding. There is more to decoding than pronouncing letter patterns. Pseudoword tests provide a useful assessment of that part of the process, but not of the rest.

As Richard L. Venezky so aptly described the process:

“A third function of phonics is to generate a pronunciation for a word…. This

function is problematic, in that the imperfections in English orthography make

such generation uncertain. If a word is totally unknown, the reader has little basis

for deciding whether any particular pronunciation is correct or not (Venezky, 1999, p. 202)

Phonics is a tool for helping readers to decode the words in a text. But that is a necessarily imperfect process due to the complexity of the English spelling system. Some “experts” throw up their hands, ready to surrender; for them, phonics would be useless because of the complexity of our spelling system. But as Venezky points out, readers don’t need to arrive at exact pronunciations. Reasonable approximations are good enough, and then the readers make adjustments and consider alternatives based on their knowledge of the English language.

Nonsense tests, by their very design, can tell us whether students have managed to master particular spelling patterns, but they prevent students from any kind of self-evaluation and adjustment of pronunciation, key aspects of decoding. As such, these tests may do a good job of evaluating student learning from a decoding program, but they are unlikely to do equally well in predicting later reading achievement, as measured by oral reading tests, or reading comprehension tests.

What do the research studies have to say about the usefulness of these measures?

For the most part, word identification tests and nonsense word reading tests tend to be interchangeable early on. There are copious amounts of validation data showing the value of both (e.g., Fien, Baker, Smolkowski, Kame'enui, & Beck, 2008; Vanderwood, Linklater, & Healy, 2008). They both work reasonably well (i.e., there are high correlations between these measures and other reading tests).

However, in direct comparisons in which students are taking both tests so that they can be evaluated head-to-head, the word identification tests tend to do a bit better. For example, in one well-done study it was found that word ID tests provided a “clearer index of reading growth” (Clemens, Shapiro, Wu, Taylor, & Caskie, 2014). Early in first grade, the tests were indistinguishable, but by second semester the word identification tests inched ahead.

Similarly, in a very large study of first graders (n = 3,506, from 50 schools), it was reported that the Nonsense word Fluency tests did the best job of predicting end of year reading fluency and comprehension for most kids (Fien, Park, Baker, Smith, Stoolmiller, & Kame'enui, 2010). There are other studies of this with similar results (e.g, Fuchs, Fuchs, & Compton, 2004). However, this was not true for the higher achieving students. As kids’ reading advanced, leaving out those word identification skills that Venezky noted becomes a real problem.

By third grade the correlations between NSW and word ID separate to a greater degree with the real word performance becoming the best predictor of ORF for most kids (Doty, Hixson, Decker, Reynolds, & Drevon, 2015).

Finally, a recent meta-analysis of data show that across many studies, word ID tends to have the best relationship with various reading outcomes (January & Klingbeil, 2020).

None of these differences just noted are especially large, though they are often statistically significant. Nevertheless, some authorities suggest including both in early reading inventories, and that makes a certain kind of sense since they tap a slightly different array of skills.

I certainly have no problem with ongoing monitoring of decoding skills with nonsense words, alongside a word reading check to determine how well kids can read those most frequent words.  

If you are only going to give one, and your specific interest is monitoring phonics progress in grade K-2, I’d go for a real word reading test – especially second semester of grade 1 or later and with my highest achieving schools. Those tests should do a slightly better job of revealing student progress towards success in reading. Just make sure, given your purpose, that the word ID test that you choose includes many words with regular spelling patterns.

But remember the differences here aren’t large. In a different situation (e.g., I’m a school psychologist and a student has been referred to me due to a concern about his/her phonics ability), I would likely give you a different answer. You really can’t go too far wrong in this case.


READ MORE: Shanahan On Literacy Blog

References

Clemens, N. H., Shapiro, E. S., Wu, J., Taylor, A. B., & Caskie, G. L. (2014). Monitoring early first-grade reading progress: A comparison of two measures. Journal of Learning Disabilities, 47(3), 254-270. doi.org/10.1177/0022219412454455

Doty, S. J., Hixson, M. D., Decker, D. M., Reynolds, J. L., & Drevon, D. D. (2015). Reliability and validity of advanced phonics measures. Journal of Psychoeducational Assessment, 33(6), 503-521. doi.org/10.1177/0734282914567870

Fien, H., Baker, S. K., Smolkowski, K., Smith, J. L. M., Kame'enui, E. J., & Beck, C. T. (2008). Using nonsense word fluency to predict reading proficiency in kindergarten through second grade for English learners and native English speakers. School Psychology Review, 37(3), 391-408.

Fien, H., Park, Y., Baker, S. K., Smith, J. L. M., Stoolmiller, M., & Kame'enui, E. J. (2010). An examination of the relation of nonsense word fluency initial status and gains to reading outcomes for beginning readers. School Psychology Review, 39(4), 631-653.

Fuchs, L. S., Fuchs, D., & Compton, D. L. (2004). Monitoring early reading development in first grade: Word identification fluency versus nonsense word fluency. Exceptional Children, 71(1), 7-21. doi.org/10.1177/001440290407100101

January, S. A., & Klingbeil, D. A. (2020). Universal screening in grades K-2: A systematic review and meta-analysis of early reading curriculum-based measures. Journal of School Psychology, 82, 103-122. doi.org/10.1016/j.jsp.2020.08.007

Vanderwood, M. L., Linklater, D., & Healy, K. (2008). Predictive accuracy of nonsense word fluency for English language learners. School Psychology Review, 37(1), 5-17.

Venezky, R. L. (1999). The American way of spelling. New York: Guilford Press.

Comments

See what others have to say about this topic.

Tim Rasinski Aug 26, 2023 02:57 PM

Tim, I appreciate your thoughtful essay. BTW - thought you were a Tigers fan ????

Timothy Shanahan Aug 26, 2023 03:05 PM

Tim--

Thanks. My poor Tigers... they never seem to make a good trade! I got to see them this week and they have some fine young players, maybe the future is not so bleak.

I have both a National and American League favorite.

tim

Lauren Aug 26, 2023 03:14 PM

Many of these online tests are timed. The one we are using is one minute. I have many students who seem to have a pretty good general ability to decode, but they get a "way below" score due to the timing issue. Some students are just slow, careful thinkers, and the timing really throws them off. Can you comment on the element of "timing" in assessment in general. It is indicated that fast responses to the test indicate fluency. I'm not so sure....

Elizabeth Will Aug 26, 2023 03:18 PM

How biased and discriminatory are these 'tests' for students who speak variations of English? What does the research say about the validity of these tests for students who speak a variety of English that doesn't conform to the 'rules' of mainstream American English? Are we getting, with these nonsense tests or even word identification tests, a true picture of the language competence of students who speak a variety other than MAE?

Lauren Aug 26, 2023 03:31 PM

Sorry, one more question. We have some students reading above grade level or way above grade level, but they fail the phonics/nonsense word parts of these tests. Should we put them in a group to teach beginning phonics skills?

Tammy Aug 26, 2023 03:37 PM

Like Lauren, I am concerned with the time pressure of the ORF assessments. I see high levels of anxiety among students impacting their accuracy and even their willingness to stay with the task. I would love your thoughts.

Lisa Schreyer Aug 26, 2023 03:45 PM

Dr. Tim -Your timing is impeccable! I’ve spent this week questioning which screeners would be most informative & efficient to target interventions as a reading interventionist in gr. 1-3. I’ve trained staff in Dibels yet always unsure which results to give more weight to: NSW or WRF. Your essay this week provided the guidance I needed & alleviated my mounting anxiety. Would you say then, for 2nd & 3rd grade BOY benchmark start with ORF, analyze results and then determine if both NWF & WRF is needed? For example, if students score strategic (not quite meeting benchmark) progress monitoring WRF & ORF would be the more informative results to determine progresss? Only those scoring well below benchmark (below 20%?) administer the NWF as well? My concern would be those students require intensive intervention to close gaps in phonics knowledge. Also, would you recommend a spelling diagnostic (WTW) for these students to give me specific info on their decoding/encoding weaknesses?
Finally, Phillies. Phillies. Phillies. Go Phils!

Lisa

Danna Sermersheim Aug 26, 2023 03:45 PM

Hi, Tim. Thank you for the information regarding using nonsense words for assessment purposes. Could you tell me if there is current research that supports incorporating nonsense words in phonics instruction? If there have been studies showing that there is a benefit to using nonsense words as part of instruction or having students practice reading nonsense words to improve their decoding abilities, I would be very interested in reading it. Thank you!

Timothy Shanahan Aug 26, 2023 04:04 PM

Danna-
No, I am unaware of any such research.
however, teaching phonics with nonsense words is, well, nonsensical. It both invalidates these kinds of assessments and misses (and distracts kids from) the whole point of phonics -- to help kids to read (it even makes it impossible for readers to make the judgments -- is this a word? dies this make sense? -- that are essential to decoding.

tim

Timothy Shanahan Aug 26, 2023 04:11 PM

Lisa--
Indeed, in grades 2 and 3 it is ORF that is most informative and the best starting point. If students were able to handle 2nd and 3rd grade texts and I wanted more detail about their decoding, my preference would be a word ID test. And, if they were reading much below that I'd feel very comfortable with a nonsense word test. (I lived in the Philadelphia area and rooted heavily for the Phillies at that time.

tim

Timothy Shanahan Aug 26, 2023 04:15 PM

Lauren--
Just because the test is timed doesn't mean the kids need to know that or that they should be encouraged. They should be encouraged to read the text as well as they can. Second, studies suggest that 3 minute reads provide a better assessment (DIBELS tries to address this by having kids do two 1-minute reads) and combining the results. The longer reading time may reduce the attention to time.

tim

Timothy Shanahan Aug 26, 2023 04:20 PM

Elizabeth--
You are not finding out anything about the language competence of anyone with these tests. You are finding out how well the students can decode English words. That is important to know with both first and second language students (that's why studies repeatedly show phonics instruction to be beneficial to English Learners). There definitely are other items to be evaluated for all students.

tim

Grace Vyduna-Haskins Aug 26, 2023 05:34 PM

In assessments I have chosen to use many less familiar words such as zap, yam, vim, quill, buff, buff, bog, puck, yen, dusk, yelp, sill, etc., thinking that they are legitimate measures of phonics skills while, at the same time, introducing new vocabulary through their use in sentences that accompany the assessments.

Andrea Aug 26, 2023 05:38 PM

I could have used this for my DIBELS presentation at school, this week. Now, it's time to dive deeper into the research! Thanks! And-- Go Cardinals! *wishful thinking for this season*

Jenny Chew Aug 26, 2023 07:30 PM

In England, we have had a national Phonics Screening Check since 2012 which is compulsory for all children at the end of their second year of school - the year in which they turn 6. It consists of 20 pseudowords and 20 fairly low-frequency regular real words. Children need to score at least 32 out of 40 or else take the test again a year later. In 2012, only 58% met the standard, but in the last 3 years before the pandemic (2017-19), the figure was up to 82%. There was no testing in 2020 and 2021 because of the pandemic, and results dropped back to 75% in 2022. The 2023 test materials can be seen here but results have not yet been released:

ww.gov.uk/government/publications/phonics-screening-check-2023-materials

Results in earlier years have been shown to correlated positively with performance in the PIRLS international reading tests done in 2016 and 2021..


Michele P. Aug 26, 2023 08:51 PM

Tim, What are your thoughts on VC and CV instructional scaffolding to help struggling readers blend 2 sounds before jumping all the way to 3 sounds? I have found some success with some students but do not know what any research says about this practice with regard to brain processing, memory, or language/phonemic understanding.
—Michele P.

Harriett Janetos Aug 26, 2023 09:00 PM

"teaching phonics with nonsense words is, well, nonsensical."

Well-yes and no. At what point does an unknown word become a nonsense word? There's a story in one of my first grade decodables that has the words 'fig' and 'jig'--neither of which has meaning for my students until I provide an explanation. But they can decode both. I'm glad Jenny mentioned the UK Phonics Screening Check because it's brilliant. Their nonsense words are monster names next to pictures of the monsters (think Shrek), which makes this assessment more meaningful for students since names are real. I use these monster names to assess my intervention students, and I've never seen them smile as much as they do during this assessment. As for baseball, I regret to announce that my beloved San Francisco Giants have been on a summer-long skid and have lost ground to everyone, including the Phillies.

John Young Aug 26, 2023 09:11 PM

“But as Venezky points out, readers don’t need to arrive at exact pronunciations. Reasonable approximations are good enough, and then the readers make adjustments and consider alternatives based on their knowledge of the English language.”

With regard to the above statement when applied in reading text is that implying that when unsure of a word children use their knowledge of the English language and in text reading meaning to identify the word. What Clay described as cross checking. That while instruction should not be based teaching three cueing it is not to deny that developing readers to engage in some form of this process.
This has has often perplexed me as I made the journey from a whole language, ex Reading Recovery teacher, to believing that the teaching synthetic phonics must have primacy in the initial teaching of reading.

Dr. Bill Conrad Aug 26, 2023 09:50 PM

The tragedy that teachers readily engage in the instruction of nonsense syllables underlies the poor training teachers get in both reading and assessment within the woeful colleges of education.

It is beyond imagination. Please stop making apologies for the the incompetence of teachers and demand better preparation and accountability.

And we wonder why almost 50 million 4th graders read below proficient on NAEP over 20 years? Enough is enough. The unprofessional K-12 education system is in need of major transformation.

Read The Fog of Education.

Go Go Sox!

Timothy Shanahan Aug 26, 2023 10:12 PM

John-

With 3-cueing kids are taught to use meaning (including pictures), syntax, and orthographic-phonological information to figure out a word -- in any order. If they use the letters and come up with a pronunciation that they don't know or that makes no sense, they are to turn to the other cues -- rather than trying alternative pronunciations based on the spelling. Indeed, kids do what Venezky suggested but not at all in the way that 3-cueing encourages.

tim

Lorri Beveridge Aug 27, 2023 01:42 AM

Important background knowledge for educators as systems adopt phonics screening tests Year 1 en mass, with nonsense words included. It was good to hear the research behind this direction, part of the back to basics emphasis in early reading teaching.

Dana Aug 27, 2023 02:21 PM

The only reason I practice nonsense word reading is my ASD students, in particular, struggle with the reasoning behind reading words that are not real and often freeze during testing.

Harriett Janetos Aug 27, 2023 03:21 PM

"The only reason I practice nonsense word reading is my ASD students, in particular, struggle with the reasoning behind reading words that are not real and often freeze during testing."

This is why the monster name test in England is a stroke of genius. A name isn't just a nonsense word--it has meaning.

Zoe Ann Brown Aug 27, 2023 04:16 PM

Thank you for your research reviews and recommendations. Everything is helpful.
I’ve spent my career of 47 years as an educator, developing curriculum and teaching English reading for/to English language language learners. Developed a couple of programs that work very well for younger kids. Now I’ve come out of retirement to work with long-term English learners 12-18 yrs old . These are kids who were born here, but don’t have proficiency in either Spanish or English. This proficiency includes what’s normally taught in ESL, as well as vocabulary, text deconstruction, reading comp and fluency, phonemic awareness, phonics and spelling patterns, all of which have been identified to be where their holes are.
They have more understanding (can attach meaning) to spoken language but limited reading skills.

I think the issue is motivational more than or equally as much as language-based. These kids of been in school for so long with so many holes and they have fallen on so far behind for many reasons. they don’t believe they’re capable. They’ve turned - off their focus is poor, listening is poor, and their classroom behaviors are not well developed. They need so much help but getting thru to them with materials developed for the K-2 market is difficult.

Please share your insights. There are so many lost kids (mostly males) who on the borderline of contributing to society and delinquency. Thanks for your wisdom. Dr. Zoe

Timothy Shanahan Aug 28, 2023 01:55 AM

Rachel--

You are correct that Ebbinghaus did studies with nonsense syllables 140 years ago. However, that he wasn't studying reading, decoding, or assessment is pertinent. Ebbinghaus' studies are not relevant to what is being discussed here.

tim

Rachel Brown-Chidsey Aug 27, 2023 06:04 PM

I think that nonsense word (pseudo word) tasks have been used for well over 100 years as well. Dr. Hermann Ebbinghaus pioneered their use in the 1880s.

Miriam Trehearne Aug 27, 2023 08:34 PM

Miriam P. Trehearne

Tim, in your most recent blog (August 26) post the following points were raised:

Lauren: Many of these online tests are timed. The one we are using is one minute. I have many students who seem to have a pretty good general ability to decode, but they get a "way below" score due to the timing issue. Some students are just slow, careful thinkers, and the timing really throws them off. Can you comment on the element of "timing" in assessment in general. It is indicated that fast responses to the test indicate fluency. I'm not so sure....

Tim: Lauren, just because the test is timed doesn't mean the kids need to know that or that they should be encouraged. They should be encouraged to read the text as well as they can. Second, studies suggest that 3 minute reads provide a better assessment (DIBELS tries to address this by having kids do two 1-minute reads) and combining the results. The longer reading time may reduce the attention to time.

Miriam: It does not take young children long to figure out that they are being timed. And of course, the ones who are struggling the most, and who know they are, will be most bothered by the experience.

Tammy: Like Lauren, I am concerned with the time pressure of the ORF assessments. I see high levels of anxiety among students impacting their accuracy and even their willingness to stay with the task. I would love your thoughts.

Danna: Could you tell me if there is current research that supports incorporating nonsense words in phonics instruction?

Tim: Danna, no, I am unaware of any such research.
However, teaching phonics with nonsense words is, well, nonsensical….

Miriam: To me timing young children during reading assessments is nonsensical.
Like, teaching phonics with nonsense words, I am unaware of any research, pro or con.

My concerns, after working with teachers around the world regarding timed assessments with young children, centre around the following issues:
the social emotional toll the test can take on young children… e.g., children frequently breaking into tears because they were not going fast enough;
the takeaway for young children about what is important in reading: speed, not meaning and enjoyment;
timing penalizing thoughtful readers and those who self-correct which may result in incorrectly identifying students as needing intervention simply because they are not reading quickly enough

Tim you state: “Remember this is about trying to do what’s best for kids”.
I think that is what we are all trying to do. Your help, including the blog is so beneficial. You and Christopher Lonigan wrote a very insightful article: Developing Early Literacy Skills: Things We Know and Things We Know We Don’t Know. It was written in 2010. Is all of the information still valid or has research changed what we should know about developing early literacy skills in 2023?
I believe that I would benefit greatly from an update as would educators around the world.
Your thoughts Tim… Miriam Trehearne

Aurelia Nov 30, 2023 03:40 PM

Do you see any value in assessing non-sense words in Spanish? We are utilizing mCLASS Lectura as a universal screener and it is a component of the assessment.

Timothy Shanahan Nov 30, 2023 03:46 PM

Aurelia--
I do if you are trying to teach students to read Spanish and are interested in knowing how well they can decode. If you are not teaching Spanish reading or have no reason to evaluate their Spanish decoding abilities, I would skep that.

tim

What Are your thoughts?

Leave me a comment and I would like to have a discussion with you!

Comment *
Name*
Email*
Website
Comments

Which Should We Use, Nonsense Word Tests or Word ID Tests?

29 comments

One of the world’s premier literacy educators.

He studies reading and writing across all ages and abilities. Feel free to contact him.