Showing posts with label Lexiles. Show all posts
Showing posts with label Lexiles. Show all posts

Sunday, October 18, 2015

To Lexile or Not to Lexile, That is the Question

Teacher question: Our school district is going wild over Lexiles because they are in the Common Core standards. I think they are overdoing it and don’t feel comfortable with some of the decisions that we are making. What are the weaknesses of Lexiles?

First, Lexiles is only one of several readability measures included in the CCSS. They started with that one, but as studies were completed they added ATOS, SourceRater, and several others.

Everyone has to remember that Lexiles (and any readability measure) is a prediction of difficulty, and there since it is a prediction there will be a certain amount of error in it. It will sometimes overestimate or underestimate the difficulty of a text. It does this because it predicts difficulty on the basis of only two variables (word difficulty and sentence difficulty).

Obviously there is more to text difficulty than that. Nevertheless, the predictions tend to be reasonably accurate. Why?  Mainly because of the consistency of authors. If an author uses simple words and sentences, he/she will probably organize their writing in straightforward ways, and the cohesive structure, tone, and so on will probably not be particularly nuanced or complex.

But that isn’t always the case. Hemingway tended to use an Anglo-Saxon vocabulary, and are there any shorter sentences in the English language?, but try to keep track of who is speaking across pages of dialogue or to grasp what the characters are feeling just on the basis of the words themselves… Good luck, 5th graders.

As I say, Lexiles can mispredict. 

The appendices to the Common Core recommend some good ways of looking at text to adjust their placements up or down a bit. Thus, Lexiles (and the other estimates) can get you close, but then you need to use some judgment. No matter what Lexiles predicts, what do you think about using this text with a bunch of kids? (And remember, readability is only one part of the text selection equation—having kids read about sex or violence or racism, etc. in school may be just as problematic if the texts are easy or difficult).

Another reason the predictions aren’t perfect has to do with the reader. The idea of Lexiles and the other formulas is that we are trying to predict readers’ comprehension, and there can be reasons from a reader’s side of the equation why a text may turn out to be easy or hard. Let’s face it, if the author and I share a lot of knowledge in common, I’ll be able to bridge the gaps that he/she leaves for me. However, If the reader has less of a grasp of the content than the author assumed, then the sledding will be a lot tougher. (Yes, it doesn’t matter if there isn’t much shared knowledge if the writer doesn’t presume such information in the writing.)  

That means when you are selecting materials you have to think about what the kids might know and whether this text addresses the topic appropriately or not. Again, separate from the complexity of language: does the text over-explain something kids will already know (boring) or under-explain new topics in clear language leaving the kids confused?

Another thing to understand is that a readability score for a text is just an average. The average will be more accurate the longer the text is (more data, greater reliability). However, many teachers and publishers will estimate the difficulty of a text, but then will have the kids read a particular chapter from that text. Different sections of a text may vary quite a bit (so the overall difficulty for a text may be 5th grade, but the chapter you are actually teaching is 3rd grade or 8th grade difficulty).

It might be a good idea to run Lexiles on the actual excerpts and not to trust that the excerpt is a good representation of the overall text.

Readability measures can be very useful predictors of difficulty, but they do not help one to write or rewrite texts for particular audiences. For example, someone might select a text that they want to use, then they find out that according to Lexiles, the text is too easy or hard for the intended purpose. What to do? It is not uncommon that teachers or publishers adjust the passage, perhaps by replacing some words or breaking up a few sentences, etc. That will change the score (making the text appear to be more suitable), but it rarely improves the situation. It should be easy to not do this one yourself, but keeping publishers from playing such games is a bigger challenge.

Like your district, I’m a Lexile fan, but that doesn’t mean that we should misuse or abuse Lexiles. It is just a tool, and one that can solve problems or create problems. Let's not create them.

Friday, December 27, 2013

How Publishers Can Screw Up the Common Core

Lexiles and other readability measures are criticized these days about as much as Congress. But unlike Congress they don’t deserve it.

Everyone knows Grapes of Wrath is harder to read than predicted. But for every book with a hinky readability score many others are placed just right.

These formulas certainly are not perfect, but they are easy to use and they make more accurate guesses than we can without them.

So what’s the problem?

Readability measures do a great job of predicting reading comprehension, but they provide lousy writing guidance.

Let’s say that you have a text that comes out harder than you’d hoped. You wanted it for fourth-grade, but the Lexiles say it’s better for grade 5.

Easy to fix, right? Just divide a few sentences in two to reduce average sentence length, and swap out a few of the harder words for easier synonyms, and voila, the Lexiles will be just what you’d hoped for.

But research shows this kind of mechanical “adjusting” doesn’t actually change the difficulty of the text (though it does mess up the accuracy of the readability rating). This kind of “fix” won’t make the text easier for your fourth-graders, but the grade that you put on the book will be just right. Would you rather feel good or look good?

With all of the new emphasis on readability levels in Common Core, I fear that test and textbook publishers are going to make sure that their measurements are terrific, even if their texts are not.

What should happen when a text turns out to be harder or easier than intended, is that the material should be assigned to another grade level or it should really be revised. Real revisions make more than make mechanical adjustments. Such rewrites engage in the author in trying to improve the text’s clarity.

Such fixes aren’t likely to happen much with reading textbooks, because they tend to be anthologies of texts already published elsewhere. E.B. White and Roald Dahl won’t be approving revisions of their stuff anytime soon, nor will many of the living and breathing authors whose books are anthologized.

But instructional materials and assessment passages that are written—not selected—specifically to teach or test literacy skills are another thing altogether. Don’t be surprised if many of those kinds of materials turn out to be harder or easier than you thought they’d be.

There is no sure way to protect against fitting texts to readability formulas. Sometimes mechanical revisions are pretty choppy, and you might catch that. But generally you can’t tell if a text has been manipulated to come out right. The publishers themselves may not know, since such texts are often written to spec by independent contractors.

Readability formulas are a valuable tool in text selection texts, but they only index text difficulty, they don’t actually measure it (that is, they do not reveal why a text may be hard to understand). Qualitative review of texts and continuous monitoring how well students do with texts in the classroom are important tools for keeping the publishing companies honest on this one. Buyer beware.

Monday, September 10, 2012

CCSS Allows More than Lexiles

When I was working on my doctorate, I had to conduct a historical study for one of my classes. I went to the Library of Congress and calculated readabilities for books that had been used to teach reading in the U.S. (or in the colonies that became the U.S.). I started with the Protestant Tutor and the New England Primer, the first books used for reading instruction here. From there I examined Webster’s Blue-Backed Speller and its related volumes and the early editions of McGuffey’s Readers.

Though the authors of those have left no record of how those books were created, it is evident that they had sound intuitions as to what makes text challenging. Even in the relatively brief single volume Tutor and Primer, the materials got progressively more difficult from beginning to end. These earliest books ramped up in difficulty very quickly (you read the alphabet on one page, simple syllables on the next, which was followed by a relatively easy read, but then challenge levels would jump markedly).

By the time we get to the three-volume Webster, the readability levels adjust more slowly from book to book with the speller (the first volume) being by far the easiest, and the final book (packed with political speeches and the like) being all but unreadable (kind of like political speeches today).

By the 1920s, psychologists began searching for measurement tools that would allow them to describe the readability or comprehensibility of texts. In other words, they wanted to turn these intelligent intuitions about text difficulty into tools that anyone could use. That work has proceeded by fits and starts over the past century, and has resulted in the development of a plethora of readability measurements.

Readability research has usually focused on the reading comprehension outcome. Thus, they have readers do something with a bunch of texts (e.g., answer questions, do maze/cloze tasks) and then they try to predict these performance levels by counting easy to measure characteristics of the texts (words and sentences). The idea is to use easily measured or counted text features and to then place the texts on a scale from easy to hard that agrees with how readers did with the texts.

Educators stretched this idea of readability to one of learnability. Instead of trying to predict how well readers would understand a text, educators wanted to use readability to predict how well students would learn from such texts. Thus, the idea of “instructional level”: if you teach students with books that appropriately matched their reading levels, the idea was that students would learn more. If you placed them in materials that were relatively easier or harder, there would be less learning. This theory has not held up very well when empirically tested. Students seem to be able to learn from a pretty wide range of text difficulties, depending on the amount of teacher support.

The Common Core State Standards (CCSS) did not buy into the instructional level idea. Instead of accepting the claim that students needed to be taught at “their levels,” the CCSS recognizes that students will never reach the needed levels by the end of high school unless harder texts were used for teaching; not only harder in terms of students’ instructional levels, but harder also in terms of which texts are assigned to which grade levels. Thus, for Grades 2-12, CCSS assigned higher Lexile levels to each grade than in the past (the so-called stretch bands).

Lexiles is a recent schemes for measuring readability. Initially, it was the only readability measure accepted by the Common Core. That is no longer the case. CCSS now provides text guidance for how to match books to grade level using several formulas. This change does not take us back to using easier texts for each grade level. Nor does it back down from encouraging teachers to work with students at levels higher than their so-called instructional levels. It does mean that it will be easier for schools to identify appropriate texts using and of six different approaches—many of which are already widely used by schools.

Of course, there are many other schemes that could have been included by CCSS (there are at least a couple of hundred readability formulas). Why aren’t they included? Will they be going forward?

From looking at what was included, it appears to me that CCSS omitted two kinds of measures. First, they omitted those schemes that have not used often (few publishers still use Dale-Chall or the Fry Graph to specify text difficulties, so there would be little benefit in connecting them to the CCSS plan). Second, they omitted widely used measures that were not derived from empirical study (Reading Recovery levels, Fountas & Pinnell levels, etc.). Such levels are not necessarily wrong—remember educators have intuitively identified text challenge levels for hundreds of years.

These schemes are especially interesting for the earliest reading levels (CCSS provides no guidance for K and 1). For the time being, it makes sense to continue to use such approaches for sorting out the difficulty of beginning reading texts, but then to switch to approaches that have been tested empirically in grades 2 through 12. [There is very interesting research underway on beginning reading texts involving Freddie Hiebert and the Lexile people. Perhaps in the not-too-distant future we will have stronger sources of information on beginning texts].    

Here is the new chart for identifying text difficulties for different grade levels:

Core Band

Degrees of
2nd 3rd
2.75 5.14
42 54
4th 5th
4.97 7.03
52 60
6th 8th
7.00 9.98
57 67
9th 10th
9.67 12.01
62 72
11th CCR
11.20 14.10
67 74

Core Band


The Lexile
2nd 3rd
1.98 5.34
420 820
4th 5th
4.51 7.73
740 1010
6th 8th
6.51 10.34
925 1185
9th 10th
8.32 12.12
1050 1335
11th CCR
10.34 14.2
1185 1385

Core Band


2nd 3rd
3.53 6.13
0.05 2.48
4th 5th
5.42 7.92
0.84 5.75
6th 8th
7.04 9.57
4.11 10.66
9th 10th
8.41 10.81
9.02 13.93
11th CCR
9.57 12.00
12.30 14.50

For more information: