AI and Reading Comprehension

Teacher question: I am writing with a question about AI and reading comprehension instruction. Our school recently circulated an AI research framework indicating levels of permissible AI use in students’ research tasks. Level 0 prohibits generative AI entirely. Level 1 allows students to use AI to translate or simplify texts. Level 2 allows AI to locate sources. Level 3 allows AI-generated explanations as sources of information. Level 4 allows students to use AI to summarize or synthesize multiple sources. Level 5 involves analyzing AI output itself as the subject of research. The policy states that AI cannot replace students’ own thinking.

I wondered whether any aspects of it might raise concerns from the perspective of how students develop comprehension through reading. Allowing AI to provide explanations of sources or to summarize and synthesize multiple texts seems as though it could bypass some of the processes that develop comprehension, such as grappling with complex texts, comparing sources, resolving inconsistencies, and constructing one's own synthesis of information.

From a reading comprehension standpoint, do you see any aspects of this kind of AI research framework as potentially concerning?

Shanahan responds:

I’ve been receiving different versions of this question for about three years. This one is particularly articulate; I’ve only included an excerpt of it. The author herself provided a good answer to her own question.

Why haven’t I responded earlier to these queries?

To tell the truth, I didn’t know how to answer. I knew of some AI tutoring schemes that seemed potentially hopeful, but that was about it.

As this query points out, AI may undermine learning as much as it could help it along. This danger seems especially pertinent to the development of reading comprehension.

Accordingly, this two-part blog/podcast entry will address some of those concerns.

It makes sense to start with my definition of reading comprehension. It is the ability to make sense of information conveyed in written language – the ability to negotiate the affordances and barriers of text.

This is an “active reader” kind of definition. It includes understanding, inferring, judging, interpreting, relating, and remembering – all actions that readers may have to implement to make sense of a text.

But the definition also emphasizes the social aspects of this sense making. Texts are not natural phenomena; they are written by or for somebody who has a communicative purpose. As such, texts may include definitions, descriptions, graphic elements, explanations, examples, analogies, repetitions, and so on, all aimed at helping the imaginary readers (those readers the author imagined would be reading the text) to get the point. Those are the kinds of linguistic and conceptual features that act as affordances. They are efforts an author makes to helping the readers to grasp the message.

Of course, authors differ in how well they do this imagining or in how effectively they address their readers’ communicative needs. However, even if they were to do these things to perfection, there’s always the chance that some unexpected reader may come along who will be unable to make use of some of these affordances. They might even be perplexed by them. Perhaps the author’s diction excludes somebody, a metaphor miscarries, or a reader simply doesn’t know what to do with a complicated graphic element. When that happens, these conceptual and linguistic elements themselves may become barriers to understanding.

Readers, to comprehend, must take advantage of the affordances to a sufficient degree and surmount enough of the barriers to make sense of a text.      

Learning to read means learning to do this with a wide variety of texts; texts that vary in content, style, purpose, structure, genre, language features, and, yes, difficulty.

Since the 1940s, teachers have been told that kids learned reading best when taught with texts at “their reading levels.” Nevertheless, research has overwhelmingly rejected that idea (Shanahan, 2025). Kids make greater gains when they get a chance to try to make sense of texts they can’t yet read reasonably well. Such texts give them the opportunity to grapple with and figure out some of those affordances and barriers.

Given that starting point, the idea of having AI “summarizing or synthesizing” texts for students seems like a really bad idea.

I find exercise to be tiring, sweaty, and often boring. Whether I’m running, swimming, bicycling, or lifting weights, I’ve often fantasized about hiring someone to do those things for me. That way I could easily exercise 3-4 times a week and still do everything else I want to do.

Now, Cyndie, my wife, is a bit of killjoy. She’s been downright discouraging about my hiring idea. She is steadfast in her belief that I will have to do my own exercise if I’m to benefit. The person who does the exercise is the one with stronger bones and muscles and clearer lungs.

It’s the same when it comes to learning to read. No one else can do that for you – not even machines that have consumed hundreds of billions of words.

These days one of my big concerns about AI has to do with its use to render texts understandable for readers. Go online and you’ll fine scads of sites that claim they can improve kids’ reading achievement by matching them to texts using AI. Also, many teachers are having AI translate the texts in their curricula to the kids’ supposed reading levels. (These schemes seem to meet your school districts’ Level 1 criteria, seemingly meaning that it is a rather limited intrusion of AI).

I’m concerned about such approaches.

First, I’m unsure whether AI can even make texts easier to understand. There are only a handful of studies, and they suffer from mixed results and inadequate analysis. For the most part, the research shows that AI can alter texts in ways that lowers their grade level readability estimates – but it isn’t clear whether they make the texts easier to comprehend (Abreu, et al., 2024; Nasra, 2025; Picton, et al., 2025; Zou, et al., 2026). 

I’m not at all surprised that they can transform a 1000L text (suitable for middle school readers) into a 500L one that would appear to be appropriate for the primary grades. No question about it, breaking sentences down and replacing some vocabulary words can make a text look easier.

The issue is whether shortening a few sentences and swapping out some vocabulary improves anybody’s comprehension. Over the years, studies have usually said, “no.” That kind of revision rarely works (e.g., Mac, et al., 2025), and there are even studies in which the researchers have revised texts in ways that made them score harder on readability, and, yet were more comprehensible to children when tried out.

Despite their value for predicting how well kids will comprehend text, readability schemes have been lousy guides for text revision.

I asked ChatGPT to revise a page of Little House on the Prairie. It supposedly translated this 5^th-6^th grade appropriate text into one that would be readable by 3^rd-4^th graders. For instance, look at this change:

Original:

“They drove away and left it lonely and empty in the clearing among the big trees, and they never saw that little house again.”

Revision:

“They drove away and never came back.”

The revised sentence seems easier. But it fails to convey the same information, a problem that has plagued some efforts to use AI to produce readable health documents. If you don’t believe me, let’s ask AI.

ChatGPT suggested some comprehension questions to ask about those two sentences. It provided several questions for each, but many of them couldn’t be answered with the information in those sentences (e.g., Where were they going? Why did they have to leave?) or weren’t probing comprehension (e.g., what kind of sentence is it?). I deleted those, and this is what was left:

Questions for Original Sentence	Questions for Revised Sentence
What did they leave behind? Where was the house located? Did they ever come back to the house? What does “clearing” mean in this sentence? What does “lonely and empty” tell us about the house? How do you think the family felt when they left the house? Why?	What does “drove away” mean? Did they ever come back?

Obviously, those sentences aren’t equivalent. The original appears to be more difficult. It poses greater linguistic and conceptual challenges to readers.

The revision might be more easy for kids to understand. But avoiding those challenges has no possibility of helping students to become better comprehenders. The original text is the one that offers a possibility for teaching reading comprehension.

This entry explains why you shouldn’t use AI – or most other systems – for rewriting text to meet a desired level of difficulty. At the end of this, I have included the entire text revision done for me by ChatGPT. A careful examination of it will show revisions both apt and ham handed, and I have no doubt that over time and with more human guidance than I provided, that AI could produce much better revisions (Shel, et al., 2025). However, no matter how accurate those tools and processes may become, they’ll always miss the point. If you are trying to teach kids to read, dumbing down the text in those ways will always reduce their opportunity to learn.

Using AI to revise or produce texts of certain levels of difficulty or using it to summarize and explain texts are ways teachers can avoid teaching reading comprehension, not scaffolds likely to make kids into better readers.

Our next entry will explore how AI could help literacy teachers to improve students’ comprehension.

References

Abreu, A. A., Murimwa, G. Z., Farah, E., Stewart, J. W., Zhang, L., Rodriguez, J., Sweetenham, J., Zeh, H. J., Wang, S. C., & Polanco, P. M. (2024). Enhancing readability of online patient-facing content: The role of AI chatbots in improving cancer information accessibility. JNCCN.org, 22, 1-8.

Beck, I. L., McKeown, M. G., & Gromoll, E. W. (1989). Learning from social studies texts. Cognition and Instruction, 6(2), 99–158. http://www.jstor.org/stable/3233499

Mac, O., Ayre, J., McCaffery, K., Boroumand, F., Bell, K., & Muscat, D. M. (2025). The readability study: A randomised trial of health information written at different grade reading levels. Journal of General Internal Medicine, 40(8), 1820–1828. https://doi.org/10.1007/s11606-024-09200-z

Nasra, M., Jaffri, R., Pavlin-Premrl, D., Kok, H.K., Khabaza, A., Barras, C., Slater, L.-A., Yazdabadi, A., Moore, J., Russell, J., Smith, P., Chandra, R.V., Brooks, M., Jhamb, A., Chong, W., Maingard, J. and Asadi, H. (2025), Can artificial intelligence improve patient educational material readability? A systematic review and narrative synthesis. Internal Medicine Journal, 55, 20-34. https://doi.org/10.1111/imj.16607

Picton, B., Andalib, S., Spina, A., Camp, B., Solomon, S. S., Liang, J., Chen, P. M., Chen, J. W., Hsu, F. P., & Oh, M. Y. (2025). Assessing AI simplification of medical texts: Readability and content fidelity. International Journal of Medical Informatics,195,105743.

https://doi.org/10.1016/j.ijmedinf.2024.105743.

Shen, L., Kane-Cabello, J., Candelaria, P. Y., Stratford, D., & Clemens, N. H. (2025). Can artificial intelligence tools generate text that is useful for reading practice? Learning Disabilities Research & Practice, 40(4), 191-204. https://doi.org/10.1177/09388982251352564

Zou, Y., Kuek, F., Ng, K. H., & Cheng, X. (2026). Comparative analysis of text readability and writing styles in AI-generated vs. Human-written academic abstracts. PloS One, 21(4), e0343163. https://doi.org/10.1371/journal.pone.0343163

 

Original Passage	Revision
A time ago, when all the grandfathers and grandmothers of today were little boys and little girls or very small babies, or perhaps not even born, Pa and Ma and Mary and Laura and Baby Carrie left their little house in the Big Woods of Wisconsin.	A long time ago, when today’s grandparents were still children or not even born yet, a family named Pa, Ma, Mary, Laura, and baby Carrie lived in a small house in the Big Woods of Wisconsin.
They drove away and left it lonely and empty in the clearing among the big trees, and they never saw that little house again.	One day, they decided to leave their home. They drove away and never came back.
They were going to the Indian country.	They were going to a place called Indian country.
Pa said there were too many people in the Big Woods now.	Pa said there were too many people living in the woods now.
Quite often Laura heard the ringing thud of an ax which was not Pa’s ax, or the echo of a shot that did not come from his gun.	Laura often heard axes chopping trees that did not belong to Pa, and gunshots from other people.
The path that went by the little house had become a road.	The small path near their house had turned into a busy road.
Almost every day Laura and Mary stopped their playing and stared in surprise at a wagon slowly creaking by on that road.	Almost every day, wagons passed by, and Laura and Mary would stop playing to watch them.
Wild animals would not stay in a country where there were so many people.	Because so many people were moving in, the wild animals were leaving
Pa did not like to stay, either.	Pa didn’t like that.
He liked a country where the wild animals lived without being afraid.	He preferred living in a place where animals were not afraid.
He liked to see the little fawns and their mothers looking at him from the shadowy woods, and the fat, lazy bears eating berries in the wild-berry patches.	He enjoyed seeing deer and bears living freely in the woods.
In the long winter evenings he talked to Ma about the Western country.	During the long winter, Pa talked to Ma about moving west.
In the West the land was level, and there were no trees. The grass grew thick and high.	He said the land there was flat, with no trees, and covered in tall grass.
There the wild animals wandered and fed as though they were in a pasture that stretched much farther than a man could see, and there were no settlers. Only Indians lived there.	Wild animals lived there freely, and not many settlers were there yet—only Native Americans.
One day in the very last of the winter Pa said to Ma, “Seeing you don’t object, I’ve decided to go see the West.	At the end of winter, Pa told Ma he wanted to go west.
I’ve had an offer for this place, and we can sell it now for as much as we’re ever likely to get, enough to give us a start in a new country.”	He said they could sell their house and use the money to start a new life.
“Oh, Charles, must we go now?” Ma said.	Ma was unsure. She said, “Do we have to go now?”
The weather was so cold and the snug house was so comfortable.	The weather was very cold, and their home felt warm and comfortable.