Quality Assessments for Students with Suspected/Confirmed “APD”

Today I would like to answer several questions on the subject of assessment of students with APD by providing further helpful information and links for parents and professionals seeking evidence-based assistance for students with suspected/confirmed “APD”. These are:

1. How do we help students with “APD?
2. What constitutes a good quality assessment for a student with “APD”?

The first piece of advice comes from the great Alan Kamhi. “Do not assume that a child who has been diagnosed with APD needs to be treated any differently than children who have been diagnosed with language and learning disabilities” (Kamhi, 2011, p. 270). So when students with “APD” are referred for services, the first step is to ensure that they receive good quality assessments that tap into all their areas of deficits in a highly targeted manner for functional treatment purposes.

So what constitutes a good quality assessment? Well for starters the assessors need to use parent/teacher summaries of exhibited deficits or specifically tailored checklists in order to effectively pinpoint the students’ areas of difficulty. Evaluators can’t use whichever assessments are available at hand simply because they are in their possession to evaluate abilities. That is not evidence-based practice and it will frequently result in the under-identification of deficits. That is because common comprehensive tests such as CASL-2, OWLS-II, RESCA-E, CELF-5, etc., poorly identify language and literacy deficits secondary to having weak or unidentified discriminant accuracy (cannot distinguish between language/literacy impaired students and typically developing students). 

Discriminant accuracy refers to the sensitivity and specificity of assessment instruments (Dollaghan, 2007). Sensitivity ensures that the assessment accurately identifies those students who truly have a language/reading disorder as having a disorder. Specificity ensures that the assessment accurately identifies those students who truly do not have any disorders as typical. Sensitivity and specificity determine the test’s degree of discriminant accuracy, or the ability to distinguish the presence of a disorder  In 1994, Vance and Plante established criteria for discriminant accuracy or accurate identification of a disorder. 90% is considered good discriminant accuracy. 80% to 89% is considered fair, while below 80%, misidentifications occur at unacceptably high rates” and lead to “serious social consequences” of misidentified children. (p. 21)”  Discriminant accuracy constitutes the most important information about the assessment. If the test has low sensitivity and specificity or if that information is missing from the test manual; OTHER psychometric properties simply do not matter!

Currently, TILLS is the only comprehensive assessment with strong psychometric properties. However, it is also not without numerous limitations! While it frequently appropriately identifies students with language and literacy deficits, there’s definitely a subset of impaired students who will not be identified by the TILLS administration alone! Appropriate grade level supplemental testing will absolutely be required in order to correctly identify the student’s presenting deficits.

So what do I mean by appropriate? With respect to this post, I am referencing two particular things:

1. assessment tasks sensitive to language and literacy disorder identification, as well as,
2. ensuring that literacy abilities are sufficiently assessed in the context of the provided assessment.

There are a number of assessment tasks sensitive to language and literacy impairment identification. These include but are not limited to following directions, nonword repetition, phonemic awareness, orthographic knowledge, nonword reading, rapid naming, etc. In the absence of strong assessment comprehensive assessment instruments such as TILLS, evaluators can effectively use these tasks in order to assess the students’ language and literacy abilities. However, even with access to strong standardized tools, evaluators need to administer several clinical assessment tasks strongly associated with academic success. These are:

1. Narrative/discourse assessment (task selection is age dependant)
2. Pragmatic language assessment
3. Reading assessment
   1. Elementary
   2. Adolescent
4. Writing assessment
   1. Elementary
   2. Adolescent

Here’s why the above tasks should be a mandatory part of all comprehensive evaluations. Firstly, it is IMPOSSIBLE to separate language from literacy for assessment purposes. Language is literacy, and eventually, all language disorders turn into learning disabilities! Assessing language only leaves a huge gap with respect to understanding the student’s limitations in the areas of reading, spelling, and writing.

Now at this juncture, many SLPs will say that they have far too much on their plate to add literacy to it. While I completely agree and sympathize with the truly excessive caseloads burdened upon my colleagues, I must point out the obvious.

Refusal to assess and treat literacy along with language is the equivalent of a head in the sand scenario. What will merely happen is that our students will continue to make limited gains and languish longer on our already massively bloated caseloads. In turn, adding literacy goals to language goals will expedite their therapy gains and result in faster dismissal from therapy services.

At this juncture, a number of colleagues will point out that they will they will leave the literacy assessments to learning consultants, psychologists, neuropsychologists, etc. because they are supposedly better equipped to assess literacy. Here are some reasons why I have to respectfully disagree.

1. I have never seen the above professionals analyze assessment results with the same thoroughness as SLPs. Our training places us in a unique position of being able to analyze reading, spelling, and writing errors from the standpoints of phonology, morphology, orthography, semantics, pragmatics, etc.
2. Presently the vast majority of educational tests lack discriminant accuracy and is not suitable for disorder identiffication. Tests such as Woodcock-Johnson IV Tests of Achievement (WJ IV-ACH), Woodcock-Johnson IV Tests of Oral Language (WJ IV-OL), Wechsler Individual Achievement Test Fourth Edition (WIAT-4), Kaufman Test of Educational Achievement Third Edition (KTEA-3) were developed to rank children within the range of the general population. There’s no mention of sensitivity and specificity in their respective technical manuals, and their discriminant accuracy for the purpose of disorder identification is unknown. As a result many students can do quite well on these test and be reading, writing or oral language impaired. In contrast the Feifer Assessment of Reading (FAR) does have discriminant accuracy and while its specificity of .98 is excellent, it is not as relevant as its sensitivity of .67, which is unacceptable. Furthermore, the FAR addresses four specific subtypes of dyslexia: dysphonetic dyslexia, surface dyslexia, mixed dyslexia, and reading comprehension deficits. While, these subtypes are appealing because of a belief that they will guide treatment practices, poor readers can acutally present with a variety of reading difficulties.  Consequently, there’s poor research evidence to support them ( Zoubrinetzky, Bielle & Valdois, 2014)  The number of symptoms of dyslexia described in the literature exceeds the number of subtypes, and underlying relations remain unclear (Tamboer, Vorst, & Oort, 2016). “Current models of dyslexia indicate that single deficit models are inadequate to account for the variability found in dyslexia (e.g., Pennington et al., 2012; Perry et al., 2019)”.
3. SLPs are the only individuals who create functional goals for remediation purposes. Neuropsychologists and psychologists do not create goals for treatment purposes, although many learning specialists do see students for reading intervention in school setting.

Here’s another reason why it’s a really bad idea to separate language and literacy for testing purposes. After 4th grade, the rate of vocabulary growth is significantly related to 4th-grade word reading (Duff, Tomblin, & Catts, 2015). Translation – the knowledge gained by students now comes from books vs. teachers. It is simply not possible to attain the same amount of information orally from teachers as compared to the vast amount of knowledge one can learn from books, being a competent reader. So even as compared to the literate students with language disorders, language-impaired students with limited literacy skills will gain significantly less benefit from therapy services due to the Matthew Effect. What is the Matthew Effect? It is a condition in which early success in acquiring reading skills leads to later reading successes as the learner grows. In turn, failing to learn how to read before 3rd or 4th grade often creates lifelong problems in learning new skills (Kempe, Eriksson‐Gustavsson, & Samuelsson, 2011). Students who fall behind in reading read less, increasing the gap between them and their peers. This reading difficulty will follow in other subjects and will overall result in less attainment of knowledge across academic subjects.

So now that we have unequivocally established that both language and literacy skills of students with suspected APD need to be assessed, I want to explain further the reasons why certain abilities are strongly correlated with academic success.

Why Assess Narrative Abilities?

Poor discourse and narrative abilities place children at risk for learning and literacy-related difficulties including reading problems (McCabe & Rosenthal-Rollins, 1994). Narrative analyses help to distinguish children with developmental language impairment (DLD) from their typically developing peers (Allen et al 2012) Narrative weaknesses significantly correlate with social communication deficits (Norbury, Gemmell & Paul, 2014). Students with decreased narrative abilities evidence numerous social communication deficits. Students with poor oral language competence display poor reading comprehension and written composition abilities even in the presence of relatively intact non-word reading as well as  reading fluency skills

Why Assess Pragmatic Language Abilities?

Children with language deficits are impaired in multiple areas of language. Researchers found evidence that children with language deficits manifested pragmatic difficulties in conversational contexts with partners and were unable to adequately adjust to the needs of others in social interactions, which resulted in fewer peer relationships, and less satisfaction with peer relationships (Brinton, & Fujiki, 1993, Brinton & Fujiki,  1995; Brinton, Fujiki,  & Powell, 1997; Fujiki, Brinton  & Todd 1996). Children with DLD [as mentioned in the previous post and handout research indicates that presenting symptoms of APD are in fact symptoms of language and literacy deficits], also present with concomitant social communication difficulties, which if left untreated will significantly adversely affect their academic outcomes (reading and writing) as well as future life success (Botting & Conti-Ramsden, 1999).

Why Assess Reading Abilities?

Solid reading fluency and comprehension abilities allow students to effectively partake in all academic subjects. Having deficits in these areas will result in adverse academic outcomes. Missing deficits in these areas will result in poor intervention gains since the students will not effectively master their therapy goals.

Why Assess Writing Abilities?

Solid writing abilities allow students to write competent essays, book reports, and subject-specific projects and prepare them for college entrance. Similar to the deficits in reading, having writing deficits will result in adverse academic outcomes. Once again missing deficits in these areas will result in poor intervention gains since the students will not effectively master their therapy goals. This will in turn result not just in poor post-secondary (college) but also poor vocational outcomes (job market).

So what is the best way to perform these assessments? Below are some useful links to free posts and paid products (denoted by *) on this subject along with a few clinical observations.

How to Clinically Assess Narrative and Discourse Abilities?

1. Clinical Assessment of Narratives in Speech Language Pathology*
2. The Importance of Narrative Assessments in Speech Language Pathology
3. Analyzing Narratives of School-Aged Children
4. Identifying Word Finding Deficits in Narrative Retelling of School-Aged Children

How to Clinically Assess Pragmatic Abilities?

1. Social Scenes for Assessment and Treatment Purposes
2. Assessment of Social Communication From Toddlerhood Through Adolescence*
3. Social Pragmatic Assessment and Treatment Bundle*
4. The Role of Pragmatic Language in Reading Comprehension and Written Expression: Focus on Assessment*

How to Clinically Assess Reading Abilities?

Standardized tests of reading fluency and comprehension leave a great deal to be desired. All standardized tests of reading possess limitations. To illustrate, the Test of Reading Comprehension Fourth Edition (TORC-4) assesses untimed reading comprehension abilities primarily via multiple-choice questions of reduced complexity. This allows for a score over inflation as even poor readers have the opportunity to guess the correct answers 25% of the time. Furthermore, the presence of certain words in multiple-choice responses may trigger the student to correctly choose that answer even in the presence of poor reading fluency and reading comprehension skills. In contrast to the TORC – 4, the Gray Oral Reading Tests – Fifth Edition (GORT-5), is a timed reading test that assesses reading fluency (rate and accuracy) as well as reading comprehension via open-ended questions. Unfortunately, it is also not without certain limitations. Many of the answers to the open-ended questions are very factual and can be guessed if the student possesses some adequate background knowledge and vocabulary awareness.

So what is the best way to assess reading comprehension? For starters, standardized reading assessments such as the GORT-5 or the TORC-4, etc, are a good start to establish basic reading competence and ensure that the student has a solid mastery of foundational basics. Clinical grade-level reading assessment is the next step as it allows the clinicians to determine the student’s reading abilities on a deep vs. shallow level.

The most effective way is via the following methods:

1. Asking abstract verbal reasoning questions
2. Asking to define literate vocabulary words
3. Asking to state the main idea of the passage
4. Asking to summarize the passage

The above methods will reveal a true understanding of passage content. In contrast, multiple-choice questions and factual open-ended questions will tap into the student’s shallow knowledge of the passage and may result in an illusion that the student understands the passage,  but are not adequate enough to ascertain true comprehension of passage content.

Useful links:

1. Clinical Assessment of Reading Abilities of Elementary Aged Children
2. Adolescent Assessments in Action: Clinical Reading Evaluation
3. The Role of Speech Language Pathologists (SLPs) in Assessment and Management of Dyslexia
4. Comprehending Reading Comprehension
5. Language and Literacy Assessment Tasks and What They Measure
6. Comprehensive Assessment of Elementary Aged Children with Subtle Language and Literacy Deficits
7. The Role of Pragmatic Language in Reading Comprehension and Written Expression: Focus on Assessment*
8. Reading Fluency and Reading Comprehension Master Course*
9. The Science of Reading Literacy Certificate for SLPs*

How to Clinically Assess Writing Abilities?

1. Clinical Assessment of Elementary-Aged Students Writing Abilities : Suggestions for SLPs
2. Components of Qualitative Writing Assessments: What Exactly are We Trying to Measure?
3. Writing Assessment and Intervention Master Course*
4. Spelling and Morphology Master Course*

There you have it! I hope that I have adequately addressed your queries regarding quality assessments of students with “APD” and in this process offered you some new and useful information you can apply in your clinical practice!