Home | Clinical Café Archive | February 2004

Answering a question from a reader—How do clinical validity studies apply to the students that I test?

Download this article in a printable PDF file format:

PDF File

Question:
Simply Stated “Example: I have a hearing impaired student (moderate range) who wears hearing aids and a Com-Tek in the classroom. When I gave him the Oral and Written Language Scales (OWLS), he was below average as compared to the general population. However, he was fairly consistent with the clinical validity sample for hearing-impaired students. How is this reported? What does it mean?”

Answer by Tsuey-Hwa Chen, Ph.D.
AGS Publishing, Associate Director of Research

Studies involving comparisons of test performance between population subgroups have often been used as part of the construct validation process for a test. Construct validity data establish the degree to which the underlying psychological construct, or trait, that the test purports to measure is actually measured. One source of evidence of construct validity is the study of the differential performance levels achieved by the various clinical subgroups of the population. The differences in test scores exhibited by the clinical subgroups provide evidence of the test’s ability to distinguish among individuals with mental, psychological, or physical conditions likely to affect the construct measured by the test.

To permit systematic statistical evaluation of the test performance of clinical groups, the clinical conditions included in the studies need first to be carefully chosen based on their theoretical relevance to the construct measured. Random sampling (usually stratified according to some demographic characteristics) of subjects within a specific clinical subgroup is then carried out to select a representative sample for the study. Each clinical case is matched on a variety of demographic variables (such as age, gender, race/ethnicity, socioeconomic status, and region) with a “normal” case in the standardization sample. This is to prevent the influence of irrelevant sources of variation on the subgroup performance.

Upon selection of a matched control (“normal”) group, data are analyzed using t tests for paired samples. Paired t tests are used because the samples are matched in terms of their demographic characteristics, and are therefore not statistically independent. The results provide important empirical evidence for the validity of the test as a clinical tool.

Many educational, psychological, and language tests often are used with individuals who are exceptional in some way. Clinical validity studies can provide important diagnostic information on individuals with special conditions. For instance, the clinical studies of OWLS involving individuals with a hearing impairment showed statistically significant differences in all scales (Written Expression, Listening Comprehension, Oral Expression, and Language Composite). Students with hearing impairment who are administered OWLS are, in general, expected to score significantly lower than the norm sample (general population).

The following is an example of how the OWLS clinical validity information can be used. Johnny, a fourth-grader diagnosed with a hearing impairment who wears hearing aids and a Com-Tek in the classroom, was given the OWLS and scored significantly lower than his age and grade peers in the norm sample. However, when compared to the hearing-impaired clinical subgroup, his scores were found to be close to the average of that group.

This indicates that Johnny’s language performance is consistent with his clinical diagnosis and that his performance is within the typical range of individuals with a hearing impairment. Further use of the information depends on the purpose of the testing. If the test is to be used for providing a performance profile to identify Johnny’s strengths and weaknesses, information on Johnny’s scores on other tests (such as achievement in reading, math, and so on) needs to be examined along with his scores on OWLS. On the other hand, if the test scores are to be used for program placement, then they need to be gauged against some established cutoff or benchmark for qualification.

At any rate, clinical studies provide useful information not only for the continuing validation of the test itself, but also for individuals who take the test.

Send us your "What I'd like to learn about tests this year" list

As your partner in testing, we'd like you to know what we do, how we do it, and why. In turn, we'd like to know what other information we could provide to help you in your jobs. So send us your "What I'd like to learn about tests this year" list to webmaster@agsnet.com. And we'll try to fulfill your wishes.

SLP Discussion Center

As always, we'd like to thank you for your ongoing service to people with communication needs and to remind you that we are here to support you with that effort. If you'd like to discuss this topic further, please feel free to use the SLP Discussion Center as the vehicle for an ongoing discussion with your colleagues. Should you have questions regarding these or other Pearson Speech and Language products, we welcome your phone calls at 800-627-7271 or use our web site at http://ags.pearsonassessments.com.