Practice Effects in Testing
A recent question came to us from a colleague in Pennsylvania.
Q: What is current thinking on practice effects with standardized testing? How often is it ok to repeat tests like the PPVT-4, the CASL, or the CELF-4?
A: It depends (sorry, we know it’s easier to have black and white answers!). Most tests should have information in the manual about a development study in test-retest reliability–that is, the reliability of an individual’s performance over time. To help determine the risk of “practice effects” (i.e., low test-retest reliability), you need to consider the domain being measured, what research has been done to show the impact of practice effects between administrations, and the circumstances of your original administration.
As an example, in the PPVT-4 manual, pages 55-57, there is a description of the test-retest study completed during standardization. The window of time between administrations was a minimum of 14 days and averaged four weeks. In this 300+ person study, the reliability of the scores averaged a very high .93, which means that the PPVT-4 is quite resistant to practice effects given that window of time between administrations. Other tests will have different retest windows and should give guidance on a recommended “wait time” between administrations, with the usual caveats. Certainly, CELF-4 and CASL include this information in their manuals as well.
Another option to consider is the use of parallel forms, where available. In the case of PPVT-4 or EVT-2, for example, a second parallel form exists and you may choose to use the alternate form (i.e., a completely different but similar in difficulty item set) for your next test administration. This is one of the benefits of having two (or more) forms of a test.
Finally, for those tests without parallel forms for children, one set of giudelines might be that you allow enough time to elapse so that:
1. the examinee is now in the next norm group (e.g., 3-6-12 month interval, depending on the content and the norms,
2) the examinee no longer remembers the test items, OR
3) the examinee appears to have made progress (otherwise, why test?)
A final consideration is that if the examinee is sick or has other reasons for not participating in the original administration, you probably can feel confident testing right away again as soon as the individual feels better.
Hope this helps…as always, it’s a somewhat nuanced answer depending on the situation, examinee, and test. The best advice is to consult the test manual for direction. Feel free to continue the conversation with your comments below!