Psychology Science

Adaptive Success Control in Computerized Adaptive Testing

Computerized adaptive testing, or CAT, works inside the framework of probabilistic test theory by tuning the difficulty of each item to the ability of the person answering it. The goal is to squeeze the most information out of every single item. Do that well and you gain on two fronts: the test becomes more economical, and it feels more reasonable to the person taking it.

There is a catch that earlier studies have pointed to. When an item is calibrated so the respondent has roughly a fifty-fifty chance of answering correctly (a success probability of p=0.5), some people feel pushed too hard. They struggle to reach a level of certainty they are comfortable with inside a sensible amount of time. So they slow down, and the time spent on each item climbs.