Did a "Traumatic" Test Question Create Racial Bias?

On the second day of a 2019 high-stakes assessment, Massachusetts 10th graders faced a racially insensitive question, which was quickly invalidated. However, a remaining controversy is whether this exposure created a racial bias in performance on the remaining test items. We present results from a pre-registered analysis suggesting the controversial question reduced the comparative performance of Black students by a small amount (approximately 0.006Ϭ). However, we also find a wide dispersion of such effects when examining similarly small sets of test items from prior state assessments that lacked a controversial question which suggests the 2019 assessment was not distinctive.