The Research is In: Our Approach to Testing is Wrong

We used to rely upon the observations of parents, teachers, and psychologists to explain what was happening inside the brains of children.

And those observations tell us children are a highly variable bunch!

Since the 1930s, we have treated testing as a valid objective measure of childhood development. We have used it to define norms and deviations from norms. Children are mapped to a one-dimensional categorical model based on age and, sometimes, gender.

With the advent of neuroimaging, we can now see inside brains as they work. New images show that development is highly variable (surprised?). Megha Sharda, Nicholas Foster, and Krista Hyde contend that conventional methods for understanding typical and atypical development are far too simple. The current categorical approach treats human variability as “noise” to be analyzed away.

Sharda writes that Twentieth Century measures miss factors that contribute to outcomes, being “insufficient, not only for neurodevelopment disorders but also for typical development.”

Twentieth Century models and beliefs about development and education culminated in the passage of No Child Left Behind at the beginning of the Twenty-First Century. Its architects believed that categorical assessment would lead to actionable information to improve student achievement.

Since the passage of NCLB, the United States has slipped backward internationally in math on the Program for International Student Assessment (PISA) and has had no change in reading. Worse, some states, such as Florida, have adopted policies directly contradicted by research, such as retaining third graders who fail to achieve a proficient level on the third-grade ELA assessment.

In 2011, National Research Council (NRC) reviewed the research and concluded that the fundamental premise of NCLB was false. Sixteen years of data later and we still don’t know how to use information from testing to improve education and achievement. We can continue on the same path, but that would be the definition of insanity.

Education is on the cusp of a breakthrough such as we have seen in economics. Traditional economics said all humans act rationally in their best interests. Newer, behavioral economic models examine how emotional, cognitive, social, psychological and related factors cause humans to behave in predictable, but not rational ways.

The Twenty-First Century growth in knowledge about cognitive development happened alongside NCLB. Human variability is not merely noise, but a contributing factor to our inability to effectively use categorical testing to improve education outcomes.

The research does not support continuing to categorically test children every year from third to eighth grade. Retaining some checkpoint testing has informational value, but there is no justification for continuing to test children every year for six years in a row.

The NRC’s review is a call to reinvest in basic research to identify measures that actually result in positive effects on achievement. That research should focus on finding sensitive windows for identifying factors that lead specific children to underachieve.

We also need tools to help us separate variability from a neurological or psychiatric disorder. And we need politicians and educational reformers to build human variability into their models for assessing progression through standards.

Reducing categorical testing and replacing it with tailored testing to catch students at risk of underachieving would better meet the goals of having our students college, career and technical college ready.

The beauty of using materials like Boom Cards and other self-grading and adaptive materials is that they allow teachers to unobtrusively and continuously conduct formative assessments and identify areas for intervention or enrichment. Rather than disrupting classroom flow they can seamlessly be integrated.

Last blog post we asked does the Every Student Success Act (ESSA) get it just right? We think amending ESSA to test for science, mathematics, and reading or language arts no more than once in each of the windows of (a) 4th to 6th grade, (b) 6th to 9th grade, and (c) 10th to 12th grade would be the first step towards a research-based assessment policy.

Drawn from:

National Research Council. (2011). Incentives and Test-Based Accountability in Education. Committee on Incentives and Test-Based Accountability in Public Education, M. Hout and S.W. Elliott, Editors. Board on Testing and Assessment, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. https://www.nap.edu/read/12521/

Sharda, Megha, Nicholas E.V. Foster, and Krista L. Hyde. “Imaging Brain Development: Benefiting from Individual Variability.” Journal of Experimental Neuroscience 9.Suppl 1 (2015): 11–18. PMC. Web. 6 Apr. 2017. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4667561/