Assessment of thinking skills may skew children from poor urban areas in the US

Some assessment tools that measure children’s thinking skills in the U.S. may have misrepresented poor urban students because they are modeled after wealthier — mostly white — populations.

In a recently published study of nearly 500 children from poor urban communities in the United States, researchers found that a widely used assessment that measures the development of thinking skills called “executive functions” does not provide a complete and accurate assessment of student achievement. The study attributes this to likely cultural bias in the assessment design and suggests that this may be replicated in other similar instruments. The work was published in Developmental science.

Any such design flaw could have affected the growing hull research this suggests that children from poorer families tend to start school with less developed executive functions. “Executive functions” is a collective term for the set of basic thinking skills required to perform daily tasks and training. These include working memory, self-control, the ability to ignore distractions, and switch between tasks easily. Children with good executive function tend to have better test scoresbetter mental health and more working capacity.

One common method of measuring the healthy development of these skills is to ask teachers to fill out questionnaires about observed children’s behavior. The results can potentially help identify children or whole groups who need additional support. They also provide a rich source of data for research on how executive functions develop.

In a new study, researchers found that one of these teacher rating scales, widely used in the United States, had limited value in evaluating poor urban students. Specifically, they found that the executive function test of a version of the Behavior Assessment System for Children (BASC), called the BASC-2, “is not a good reflection of the day-to-day executive function behavior of children from schools in poor communities. “

A team from Cambridge University (UK) and Virginia Commonweath University (USA) suggests that the likely reason is that this scale and others like it were developed using a non-representative sample of children.

Researchers noted earlier that these estimates are usually modeled on children who are mostly from comfortable socioeconomic conditions. By mapping their observed behaviors onto executive functions, they may falsely assume that these behaviors are “normal” markers for any child of the same age. In reality, children’s different backgrounds and life experiences may mean that executive functions are expressed differently in different groups.

Annie Zonneveld, from the University of Cambridge’s Faculty of Education and first author of the study, said: “There is a big question in how we measure executive function: are we really using the right tools? If they’re based on white, middle-class students, we can’t be sure they’ll really work for the entire population. We can see evidence of that here.”

Michelle Ellefson, professor of cognitive science in the Faculty of Education, said: “Teachers can give us really valuable data about children’s executive functions because they can monitor development in a way that we haven’t been able to replicate in the lab, but they need effective measures to this means that assessments must be based on information about children from different backgrounds.’

According to the Children’s Fund, about 14% of children in the United States live in poverty. While almost 50% of all children with ethnic minority families, 71% of those living in poverty are from these segments of the population. However, most psychometric research on executive function has focused on white middle-income or affluent families. It was never clear how generalizable his results were.

The new study examined the executive function components of two versions of the BASC: BASC-2 and BASC-3. They ask teachers to observe children’s daily behavior and rate on a scale from “never” to “always” how much they agree with statements such as “acts without thinking,” “easily distracted,” “can’t wait for a turn,” “independent ” and “argues when he is denied in his own way.” They then extrapolate information about the children’s executive functions based on the responses.

The researchers analyzed two sample groups of children around the age of 9 or 10, all from public schools in poor urban areas of the United States. A total of 472 children took part in it. The first sample was assessed using the BASC-2; another using BASC-3.

Both groups also completed six computer-based tasks that psychologists and neuroscientists use in laboratory tasks to measure certain executive functions. The researchers looked at how well the results of these computerized tasks — accurate but difficult to complete in large groups — matched the results of surveys administered by teachers.

Findings indicated that while the BASC-2 provides a reasonable overview of students’ overall executive functioning, it does not capture precise details about specific functions such as working memory and self control. The BASC-3 was significantly more effective, probably because it uses a different and more focused set of questions.

“BASC-2 is widely used in archival datasets and contributes to academic research on how executive functions develop,” Ellefson said. “It’s very important to recognize that, without modification, this is not an appropriate basis for making judgments about certain groups of children.”

The score is just one of many surveys that measure children‘s cognitive development in different countries. “It’s important that we know how these tools establish a baseline understanding of ‘typical’ development,” Zonneveld said. “If they’re based on a predominantly white population from affluent suburbs, they’re not necessarily going to be as representative as we might hope.”