Assessment of Students’ General Cognitive Ability with Computer-Based Complex Problem Simulations


CALL: 2008

DOMAIN: LM - Challenges for the Educ.System,Labour Market,Social Protection incl. Territorial Aspects


LAST NAME: Brunner



HOST INSTITUTION: University of Luxembourg

KEYWORDS: general cognitive ability, complex problem solving, computer-based assessment

START: 2009-04-01

END: 2012-03-31


Submitted Abstract

General cognitive ability (“Intelligence”) is among the most important and useful psychologicalconstructs capable of predicting health and longevity, academic success, as well as success on the job. Atthe same time it is one of the most controversially discussed scientific concepts: On the one hand it hasbeen impossible to find a generally accepted definition of intelligence so far. On the other hand, manyheated debates arose from the (erroneous) idea that intelligence is an individual characteristic that iscompletely genetically determined and that is not fostered by formal schooling. Nevertheless, intelligenceas assessed in typical intelligence tests represents a general cognitive resource that substantively aids tosuccessful problem solving across content domains. Thus, it comes as no surprise that intelligence testshave been shown to predict key outcomes such as academic or occupational success. However, one majorshort-come of typical intelligence measures is that these instruments lack face validity for their predictivecapabilities. Further, intelligence tests use static problem formats with a relatively limited amount ofcomplexity and generally do not provide information on test takers’ problem representations. The presentproject aims to overcome these severe limitations of typical intelligence tests by using complex problemsolving scenarios as an alternative assessment instrument of students’ general cognitive ability. Within acomputer-based environment real-world situations with a varying degree of problem complexity will besimulated. The underlying causal relations are grounded on linear structural relations among a varyingnumber of variables. Thus, problem complexity may be increased by (a) the number of variables that canbe manipulated and (b) by the number and quality of relations among the variables within the scenario.Two performance scores will be obtained: (a) students have to manipulate the variables of the problemscenario to achieve certain goal states. (b) The quality of students’ understanding of the problemsituations will be assessed by means of causal diagrams which students use to represent their thinking onthe relations among the scenario’s variables. Data will be obtained from 200 students who attend thegrade levels 9, 11, and 13, respectively (total N = 600). These are the grade levels at which many studentsmake their transition from school to the workplace therefore making an assessment of students’ generalcognitive ability particularly relevant. All students will also take a typical intelligence test in order toassess the degree of overlap between the classic approach of intelligence testing and our novel computerbasedproblem solving instrument. We will also collect school grades and teacher judgments of theirstudents’ cognitive competencies as an external educational criterion of students’ job-related abilities.Our instrument will be implemented in the open-source testing platform TAO that has been jointlydeveloped by the University of Luxembourg and the CRP Henri Tudor (FNR grant FNR/06/05/16).Future projects may use our computer-based simulation environment for the training of students’problem-solving skills as well as the assessment of domain-specific problem solving strategies andknowledge representations (e.g., in mathematics).

This site uses cookies. By continuing to use this site, you agree to the use of cookies for analytics purposes. Find out more in our Privacy Statement