A Quality Control System for Assessment Processes by Means of a Collaborative Internet-based Assessment Platform - TAO-QUAL

Coordinating Institution: Université du Luxembourg
Contracting Partner(s): CRP Henri Tudor
From: 01/04/2007
To: 31/05/2010
Budget: 300,000.00€
Contact(s): Busana Gilbert , Martin Romain

Summary

The open source platform TAO (from the French acronym of computer-based testing: testing assisté par ordinateur) generates a general and open framework for computer-assisted test development and delivery issues. The TAO platform has the ability to integrate all the actors of the entire computer-based assessment process.

The process model allows the development and release of computer based psychometrical and educational tests and assessment campaigns. At a time where more and more education and training systems are heading towards output-oriented systems, the existence and availability of high quality assessment instruments for an ever growing variety of measurable competencies becomes more and more important.

As these assessment instruments will be the most important steering instruments in output-oriented systems, not only the issue of the quality of these instruments but also the question of the quality of the assessment process into which these instruments are embedded has to be put forward. The present project focuses on the quality control of the assessment process. In order to optimize and evaluate the workflow of TAO - on the base of a review of the relevant literature, criteria and steps of the assessment process will be created - it is necessary to develop a quality control system for assessment processes. With respect to different assessment purposes it is very difficult to define one unique canonical model of psychological assessment. Therefore a more flexible modelling approach towards different assessment processes is more appropriate.

The TAO-platform is built upon an existing modular, web-based architecture permitting the development of multiple item banks, as well as the compilation and the internet release of computerized tests. In order to ensure the quality of the assessment process (beyond quality considerations of the test instrument itself) of the generic open-source platform the assessment process will be tested and evaluated on the basis of three use cases: review and adaption of item content, item translation and coding of open-ended responses. The feasibility of this approach will be documented in the usability lab. Therewith the quality of the workflow process and the assessment process of the open-source platform TAO will be evaluated and if necessary adapted. The TAO project is a joint development of the EMACS research unit of the University of Luxembourg and the CITI department of the Centre de Recherche Public Henri Tudor. The extension of the project was made possible with the support of the TAO-Transfer project.

Publications
  • Koenig, V., Busana, G., & Scherer, M. (2009). Testing the national adapted and translated CAPI version. Meeting of IT coordinators in Luxembourg, 14-16 September, OECD internal project paper.
  • Koenig, V., Busana, G., & Scherer, M. (2009). PIAAC National English Background Questionnaire: Quality Assessment. Meeting of the National Project Managers in Barcelona, Spain, 23-27 March, OECD internal project paper.
  • Busana, G., Koenig, V., Martin, R., Latour, T., Plichart, P., Jadoul, R. (2008): A web-based quality control system for assessment processes. International Journal of Psychology, 43, p. 466.
  • Martin, R., Busana, G., Plichart, P., & Latour, T. (2007). TAO: architecture and use cases of a collaborative, internet-based platform for computer-assisted testing. European Association for Research on Learning and Instruction Conference in Budapest.
  • Martin, R., Busana, G., Plichart, P., & Latour, T. (2007). TAO: Use Cases & Item templates. European Association for Research on Learning and Instruction Conference in Budapest.
Project Website:

Figure : Work-Flow TAO