Testing collaboration at school

The PISA assessment of school performance will soon include Collaborative Problem Solving. How will it be tested, and how will its introduction alter the curriculum?

Students working together in a groupCollaborative and problem solving skills are receiving increased attention in the discourse on nurturing students’ competencies for a globalized, knowledge-based economy. But while there has been extensive educational research on fostering these skills, studies which consider the assessment of them seem to be much less developed.

Now, the Organisation for Economic Co-operation and Development (OECD) has set its sights on a large-scale assessment of Collaborative Problem Solving (CPS). It has made CPS a major area for primary development in the 2015 version of its Programme for International Student Assessment (PISA). In PISA 2015, the CPS proficiency of students is to be assessed by the extent to which they can respond to requests and initiate actions or communications to advance group goals. The test taker will be given a collaborative task in which he or she will work with one or more others to solve a task together. It might be a hidden profile task, in which the test taker has access to certain information which the other participants have not, or vice versa. These tests have to be administered to scores of countries, and face the constraints of computers that are not networked. It is therefore planned to use computer agents in the role of collaborators in the PISA tests.

When I presented the proposed PISA assessment of CPS to an audience of educationalists in Chile, I was asked: ‘Would it be fair to assess students in CPS when they are not taught these skills at school?’ Precisely. The administration of these new assessments on an international basis will drive change in educational systems around the world, encouraging the inclusion of CPS in the curriculum.

How will the CPS agenda be implemented in the countries and regions participating in PISA? Schools might try to prepare students to be better collaborative problem solvers by including curricular activities that involve collaborative work and which at the same time deepen understanding of the disciplinary content concerned. When it comes to providing situations and platforms for students to work together, there is much research in the Computer-Supported Collaborative Learning community which can be tapped into. For the past two decades, this community has articulated design principles and designed interactive environments to support collaboration. These researchers now have an opportunity to rise to the challenge of making their work relevant, informing policy makers of the findings and outcomes of collaborative learning research, and putting them to practical use in schools and in learning.

My own work with Professor Miguel Nussbaum from the Catholic University of Chile shows that there is a gap, requiring further research, between existing assessment methods and the proposed PISA assessment framework for CPS. In the past, we and other researchers have designed pedagogies and environments to enable collaboration. In school-based research, we co-designed lessons and activities with teachers, and treated collaboration in a holistic way, focusing on its outcomes and processes. The PISA CPS assessment framework, however, proposes a matrix of skills that include items such as ‘communicating with team members about the actions to be/being performed’, ‘following rules of engagement’, and ‘monitoring results of actions and evaluating success in solving the problem’. The assessment tests for such specific skills and competencies, used collaboratively. However, when students are more aware of the skills and competencies in the matrix cells, how does this awareness help or hinder collaboration? Will students work together in the test to score in those skills? If teachers teach or students learn to collaborate in ways driven by the assessment framework, how do we ensure students will work together in a natural, voluntary and fluent way that results in high performance?

Looking beyond these concerns, the PISA CPS test will probably drive education reforms and curriculum changes, the success of which will require partnership between policy-makers, school practitioners and researchers. In five years, we shall be able to witness innovative ways in which mathematics, sciences and the arts subjects will be taught in schools worldwide, and which incorporate CPS. Such curricular reforms should create opportunities for practice-driven research on preparing students to be better collaborative problem solvers, and more effective team players in the workforce.

Chee-Kit Looi is a Professor of Education at the National Institute of Education, Singapore.

Share this page