Assessing critical evaluation skills of undergraduate science students

K. Colthorpe, H. Mehari Abraha, K. Zimbardi and L. Ainscough, School of Biomedical Sciences, The University of Queensland, St Lucia, QLD 4072, Australia.

The ability to critically evaluate and use evidence from one’s own work or from primary research literature is invaluable to any researcher. But the benefits of these skills go beyond the needs of scientific research, as the ability to create coherent scientific arguments is based on one’s ability to support claims with evidence (Bugarcic et al., 2014; Toulmin, 2003). The skills required to critically evaluate and use evidence include the ability to identify strengths and weakness of primary literature; to gauge the impact of research findings on a field; to identify gaps in a field that require more research; and to contextualise findings within a field (Blommel & Abate, 2007).

This study examined the development of undergraduate science students’ abilities to critically evaluate and use evidence, through an analysis of the discussion sections of their written laboratory reports from an inquiry-based laboratory class. In order to gauge these skills, a model of epistemic levels was developed to analyse the way in which students use evidence to support their claims. In addition, the discussion sections of published scientific research articles were subjected to the same analysis, and comparisons made between student and expert authors.The student reports analysed (n=42) used evidence in a variety of ways, using 6.6±0.2 of the nine categories identified in the model. Most often referring to literature indirectly (29%), and least commonly highlighting limitations of literature (2%). The reports averaged 25.5±1.9 instances of evidence use across 762±35 words. The number of references cited varied from 5 to 23, averaging 10.1±0.5 per report. There were significant positive correlations between grade and number of references, length, and number of instances of evidence use, with these two latter variables also being strongly correlated. When compared to students, instances of use of evidence in scientific research articles (n=7) were very similar, averaging 7.3±0.3 of the nine categories of evidence use. However, expert authors were significantly more likely to refer to their own results than students, with this being the commonest (37%) type of evidence use, and cited significantly more references.

The inquiry-based curricula provided opportunities for students to development their skills in critical evaluation of evidence use, and the construction of arguments based on their own findings. Analysis of their laboratory reports showed that students, by the completion of the second year of their undergraduate degree program, already have expertise approaching that of published authors. These findings demonstrate that it is possible to provide valuable broad-scale undergraduate experiences to all students in a cohort, giving them exposure to the methods and communication processes of research, and an opportunity to hone their critical evaluation skills.

Blommel ML, Abate MA. (2007) A rubric to assess critical literature evaluation skills. Am J Pharm Educ 71(4), 63.

Bugarcic A, Colthorpe K, Zimbardi K, Su HW, Jackson K. (2014) The development of undergraduate science students’ scientific argument skills in oral presentations. Int J Innov Sci Math Educ 22(5), 43-60.

Toulmin SE. (2003) The uses of argument. Cambridge University Press.