Using formative feedback to identify and support first year chemistry students with missing or misconceptions. A Practice Report

Students entering tertiary studies possess a diverse range of prior experiences in their academic preparation for tertiary chemistry so academics need tools to enable them to respond to issues in diversity in conceptual models possessed by entering students. Concept inventories can be used to provide formative feedback to help students identify concepts that they need to address to improve construction of subsequent understanding enabling their learning. Modular, formative learning activities that can be administered inside or outside of class in first year chemistry courses have been developed. These activities address key missing and mis-conceptions possessed by incoming student. Engagement in these learning activities by students and academics will help shift the culture of diagnostic and formative assessment within the tertiary context and address issues around the secondary/tertiary transition. This diagnostic/intervention framework is currently being trialed across five Australian tertiary institutions encompassing a large heterogeneous sample of students. year with

Modular, formative learning activities that can be administered inside or outside of class in first year chemistry courses have been developed. These activities address key missing and mis-conceptions possessed by incoming student. Engagement in these learning activities by students and academics will help shift the culture of diagnostic and formative assessment within the tertiary context and address issues around the secondary/tertiary transition. This diagnostic/intervention framework is currently being trialed across five Australian tertiary institutions encompassing a large heterogeneous sample of students.

Background
Constructivist learning environments are most effective when the learner and teacher are both aware of the existing conceptual models that learners possess to enable them to extend and apply their understanding rather than resort to rote learning (Taber, 2001). As students transition into, and engage in, the new tertiary learning environment, it is important to assist them to maximise the effectiveness of their learning which requires measurement or diagnosis of their existing conceptual understanding. One of the challenges in teaching chemistry is to encourage students to recognise their existing knowledge and conceptual understanding and then apply them in new learning situations (Schraw, Crippen, & Hartley, 2006).
A typical Australian tertiary first year chemistry cohort is characterized by large numbers (300-1500) of students who represent a diverse range of academic abilities, interests, and motivations for learning. Students can be enrolled in one of approximately forty possible programs of study that require chemistry such as engineering, medicine, pharmacy, dentistry and other health sciences. Many students have completed secondary chemistry studies while other students enrol with no prior secondary chemistry experience. This diversity in chemistry conceptions possessed by students provides a unique opportunity to research and develop mechanisms to differentiate learning support for students at risk of failing in tertiary chemistry studies.
In this Office of Learning and Teaching project, academics in five universities across three states are collaborating to develop and implement a diagnostic and intervention activity framework to provide support to entering first year students who have poorly formed conceptions in chemistry. In this Practice Report, we discuss the outcomes of the first stage of the intervention and the challenges that arose during the process.
Chemistry concept inventories (CCI) are multiple choice tests that have been considered widely as a route to exploring students' existing conceptions (Mulford & Robinson, 2002;Othman, Tregaust, & Chandrasegaran, 2008;Pavelich, Jenkins, Birk, Bauer, & Krause, 2004;Potgieter, Davidowitz, & Venter, 2008). Despite the published outcomes of concept inventories in terms of common misconceptions, there are few interventions or learning tools that a teacher can apply as an intervention for students to address incorrect or naive mental models. We are addressing this deficiency by assembling web-based tools that academics can apply as appropriate in their own tertiary contexts. The role of feedback linked to formative assessment as the core part of the intervention tool is illustrated in Figure 1.
Feedback is particularly important for first year students because they are coming to terms with the changes of environment, expectations, teaching approaches and forms of assessment. In this context, Hattie and Timperley's (2007) three questions are particularly relevant: "Where am I going?", "How am I going?" and "Where to next?" Formative assessment is critical to "How am I going?" and the feedback is just as valuable for the instructor as for the students to align teaching and learning activities.
The efficacy of matching diagnostic assessment with a post-test intervention has been established in chemical education research (Heredia, Xu, & Lewis, 2012;Treagust et al., 2011). The challenge is to engage students in their individual feedback and enable them to respond to it, especially in the large class setting. This scale and the corresponding low instructor/student ratios make it imperative to develop a student's ability to reflect on and self-regulate their own learning. Useful guidelines for the use of feedback to promote self-regulation (Nicol & Macfarlane-Dick, 2006) include the clarification of good performance, facilitation of self-assessment and encouraging feedback and dialogue. In the pilot intervention of this project, student perception and utilization of individual feedback from the diagnostic assessment is being explored. In this study, once alternative conceptions have been identified, students are placed in environments where they can challenge their conceptions through alternative frameworks (Chi, Slotta, & Leeuw, 1994).

Context and intervention
Seven first year chemistry course coordinators and a secondary teacher educator, representing multiple tertiary institutions across Queensland and New South Wales, two of the eastern states in Australia, comprise the core project team. A Concept Inventory Diagnostic Inventory test has been assembled from multiple literature instruments and also informed with pilot data collected between 2009−2012 (Lawrie, Schultz, & MacAskill, 2012;). Students answer sets of five questions around five core concepts and their combinations of responses are used to identify key misconceptions. The five core conceptual clusters that are being addressed in this project are: phase changes, energy transfer, solutions, conservation of matter and equilibrium. These concepts are tested at the level encountered in secondary chemistry (or junior science) and are common to the syllabi across multiple Australian states.
Semester 1, 2013 involved the following activities: • Week 1-2: The diagnostic instrument was delivered online across all participating institutions; • Week 3-4: Students received individual feedback indicating the concepts they are performing well on and those that they need to address to enable success in tertiary chemistry; • Week 4: A link to a web-based activity that addresses the basis of a misconception was made available to the students and their progress monitored by concept check questions; and • Weeks 5-13: The authors undertook evaluation of longer term impacts through formative and summative assessment of related concepts.
Evaluation of the intervention includes student interviews and learning outcomes at the end of semester 1. The experiences of the academics involved in the delivery of the interventions have been collated to enable improvement in the next iteration of the study. One of the key challenges has been to deliver activities that engage students in active learning while adjusting their mental models. The pilot intervention, which addresses the concept of phase changes, combines interactive simulations with everyday phenomena inviting students to reflect on reasonable explanations for phenomena. Cognitive conflict (dissonance) has been applied as a strategy to encourage students to critically appraise their conceptual models (Linenberger & Lowery Bretz, 2012). Students have been challenged to consider whether their conceptions are valid.
We are placing a spotlight on articulation between the secondary and tertiary contexts which will enable us to inform practice in both sectors particularly in the context of the National Curriculum (Australian Curriculum, Assessment and

Perspectives of conference delegates
The challenges in engaging first year chemistry students in self-regulated learning based on formative feedback that were presented at the session generated discussion around several factors related to how students recognise and respond to formative feedback. Two key perspectives developed from practitioner experiences regarding the motivation of students to engage with feedback.

Participants'
suggestions: The suggestions, resulting from peer discussions, are clustered into four approaches to delivering formative feedback that were perceived would likely to encourage students to respond: • Dialogic interactions: providing feedback via a tutor in face-to-face sessions or in the form of generic feedback in large class contact sessions; • Individual feedback: delivered through a variety of mechanisms including online (email, SMS, Facebook);

•
Strategic timing: feedback is either released before the grade is released to encourage students to read it or is linked to future studies in the form of oral or written feedback from the previous semester's final exam delivered at the beginning of the following semester; and

•
Extrinsic motivation: a very small summative mark is associated with students responding to feedback.

Critique of suggestions:
These approaches encompass several feedback types and modes of delivery (Shute, 2008) but do not address the purpose of the feedback or how the student may respond. The issue of moving students from considering feedback purely from a motivational or informational stance (Hattie & Timperlay, 2007), which is typical of high school experiences, to becoming more self-aware of their learning, was recognized as challenging during discussion. The consideration of strategic timing aligns with recommendations for sustainable feedback practices (Carless, Salter, Yang, & Lam, 2010) where the provision of formative feedback over an extended period of time, and engagement in dialogue prior to a subsequent but related task, may promote student awareness and self-evaluation of their learning.

Sustaining engagement of first
year students in self-directed learning based on critical thinking The current situation: Presentation of data from the outcomes of the pilot implementation of the project highlighted that students possessed low motivation in engaging in self-regulated online activities that challenged their thinking.
A possible direction: During subsequent discussion, strategies for sustaining engagement of students in active-learning exercises outside the classroom all involved linking the self-regulated activities to future summative assessment. The general feeling was that it is important to make the purpose of the activities explicit to students through some form of explanation (written, tutor-delivered or peer-dialogue) and/or detailed assessment criteria (rubrics).

The future directions of the project
Two key themes arose during discussions that will inform the future activities in this project: timing of formative feedback and further exploration of strategies to encourage student engagement in selfregulated activities. In regard to the latter, a recent thematic analysis of literature (Evans, 2013) highlights the complexity of self-regulation as a construct and considers how encouraging students to seek and use feedback to develop self-awareness skills requires multiple strategies and not single interventions. It is the authors' view that self-awareness skills are critical for first year students in transitioning to tertiary studies and so project activities will explore provision of ongoing feedback with scaffolding along with dialogue to enhance student self-monitoring and evaluation. Increasing the value and relevance of feedback to future tasks and shifting the balance away from passive delivery of "expert" feedback will also be trialed to encourage students to recognize different forms of feedback. Evaluation of how students respond to these strategies will be a key focus in the next stage of this project. article do not necessarily reflect the views of the Australian Government Office for Learning and Teaching.