Identification of at-risk students and strategies to improve academic success in first year health programs . A Practice Report

The transition to university is a difficult process for many students, having a negative impact on their academic performance, ultimately resulting in failure or withdrawal from one or more courses in their first semester. This practice report describes a profile analysis and readiness assessment designed to identify students at high academic risk. Students so identified were offered additional workshops to address assumed knowledge and academic skills. Attendance at the workshops correlated with improved academic outcomes. Please cite this practice report as: Pearson, A., & Naug, H (2013). Identification of at-risk students and strategies to improve academic success in first year health programs. A Practice Report. The International Journal of the First Year in Higher Education, 4(1). 135-144. doi: 10.5204/intjfyhe.v4i1.152 This practice report has been accepted for publication in Int J FYHE. Please see the Editorial Policies under the ‘About’ section of the Journal website for further information. © Copyright of practice reports is retained by authors. As an open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings. ISSN: 1838-2959 Identification of at-risk students and strategies to improve academic success ... A Practice Report 136 | The International Journal of the First Year in Higher Education, 4(1) April, 2013 Introduction The transition to university has historically been a poorly supported and difficult process, yet the first year experience at university is a critical time for establishing sound patterns of study and academic engagement (James, Krause & Jennings, 2010, p. 72). Many universities are working diligently to improve their students’ transition. Griffith University is committed to providing students from diverse social and cultural backgrounds with an excellent university experience in order to promote student success. For the majority of students, attrition is detrimental, in that they fail to succeed in an academic journey that would have realised considerable social and economic benefits had they completed their degree (Spence, 2012, p. 1). Student engagement is a wellestablished key contributor to student success: Students who are engaged and who learn to be successful early in their university studies are more likely to achieve and persist with their studies (Krause & Coates, 2008, p. 495; Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008, p. 542). Sustained engagement hinges on students’ early success and sense of capability in their studies (Lizzio & Wilson, 2004, p. 2), presenting a challenge for students who may be academically unprepared or educationally disadvantaged. This challenge is particularly acute with studies in science-related courses, which rely heavily on previous foundational knowledge (Dickson, Fleet & Watt, 2000, p. 60). Students disadvantaged educationally, socially and financially are at-risk of failure and withdrawal and higher education institutions have a responsibility to provide the necessary learning environment for the engagement of all commencing students (Coates, 2005, p. 26), including at-risk students. Identifying and monitoring at-risk students within large first year cohorts can be problematic without a cohesive process embedded into the curriculum. As described by Kift, Nelson and Clarke (2010), “from multiple starting points, all students are on a journey to becoming self-managing or selfdirected learners and the first-year curriculum must help get them there” (p. 3). Early identification and intervention procedures for at-risk students are essential, as the first major assessment task for the courses described here are in weeks 5-6. While early, low-stake assessment tasks are important for student self-regulation of learning, they can improve or diminish students’ sense of capacity. To avoid the latter, an immediate follow-up with specifically tailored support processes are required to give practical and relevant academic assistance. In this Practice Report, we describe an effective at-risk student identification procedure in the form of a readiness assessment and an effective support process in the form of voluntary workshops which are available to the students from week 2. Background and context At Griffith University, students entering a range of health programs undertake three common courses in their first semester—a physical science course and two life science courses. These courses cater for a large number of students with diverse academic abilities. In 2012, there were 941 students enrolled in at least one of these semester one courses. One of the challenges in presenting a common year of essential knowledge in physical and life sciences is identifying


Introduction
The transition to university has historically been a poorly supported and difficult process, yet the first year experience at university is a critical time for establishing sound patterns of study and academic engagement (James, Krause & Jennings, 2010, p. 72).Many universities are working diligently to improve their students' transition.Griffith University is committed to providing students from diverse social and cultural backgrounds with an excellent university experience in order to promote student success.For the majority of students, attrition is detrimental, in that they fail to succeed in an academic journey that would have realised considerable social and economic benefits had they completed their degree (Spence, 2012, p. 1).Student engagement is a wellestablished key contributor to student success: Students who are engaged and who learn to be successful early in their university studies are more likely to achieve and persist with their studies (Krause & Coates, 2008, p. 495;Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008, p. 542).
Sustained engagement hinges on students' early success and sense of capability in their studies (Lizzio & Wilson, 2004, p. 2), presenting a challenge for students who may be academically unprepared or educationally disadvantaged.This challenge is particularly acute with studies in science-related courses, which rely heavily on previous foundational knowledge (Dickson, Fleet & Watt, 2000, p. 60).Students disadvantaged educationally, socially and financially are at-risk of failure and withdrawal and higher education institutions have a responsibility to provide the necessary learning environment for the engagement of all commencing students (Coates, 2005, p. 26), including at-risk students.Identifying and monitoring at-risk students within large first year cohorts can be problematic without a cohesive process embedded into the curriculum.As described by Kift, Nelson and Clarke (2010), "from multiple starting points, all students are on a journey to becoming self-managing or selfdirected learners and the first-year curriculum must help get them there" (p.3).
Early identification and intervention procedures for at-risk students are essential, as the first major assessment task for the courses described here are in weeks 5-6.While early, low-stake assessment tasks are important for student self-regulation of learning, they can improve or diminish students' sense of capacity.To avoid the latter, an immediate follow-up with specifically tailored support processes are required to give practical and relevant academic assistance.In this Practice Report, we describe an effective at-risk student identification procedure in the form of a readiness assessment and an effective support process in the form of voluntary workshops which are available to the students from week 2.

Background and context
At Griffith University, students entering a range of health programs undertake three common courses in their first semester-a physical science course and two life science courses.These courses cater for a large number of students with diverse academic abilities.In 2012, there were 941 students enrolled in at least one of these semester one courses.
One of the challenges in presenting a common year of essential knowledge in physical and life sciences is identifying those students who are in need of extra assistance.One strategy that universities are implementing involves the delivery of diagnostic tests in order to identify at-risk students.This allows targeted remedial assistance and informs university staff of students' actual abilities (Barry & Chapman, 2007, p. C37).
In an effort to improve student success and retention, we needed a procedure for early identification of students at risk of failure in or withdrawal from semester one courses.The purpose of identifying students early is to direct appropriate students to early intervention workshops which cover assumed knowledge and academic skills.Within our large interdisciplinary cohort of first year students, only two-thirds of students entered in 2012 with an OP1 score.As such, student OP score alone was not sufficient in the early identification process.Submission of the first assessment task (Nelson, Duncan & Clarke, 2009, p. 4) was also insufficient, as the first assessment tasks in each course were in compulsory labs where the non-submission rate was only 4.7%.Failure on mid-semester exams was also considered as a marker.However, mid-semester exam results are not available until ~week 8, by which time the workshops had concluded and it was considered too late for optimal strategic intervention.
In this Practice Report, we evaluated (i) student OP; and (ii) student performance on a readiness assessment as predictors of success in first year university courses within the discipline of health science.In this report, academic success was determined by student performance in three large semester one health courses.Our findings show that both student OP scores and performance on a readiness assessment were strong markers for identifying students at high risk of failure or withdrawal.However, as one-third of students did not have an OP score, the readiness assessment score was more comprehensive in identifying at-risk students.

Results and discussion
For this report, student success was measured for each student by their pass rate in the three core semester one courses.
Non-passing grades were classified as either fail (i.e., students who sat the end of semester exam and failed the course) or withdraw (includes students who did not sit the end of semester exam and students who withdrew from the course).For all students across the three courses the fail rate was 14.6%, the withdraw rate was 7.0%, giving a combined fail and withdraw rate of 21.6%.

Analysis by OP score
A linear regression was performed on student OP score and the fail and withdraw rate.There was a highly significant correlation (p = 5 × 10 -33 , r = 0.516, N = 461).The large number of students contributes to the p value; whereas the r value indicates the strength of the relationship.Figure 1 shows the mean fail and withdraw rate for students at each OP score, with 95% confidence intervals.The R 2 value of 0.9172 reported in Figure 1 is higher than the r value of 0.516 reported above, as the data presented in Figure 1 is Students with an OP score of 12 or more had a markedly higher combined fail and withdraw rate (57%) than students with OP scores of 1-11 (11%).Of the students with an OP score, only 17% had an OP score of 12 or more, yet they accounted for 51% of the fails and withdrawals.This correlation between high school score and tertiary achievement has been reported by others and is particularly true in sciencerelated courses (Dickson et al., 2000, p. 60).In addition, a poor high school score may diminish a students' sense of capacity before commencing university, which highlights the importance of capacity building interventions, in the form of workshops, such as those described in this present analysis.

Readiness assessment
In 2012, a readiness assessment was given to all students in a week one laboratory session.The week one quiz consisted of 16 maths questions at a Year 10-12 level (age 15-17) on scientific notation, conversions, algebra, and quadratic equations.Students were not permitted to use calculators.The aim of the quiz was to determine the level of assumed knowledge, i.e., mathematics skills that are not taught in the semester one courses, but are required to solve many processes, especially in the physical science course.A linear regression was performed on student OP and quiz scores.There was a highly significant correlation (p = 2 × 10 -27 , r = -0.501,N = 405), indicating that the quiz is a valid measure of the ability of students entering these programs.Figure 2 shows the mean quiz score rate for students at each OP score, with 95% confidence intervals.Mean quiz score

Student OP Score
A linear regression was then performed on quiz result and the fail and withdraw rate.There was a highly significant relationship (p = 8 × 10-20, r = -0.344,N = 653).Figure 3 shows the mean fail and withdraw rate for each quiz result, with 95% confidence intervals.Students who performed well on the quiz had lower fail and withdraw rates than students who performed poorly on the quiz.This shows that the quiz is extremely valuable in identifying in week one the at-risk students that would benefit from additional support.A linear regression was then performed on quiz result and the fail and withdraw rate.There was a highly significant relationship (p = 8 × 10 -20 , r = -0.344,N = 653).Figure 3 shows the mean fail and withdraw rate for each quiz result, with 95% confidence intervals.Students who performed well on the quiz had lower fail and withdraw rates than students who performed poorly on the quiz.This shows that the quiz is extremely valuable in identifying in week one the at-risk students that would benefit from additional support.
It is not surprising that performance on the quiz correlates with OP score, as the quiz tests assumed knowledge.However, this correlation is further reflected in student performance in first semester courses, making the combination of OP score and quiz score a powerful predictor of academic risk (see Table 1).In addition, the level of assumed knowledge varied in the semester one courses.For example, one life science course presents human anatomy and physiology concepts which are completely new to most students, while the physical science course presents material that many students may have covered in high school.Regardless of the level of assumed knowledge in the course, both OP and quiz score had highly significant correlations with academic performance in all three courses.

Inter-course correlations
There were extremely significant correlations of performance across the three courses (p < 10 -160 , r > 0.8, see Table 1)-students who performed well in one course were generally performing well in other courses, while students who failed or withdrew from at least one course also struggled in other courses.This indicates that performance was correlated with student ability and time on task rather than course-specific content.Table 1 also shows the significant correlations between OP score, quiz results, and performance in the three courses.OP score was an excellent predictor of performance in all three courses (p ≈ 10 -60 , r ≈ -0.7).The negative correlation coefficient is due to the OP score, where 1 is the highest and 25 is the lowest.Whilst the quiz contained only maths questions, it was still an excellent predictor of performance in both life science courses (p < 10 -30 , r > 0.4) as well as the physical science course which contains considerable maths content (p < 10 -70 , r = 0.65).

intervention strategy -Essentials Workshops
Following the quiz in week one, students who scored less than 12 out of 16 were advised to attend the Essentials Workshops, though all students were eligible to attend the workshops.The workshops ran from weeks 2 to 7, with each workshop containing a session on either biology or maths, and a session on academic skills.A summary of the workshops is provided in Table 2.
The content of the workshops has been developed over several years, and has been designed to support students in areas that have been identified as contributing to poor student performance.The timing of the workshops was designed to give students the best preparation to succeed, for example, the biology and exam preparation workshops were held in the lead-up to the biology mid-semester exams.
The high tutor/student ratio (two tutors and ~15 students per class) in the workshops led to personalised learning, where the tutors would identify the needs of students and adapt the workshop accordingly.Student-student and studenttutor interaction was encouraged through learning experiences that involved collaboration and problem-solving in a supportive learning environment.As such, students felt confident to ask questions, talk in front of the class, and explain concepts to other students.
The majority of students (85%) did not attend any of the workshops.These students had a combined fail and withdraw rate of 23% (Figure 4, showing 95% confidence intervals).Attendance at four or more workshops saw this rate decrease to 5%.There was an even greater improvement in the combined fail and withdraw rate for the students who scored less than 12 out of 16 on the quiz (Figure 4).Of these students, the students who did not attend any of the workshops had a combined fail and withdraw rate of 30%, whilst the rate for students who attended four or more workshops was 8%.This is an excellent result for the students who did poorly on the quiz and then attended four or more workshops.The large drop in the fail and withdraw rate is not solely attributable to the workshops, as students who are motivated to do well at university spend more time on task (such as attending voluntary workshops), which is the strongest predictor of academic success (Wilson 2009, p. 4).

Identifying at-risk students
If the combination of OP score and quiz score were used in 2012, 37% of students would have been identified as at-risk.Half of these identified students went on to fail or withdraw from at least one of the common courses.More importantly, 66% of students who went on to fail or withdraw from a course would have been identified in week one.
Of the students identified as at-risk, 25% were identified based on OP, while 95% were identified based on quiz score.Of the students who failed or withdrew from a course that were not identified, only 14% of students had both an OP score of 1-11 and a quiz score of 12+.The remaining 86% of students were lacking either an OP score or a quiz score, or both scores.

intervention strategy
Given that there is a high fail and withdraw rate for certain groups of students and that risk factors have now been identified, there is the opportunity from 2013 to provide atrisk students with targeted support.The types of interventions proposed include (but are not limited to) the following:

•
The Essentials Workshops, which have been shown to be a valuable resource for students; these workshops will continue in 2013.

•
Following a trial in 2012, in 2013, Student Success Advisers (SSAs) will be embedded across Griffith University.These SSAs will (among other roles) run tutorials and provide learning support for at-risk students.
Between the Essentials Workshops and the support provided by the SSAs, there is a wealth of resources available to students.It is believed that online support such as maths tutorials is unlikely to engage many of the at-risk students.
The major challenge is to get the at-risk students to attend the additional workshops that are designed and provided for them.It has been very challenging trying to contact them, either through email or phone.Additional tutorials that were run in semester one of 2012 were mainly attended by high-achieving students.By presenting a summary of this data to students at the start of semester one, it may encourage more at-risk students to attend the workshops and either seek or be responsive to additional support.Ensuring the SSAs have a visible presence through attendance at lectures and labs, particularly in week one, but following up throughout the semester, should also lift student attendance at workshops.We believe that motivation is the key to reaching students at-risk of failure and withdrawal, and is central to the role of the SSA.

Summary
Each year, students with a wide range of academic abilities enter health programs at Griffith University.Through a combination of OP analysis and a readiness assessment, we have demonstrated that many students at high risk of failure or withdrawal can be identified in week one.Our data indicate that an active intervention strategy consisting of workshops, and learning assistance will help reduce the fail and withdrawal rates of these at-risk students, with the aim of improved academic success and increased student retention into semester two.

Figure 1 :
Figure 1: Correlation of OP score with combined fail and withdraw rate in semester one courses

Figure 2 :
Figure 2: Correlation of OP score with mean quiz score

Figure 3 :
Figure 3: Correlation of quiz result and combined fail and withdraw rate for semester one courses

Figure 4 :
Figure 4: Attendance at the workshops correlated with lower fail and withdraw rates in semester one courses