Designing and evaluating an empowering online pedagogy for commencing students : a case study

Realising the potential for commencing students to succeed at university depends on designing a pedagogy that not only engages students in learning but also encourages their reflection on that learning. This guiding philosophy provided the impetus for a course that also needed to accommodate challenges emanating from a very diverse student cohort, a program decision to switch from an on campus to an online teaching mode, an inter-disciplinary and collaborative program emphasis and the complexities stemming from change forces currently impacting on the Australian higher education sector. These forces included changes in pedagogy, curriculum, assessment, academic identity, technology, research-informed learning and student and stakeholder expectations. This paper documents the design, development, delivery and evaluation of a first semester, first year undergraduate nursing course conducted in the Nursing Program at the University of Southern Queensland. The course integrates an engaging learning philosophy while simultaneously embracing new directions in higher education to empower commencing students.


Introduction
This paper will focus on the design, development, delivery and evaluation of a first year undergraduate nursing course conducted by the Department of Nursing and Midwifery at the University of Southern Queensland (USQ).The course has been conducted since 2006 and is one of two courses designed to assist students to develop the literacies and skills they need to succeed as learners in their higher education (HE) studies and as nursing professionals: Building Professional Nursing Attributes A (CMS) and Building Professional Nursing Attributes B (MAT).The aim of CMS is to develop students' academic and information literacies and learning, research, communication, interpersonal and team work skills as well as assisting them to begin their professional e-portfolios.Its companion course, MAT, is charged with the responsibility of developing students' numeracy and computing skills directly linked to their degree and to their later professional practice.The rationale, design and delivery and evaluation of CMS andMAT (2006-2011) have been documented (see Lawrence, Loch & Galligan, 2008;Lawrence, Loch & Galligan, 2010).In 2012, however, USQ's nursing program was offered for the first time in an online mode, in some cases augmenting its on campus delivery and in others replacing it.
The paper first describes the rationale underpinning the general nursing program including its theoretical perspectives.It then documents the changes in CMS's academic practices made in its conversion to online delivery.Thirdly, the paper outlines the evaluation methodology before summarising the two perspectivesstaff and studentused to assess the effectiveness of CMS.Finally the lessons learnt and improvements made are reviewed.

Perspectives underlying program design
The rationale for the academic practices underpinning the new flexible and online nursing education program continued to include design features from the 2006-2011 nursing program.The program was developed in inter-disciplinary collaborations between the Nursing Department, the Faculty of Sciences (mathematics and computing skills), the Faculty of Arts (academic literacy and communication skills), Learning and Teaching Support (LTS) (pedagogical reinforcement and learning and teaching guidance), The Australian Digital Futures Institute (ADFI) (online pedagogical advice) and the library (information literacies).
The impetus for curriculum design decisions since 2006 was the diverse nursing cohort.This cohort includes mature age students as well as school leavers, international, domestic and disadvantaged students such as low-socio economic, indigenous and rural and isolated students as well as second year students completing an accelerated program as Assistants in Nursing (AIN) or having completed a Technical and Further Education (TAFE) College nursing qualification.Student diversity influenced curriculum design in terms of its interdisciplinary approach, embedded practices and emerging HE initiatives.

Interdisciplinary collaborations
The nursing program team considered that the inter-disciplinary approaches adopted were more able to address diverse students' needs.Inter-disciplinary curriculum describes an integration of multiple disciplines to respond to modern working patterns, which increasingly call for multi-disciplinary team work, and challenges arising from the 21st century demand for inter-disciplinary solutions (Woods 2007).The nursing program uses a combination of disciplines approach to meet an area of common concern (Davis & Devlin 2007).
Both CMS and MAT have the responsibility for instilling the interdisciplinary whole-ofprogram approach, role modelling the holistic focus essential to students' transition to university and effective nursing practice.The courses replace de-contextualised, 'bolt-on' skills courses and disparate and 'piecemeal' efforts to support commencing students (Krause et al., 2005), where engagement and retention are left to chance.Crossing these interdisciplinary boundaries thus enables the courses to constitute an academic and social 'organising device'the glue that holds knowledge and the student experience together.

Embedded and scaffolded practices
Another design decision implemented to embrace student diversity relates to embedding and scaffolding (Pea, 2004) key university literacies.This design impetus stems from Keimig's (1983) model of learning improvements, a model confirming that generalized approaches to skills courses are less likely to be effective than those targeted at specific aspects of learning within academic courses where the need for knowledge or skill becomes apparent.Keimig also proposed hierarchical levels of support to provide for the total learning requirements of students including their needs and attitudes.These insights underpin CMS and MAT.
Critical literacy also supports the integration of embedded and scaffolded approaches in CMS and MAT.Critical literacy (Cope & Kalantzis, 2000;Fairclough, 1995;Kirkpatrick & Mulligan, 2002) theorises that university is like a culture made up of a range of literacies, discourses and cultural practices.Students' transition is then recast as a process of gaining familiarity with and demonstrating these new literacies (Lawrence, 2005).Commencing students, for example, need to rapidly master faculty, school and subject discourses, including academic, library, numeracy, research, information, administrative and technological literacies as well as the more personal literacies also crucial for success, including communication, interpersonal, stress and time management and financial literacies.These sit alongside new teaching and learning styles and a plethora of unfamiliar cultural practices.Nursing students are also required to demonstrate (in assessment) a number of disparate discipline literacies; biological, chemical, computing, nursing, communication, research (both quantitative and qualitative methodologies) and information literacies amongst others.
Their capacity to master key literacies however depends on students' awareness of the skills and knowledge they bring with them to higher education (HE).Critical literacy is again helpful as it encompasses Bourdieu's (2001) view of capitals: that when students access university they do so with various levels of academic, linguistic and socio-cultural literacies which need to be understood before students are able to master the literacies they confront.Both the institution/staff and students have responsibilities in this process.Kift (2009, p.1), from the First Year Experience (FYE) argues for 'transition pedagogy': ...the curriculum and its delivery should be designed to be consistent and explicit in assisting students' transition from their previous educational experience to the nature of learning in higher education and learning in their discipline as part of their lifelong learning.The first year curriculum should be designed to mediate and support transition as a process that occurs over time.Devlin (2011) uses the notion of socio cultural incongruence to conceptualise the differences in cultural and social capital of diverse students, for example from low SES backgrounds, and the high SES institutions in which they study.Embedded and scaffolded learning experiences, based on an assessment of the skills and abilities students bring with them, assists students to deal with the tacit expectations inherent in university practices.Devlin (2011) suggests that such an intentional design of learning, teaching and assessment acknowledges the reality of the contemporary student context and seeks to mediate student diversity in relation to the preparedness and cultural capital of commencing students.It is important, in addition, that students accept their own responsibility in this process by not only reflecting their own skill base but also building their capacities to develop these literacies throughout their studies.Lawrence (2005) reports that using communication skills like asking for help, making social connections and expressing disagreement enables students' transition and also need to be made explicit and embedded in curriculum design.

Emerging higher education and research initiatives
The twenty first century has witnessed a range of HE initiatives like a student focused curriculum (Kember, 2009) and the FYE (Nelson, Duncan & Clarke, 2009;Tinto, 2009) but there are also the change forces emanating from changing assessment needs, principally the implications of the new Threshold Learning Outcomes (TLOs) proposed as part of the Tertiary Education Quality and Standards Agency (TEQSA) quality assurance processes, and academic identity, for example, stemming from the sector's move to integrate graduate and work-based qualities and skills (Barrie, 2006;Bridgstock, 2009).Other change forces are those generated by communication technology, in this instance, helping students master and demonstrate the rapidly evolving range of technological literacies demanded by both HE and the profession (Reushle, McDonald, & Postle, 2009); recent research-informed learning, especially important for nursing students who need to manage changing professional development needs; and student and other stakeholder expectations, including professional demands for effective interpersonal and team work capacities and the political pressures regarding nursing accountability and scope of practice (Lawrence et al., 2008).
These new directions and change forces influenced CMS pedagogy, mainly in relation to technological engagement and e-learning.Forum discussions and e-tivities (see next section) were incorporated to engage students.Learning Management System (LMS) forums were used to facilitate student engagement.One of the potential strategies for measuring the attainment of TLOs, for instance, is a portfolio.CMS incorporates a professional e-portfolio as an assessment item, its mechanics taught in MAT.The question of academic identity in relation to the sector's move to integrate graduate and work-based qualities and skills is addressed by both the e-tivities and an assessment item in the e-portfolio where students are asked to reflect about their development of a graduate quality.Research informed learning is also catered for by CMS's development of information literacy, for example in relation to the online databases, while the question of stakeholder expectations is similarly managed by a portfolio reflection item addressing students' application of a nursing code of practice.
While these emerging directions informed CMS design decisions, its development benefitted specifically from recent research into online pedagogy (see Reushle, McDonald, & Postle, 2009;Salmon, 2011).The next section will describe CMS's approach.

Curriculum development
The carpe diem process (see Salmon, 2011) was used to redesign CMS online.Carpe diem provides a structured framework for course teams to design and implement e-learning designs grounded in discipline practice.It provides ways of exploring a variety of resources and lowcost, high-impact technologies to deliver a course online.While this allowed participants to become more skilled in the use of technology, they did so in the process of addressing a pedagogical design challenge.The interdisciplinary team, a learning technologist, librarian and other nursing team members, provided input and support during the process.
The carpe diem process consists of a preparation session, a 2-day facilitated workshop and a post-session used for review and re-design, if required.During the 2-day workshop, the facilitator led the team through a number of collaborative tasks including blueprinting, story boarding, team working, scaffolding of networked learning (using Salmon's 5-stage model), development of e-tivities, peer reviewing, aligned assessment and follow up actions.
Once the mission for CMS had been established, learning outcomes were articulated by the course team followed by a brainstorming exercise to determine, at a general level, the content related to each of the objectives of the course.The next step involved the participation by peers (members of the nursing team not already participating as well as critical friends) to provide feedback about the proposed e-tivities.The feedback offered by these 'reality checkers' was helpful in ensuring the activities, posts and links to assessment were studentcentred, engaging and appropriate to the anticipated level and abilities of the students.The reality checkers continued to assist the design process as their feedback was sought on the various iterations of course design, including feedback on the LMS components, for example the use of web pages, forum posts and stimulus activities.

Curriculum delivery
CMS's content was significantly reduced.There are no study modules, no textbook and no selected readings.Instead there are weekly video-lectures and e-tivities (see Figure 2).Each weekly e-tivity comprises formative assessment, explicitly linked to one of the assignments.Each student is placed in their own forum group (n=20) variously named the Resuscitators, the I V Leaguers, the Betta-Blockers, etc with an accompanying online tutor.The online tutors responded to students' posts either individually or through summaries.

Assessment
CMS assessment includes two assignments and forum participation.Assignment 1 includes four tasks building on the e-tivities: a reflective paragraph on the students' learning strengths and areas for improvements as well as specific strategies to achieve this improvement; a paragraph reflecting on an interview conducted with a fellow student; an academic writing exercise including thesis statements, main points, referenced paragraphs and reference lists (on a clinical aspect of hand washing); and an article selection (on a clinical aspect of hand washing) and evaluation exercise.Assignment 2 consists of the e-portfolio and includes reflective paragraphs on students' semester 1 experiences, their development of a nursing competency or code and a graduate quality and their capacity for technological engagement.

Methodology
The methodology included continuous evaluative processes which were applied throughout the design, delivery and evaluation of the program.The methodology used a standard method of evaluation and program development (Taylor & Galligan, 2002, developed from Guba & Stufflebeam, 1970) and includes both quantitative and qualitative data collection techniques in each cycle (see Table 1).
Two perspectives are investigated: staff and student perspectives (documented in Table 1).While these evaluation cycles began in 2006, in terms of the online design, two cycles have been undertaken: in 2012 in semester 1 (n = 365) and semester 2 (n = 153).Second and third year student cohort surveys (yet to be undertaken for the 2012 cohort) 12

Item
One-on-one sessions with students    Table 1: Relationships between the evaluation strategies and stages in the course design Within the study, a longitudinal phenomenological approach (Jeffers, 1998) is taken to understand the lived experiences of students as they become more academically prepared.This evaluation methodology is reiterated during each offering.

The staff perspective
In the initial design stage, the design team took advantage of their experiences in previous development of curricula for nursing students.In each stage, as well as in the reiteration of these stages, regular meetings are held to develop, review, moderate, and fine-tune the academic practices (Item 2).Ongoing feedback is also obtained from the leaders of the first year nursing courses during delivery and evaluation stages.Debriefing meetings with nursing lecturers are conducted throughout the teaching cycle and in the debriefing and pre-planning stages for the next course cycle (Item 3).Peer review is also undertaken (Item 4).Reality checkers and online tutors also provide feedback (Item 5).

The student perspective
In the pre-program stage, use was made of the extensive amount of feedback that had been collected from students in previous research studies (Item 6).During the delivery stage, continuous evaluation included unsolicited feedback presented in emails, forum discussions (Item 10) and intensive one-on-one sessions (Item 12).Two independent formal student evaluations are also conducted at the conclusion of each cycle: one designed to address program specific issues (Item 8); and the other, the standard university course quality survey (Item 9).Continuous feedback is obtained, principally from the e-portfolio assignment (Item 7).Students are asked to reflect about their learning and academic skills as they bridge the divide between their pre-university school and employment (both nursing and casual work) contexts (many are mature-age students) and their university and nursing contexts.The eportfolio, for example, includes an online personality, learning approach and learning style questionnaire that generates feedback for students to reflect about (see Figure 1).

Findings and discussion
This section will explore the online components of CMS pedagogy.The continuous processes at the heart of the methodology revealed both positive and negative findings.

Staff perspective
In terms of course design, staff agreed that the carpe diem method assisted the design process in many ways.Comments confirmed that the story board components helped 'strip' the content away from the course and renew a focus on students' learning outcomes.Staff responses also revealed that learning activities had been appropriately and that they closely aligned with assessment.Staff further perceived that students' forum posts, embedded as formative assessment in the e-tivities and formal assessment, ensured that student feedback was integral to course design.CMS was conducted again in semester 2 with this feedback integrated, mainly relating to LMS organisation and structure.Overall, the staff considered that the carpe diem process had a positive impact about how they viewed course design process and the course itself.However staff testimony also called for improvements mainly in forum organisation and participation.Although the forum groups were small enough (n=20), the large number of students generally made forum marking cumbersome.The wiki included in one e-tivity did not work well and was dropped in semester two.It is important to ensure that the feedback loops and training sessions for online tutors/markers are sustained.

Student perspective
The diversity of students' background knowledge and digital experience was apparent in the students' evidence.International and mature age students were often confronted by completely new literacies: Being an international student we have to face different studying environment.When I started my study in USQ I was not good at the digital world.I never used computers for my studies before.It is funny but I was really shocked when I saw so many computers in USQ (portfolio reflection).
Mature age students found the online environment too much in an otherwise busy world: For me using forums was something I had never done before so I avoided the notion from the beginning but mainly it was a culmination of things.Personally, I was working full time, looking for a house, travelling sometimes 2 hrs a day for work and uni and the online subjects were the ones that suffered for me as they were easier to "forget" about (portfolio reflection).
An issue of concern was students' lack of understanding of the importance of the online environment as a key to accessing all material and communication.There is also inconsistency with the study desk that is being tackled at program and institutional levels: Functionally: there were so many different "levels" to the moodle system, I initially found it hard to navigate.Then keeping track of the forums took me hours (which i didn't have).With everything else going on i just put it in the too hard basket and just focused on the tasks i found easier.
However there were positive findings in relation to online flexibility and accessibility: A benefit of online delivery is I enjoy the option to study in the comfort of my home as well as having the online support from the lecturers (portfolio reflection).
Initially i was confused but now i have felt that online learning is actually beneficial.It provides team learning and there are no geographic barriers for the students.We are able to access the course from anywhere and every student can post their opinions.I am really very happy to experience online learning (forum post).
Distance students reported that online delivery reduced their feelings of isolation, thus having a positive impact on their retention (Kift 2009).
For me technological engagement in the course has made my dream to study nursing a reality and has removed any concern I had about feeling isolated as a distance student (survey response).
CMS students revealed their initial reservations about CMS being online and about their participation in the forums.In many cases these fears dissipated.One student expresses her fears about online learning and how she overcame them: I was a little apprehensive about committing to online learning, having always been in face-toface classes.By completing the weekly e-tivities and tasks, I have learned how to participate in an online conference, I have highly tuned my academic writing skills, learned to use databases for research and experienced computer marked assessments.On reflection, I believe that online learning is the best way for me to complete my education, as it offers extreme flexibility in respect to 'class time'.The personal aspect of learning is not entirely lost as there is quite a bit of engagement with others in the forums.After completing the survey, I can safely say that my digital literacy skills have definitely improved.Overall, I am much more confident with online learning.
The forums were off putting for quite a few students who gradually felt more at ease: At first forums took me out of my comfort zone.Since it was a requirement I was eased into making regular contributions as well as reading those of others.I soon became used to it and also found a lot of information that was very helpful and still do (portfolio reflection).

I would probably have felt a little bit worried I'd write something incorrect or silly, but CMS has given me the confidence to write on the appropriate forums regarding class matters (survey).
At first I was hesitant to post things onto the forum as I was afraid that I would post the wrong thing, or be criticised for what I posted.Then I saw that there was no "right or wrong answer" and i became more confident in my postings (forum).
Conversely evidence suggests that forums and e-tivities increased student engagement: For me the forums have also been an excellent way to interact with fellow students through the sharing of opinions and feedback.It made me feel like I was learning collectively with other students, much like a classroom situation (portfolio reflection).
The use of short e-tivities and YouTube clips, particularly in the CMS1008 course, has provided a positive experience for me because of the variety, which tends to keep my attention (forum post).
The teaching approach incorporated a variety of stimulating activities which made it an enjoyable and effective stepping stone toward achieving my dream (forum post).
The reflective process also helped students to gauge their progress.A digital literacy survey conducted online reaffirmed to students just how much they had progressed1 .Two major issues emerged from the data however: the non-engagement of some students with the e-tivities and online forums; and its corollary the propensity for some students to only attempt the assessment.A small minority of students in both semesters did not participate in the e-tivities, despite the allocation of marks and assessment links: in s1 (n=33 or 9% with a grade average of 59.8%) and s2 (n=20 or 13% with a grade average of 62.75%)2 .Students explained that they couldn't manage the responsibility or self initiative entailed in online delivery and/or felt disengaged by the need to participate in the forums: I really don't enjoy using online forums as i don't like sharing my opinions online that's why i have really struggled with CMS, i would prefer it to be an face to face class rather than online, it is not on my timetable as it is online so i really have to try hard to actually remember that CMS is still there.
When the assignment came I thought I would be fine.I did not think that missing a few postings would be a big deal.I was very wrong.For the tasks I had done forum postings for, I did not have an issue but as I got deeper into the assignment I found it harder and harder.I regretted not doing the tasks (email).
This evidence confirms Morris' (2005) findings that in terms of online engagement, unsuccessful students were far less active in participation than successful students.Poellhuber and Karsenti (2008) argue that online engagement may be a particular problem for some students and requires more investigation.The CMS experience confirms this finding.That such non participation could be tracked and positively correlated with assignment failure and/or lower grades is a positive step forward however.That this tracking also enabled discussions with the students concerned was a positive consequence and used to empower students the following semester.

Room for Improvement and Future Research Directions
The case study paints a generally positive picture of CMS's conversion to an online course designed for a diverse first year cohort.It also reveals ongoing constructive feedback from staff and students.For a minority of students online engagement remains problematic although CMS design enables this to be tracked and confronted explicitly.However it remains a concern to be addressed each offering.Longer term there needs to be ongoing communication with nursing staff to ensure the CMS is delivering skills transparently.Future directions include the continuation of the longitudinal study.For instance, feedback from students who took the first offers of CMS in 2012 will need to be sought in their second or third years.This feedback will assist in ascertaining whether or not CMS is successful in delivering an empowering online pedagogy for commencing students.

Figure 1 :
Figure 1: Sample of an e-tivity Learning management system CMS uses USQ's LMS to generate its flexible e-learning environment.The material made available through a multi-modal in-house content creation system (i.e.print, CD and available on the web through the open source Moodle LMS).Assessment and other resources are made available online through the Moodle site (Study desk).This includes multimedia material and the interactive discussion forums.All assessment is submitted and marked electronically.The e-portfolio assignment uses the Mahara platform.
Since starting in July I have participated in a WIMBA session, produced a PowerPoint presentation with voice over, started a fabulous E-portfolio, got valuable feedback from forum postings and submitted assignments via EASE.Imagine what I am going to achieve in the next two and half years if I have done all this in just 8 weeks!(forum post).