We are in a transitional time for online learning at traditional higher education institutions- passing from a period of majority resistance to online learning and into a period of either acquiescent or enthusiastic acceptance of online learning as an inevitable disruption to traditional practices (Selingo, 2013). Generational, social and economic drivers are prompting more higher education administrators and faculty to offer traditional programs using online and hybrid models. Just as the approach to online course development requires more front-end comprehensive planning and development than traditional face-to-face courses, program level design requires a more systematic approach to cross course consistency and reliability. Pilot testing for usability and feasibility is crucial to the design process for online programs and introduces a unique set of challenges and opportunities to improve student learning and experiences.
This session will share the design, experiences, interview and survey feedback, and learning analytics gathering process from a pilot conducted for a new online doctoral program, the first online doctoral program of any kind at Emory University and the first online version of this particular degree to be nationally accredited.
In 2013, Emory University's Candler School of Theology decided to revive the Doctor of Ministry (DMin) degree program that had run for over twenty years before it was discontinued in 1995 because of financial pressure, declining enrollment and demands on faculty time. In response to a steady stream of requests to re-institute the degree, the leadership decided to revamp the program as a 90% online degree.
By taking this innovative step, Candler stands out at Emory University because the DMin program will be the first 90%online doctoral degree program on campus. This DMin degree is also the first of its kind to be accredited by the Association for Theological Schools, which approves degree programs for its 270 member association of "post-baccalaureate professional and academic degree programs to educate persons for the practice of ministry and for teaching and research in the theological disciplines." Furthermore, before making the decision to offer a new online program, Candler's faculty had only just begun to experiment with online learning components and had never offered even one class fully online. After an active year of faculty and administrative collaboration and program design and development, we knew a pilot test of the program was essential to ensure a program launch geared towards faculty and student success.
To gather usability and feasibility data from an administrative, faculty and student perspective, we took the following approach:
- Designed a pilot course to include all activities and tools of a typical week of program courses in the pilot course
- Recruited faculty and administrators who would be active in the program in the first year to help build and deliver the pilot course
- Offered the pilot to a volunteer subset of newly admitted students to the program six months prior to the program's start
- Developed learning analytics based on learning management system log data to systematically examine student experience patterns and optimize program and course level design
- Administered a survey to and conducted follow-up individual and focus group interviews with all pilot participants, including administrators, students and faculty
- Analyzed the data for design-related program implications and to inform programmatic student support infrastructure, student orientation, administrative training and faculty development
In this session, we will share experiences and results from this pilot program and ask for audience input of past experiences, reactions and suggestions. Using a collaborative digital tool, we will invite audience members to actively engage in producing a session debrief towards identifying successful strategies for implementation and evaluation of online program pilot testing. We and the audience should leave this session with a grounded understanding of programmatic level pilot design, usability data gathering techniques and cross-referencing learning analytics to inform design.
Program Track: Learning Effectiveness
Target audience: All audiences
Institutional Level: multiple
Type of session: Information Session
Media: Digital projector, wireless access