As the demands for public accountability increase for the higher education industry, institutions are seeking methods for continuous improvement in which they may demonstrate the level of quality within educational programs. Because of the rapid growth of online programs, institutions are further called upon to demonstrate that quality education is being delivered to students at a distance. This study created such a method to provide institutions offering online education an instrument for assessing quality within their programs: a quality scorecard for the administration of online education programs.
A six round Delphi study was undertaken with 43 seasoned administrators of online education programs from a variety of institutions in higher education. This panel of experts agreed upon 70 quality indicators that administrators of online education programs should examine within their programs to assess quality. A method for scoring was also developed. The original set of 24 quality indicators from the Institute for Higher Education Policy study, Quality on the Line: Benchmarks for Success in Internet-Based Distance Education (2000) were used as a starting point and were determined to be still valid in 2010, but with modifications (wording, some were divided, and some were combined). Forty-five additional quality indicators were added which resulted in a quality scorecard which provides industry agreed upon standards for online education programs to use for quality assessment. Each quality indicator has a potential range of 0-3 points with a perfect score on the scorecard resulting in 210 points.
The purpose for this study was the development of a scorecard to measure and quantify elements of quality within online education programs in higher education. The scorecard is an easy-to-use tool for online administrators to use for program evaluation. By evaluating each of the respective quality indicators within the established categories, an online administrator can determine strengths and weaknesses of their program. The identification of the weaknesses can be used to support program improvement and strategic planning initiatives. The scorecard could also be used to demonstrate to accreditors elements of quality within the program as well as an overall quality level.
A scorecard of 70 quality indicators was developed by a panel of experts with 86% of the panel have 9 or more years of experience Each indicator is worth up to three points. The administrator will determine at what level their program meets the intent of the quality indicator:
A perfect score = 210 points.
90-99% = 189-209 - Exemplary (little improvement is needed)
80-89% = 168-188 - Acceptable (some improvement is recommended)
70-79% = 147-167 - Marginal (significant improvement is needed in multiple areas)
60-69% = 126-146 - Inadequate (many areas of improvement are needed throughout the program)
59% and below = 125 pts and below - Unacceptable.
While there are rubrics being used to assess quality in online course materials, such as Quality Matters, until now, there was not an industry agreed upon instrument that was being used to evaluate online education programs. Many institutions prolifically advertise they offer quality online education but have not had a way to quantify or benchmark their programs. How do students know they are enrolling in a quality program? The scorecard developed as a result of this research study provides an instrument that could identify strengths and weaknesses of an online education program and be used as a benchmarking tool for evaluation against other like programs in the industry. The results of the scorecard may also provide valuable information for strategic planning and budgeting.
The quality scorecard was developed by a panel of online education administrators with all having more than 7 years of experience. The following table shows the distribution of the types of institutions represented.
Institutional Classification for Expert Panel Members
Public (4 year)
Public Community College (2 year)
Private (4 year)
Private Faith-Based (4 year)
Public (4 year)
Private Faith-based (4 year)
Each of the Sloan-C pillars of quality for online education are focused on establishing minimum guidelines for quality effectiveness. While this effective practice actually supports all of the pillars of quality, it may be most closely aligned with the Scale pillar - The provider continuously improves services while reducing costs because the scorecard enables the institution or program director to determine if they meet minimum quality standards in all areas of the program.
Nothing more than a pen or pencil is needed and the time necessary for all processes and procedures to be carefully examined and reviewed before filling out the quality scorecard.
The identification of quality online education programs satisfies a great need in our field and has been requested by many online education administrators as a tool for program improvement. The assessment of quality online education has never been more important as fierce competition from for-profit programs as well as many non-profits programs continues to increase and students all over the world are clicking to find a respectable degree program. Quality in education really does matter as the ultimate impact is to our students.
The interactive scorecard on the Sloan-C website is open to institutional members, see http://sloanconsortium.org/node/add/scorecard-online-program.