Developing and Reviewing Online Courses: Items for Consideration

Tags:

Clayton R. Wright, crwr77  @ gmail.com

(This document is a revision of “The Criteria for Evaluating the Quality of Online Courses” that the author released in 2003. This revised 2011 document is based on the author’s experience since the original publication date and the suggestions of various e-learning professionals. The document focuses on effective online course components rather than on the qualities of effective instructors, online facilitation skills, or learning outcomes such as the 21st Century Skills developed by the League for Innovation and the Pew Charitable Trusts.)

Quality is an elusive concept, as all who measure it can define it differently. Although quality is affected by the context in which the measurement is taken, the real measure of a quality course is whether it helps learners achieve the stated learning outcomes. In order to do so, the course should comprise a number of components and attributes that will lead to learner success. This document outlines components and attributes of courses that will help you help learners succeed in the online learning environment.

From an instructional design viewpoint, a quality online course is one that enables students to achieve learning outcomes in a flexible and effective manner. Ideally, the learning process will also be efficient, but online activities such as discussions, web searches, and collaborative assignments frequently take time, and efficiency is sacrificed for effectiveness. A quality online course, from a design perspective, takes into account the following factors:

Quality is usually achieved through the efforts of a team that may comprise a number of professionals, including the academic program head, project coordinator, subject matter expert, instructional assistant, course writer or developer, course reviewer, instructional designer, graphics or multimedia designer, copyright officer, editor, and computer and information systems specialist. You may decide to extract tasks and roles from this document and assign them to each member of your development team.  Even if one person creates an online course, this individual must apply most or all of the skills of the professionals listed above. Often, quality courses take into account the contributions of stakeholders such as learners who will take the course and employers who may hire the graduates of the course.

You should not consider the list of components and attributes as a definitive list but rather a starting point when considering what determines a quality course in your context. When developing a new course or reviewing an old one, you may want to review the items listed here and select those that are more appropriate to your situation. For example, if you are creating an online course from a constructivist point of view or want to implement problem-based learning, items listed under the heading of instructional and learning strategies may need to be modified or be deemed as inappropriate; but, items listed under other sections, such as language and copyright, would be useful to you. Some may consider that a few of the items are more applicable for online programs (especially those listed under General Information) rather than for developing and reviewing a single course. Thus, some may want to make a list of items that only applies to programs. If you are an e-learning provider, such as an instructor in a continuing education department who is offering a standalone course not associated with other e-learning courses in a program, you may need to address all the items included in this document in your course. Use this document as a guide when considering what constitutes an effective online course in your situation. Whether you are constructing or reviewing distance or blended courses, this document will help you ascertain the components and features that should be included in your Web-based teaching and learning environment.

Initially, the list of items for consideration may seem overwhelming, but each item was selected because it identifies specific course components, qualities, or procedures proven to be helpful to learners and/or instructors. When using this list, focus first on the major question or questions under each heading. Then, as you review or develop a course, refer to the dotted (or bulleted) items to ensure that these items have been addressed. As you become more familiar with the items on the list, you will need to refer to this document less frequently.

Many courses are evaluated by internal and external reviewers. Consider making up your own list of evaluative items based on the items outlined in this document. Then, ask each reviewer to evaluate a specific aspect of the online course by focusing on the list of specifications under one heading.

1.0 General Information

At the beginning of the course, are learners provided with general information that will assist them in completing the course and in understanding its objectives and procedures?

  1. A brief description of the course includes a goal and learning objectives and/or outcomes as well as the specific skills that must be mastered and exhibited at the end of the course.
  2. A program map and/or rationale indicate how this course is related to other courses in the program and the credit value of each course.
  3. Prerequisites and/or corequisites are identified, including any requirement to be a fluent user of the Internet.
  4. Learners are directed to a website that helps them determine whether online education is appropriate for them.
  5. If the course is intended for international participants, the level of reading proficiency (such as a Test of English as a Foreign Language or TOEFL score) necessary to complete the course successfully in the language in which it is written is specified.
  6. The instructor is identified including expertise and research interest. Learners are provided with a variety of ways of contacting the instructor.
  7. The availability of the instructor is specified, as well as what learners can expect from the instructor. For example, the turnaround time for responses to learners’ questions should be stated clearly.
  8. The developers, reviewers, and facilitators or tutors of the course are listed. Brief biographies may be provided to assure students of the developers’ and facilitators’ knowledge and expertise.
  9. A list of required and recommended resources includes all textbooks, courseware, and online resources necessary to complete the course. If learners must access online resources, instructions are provided for locating and accessing these resources, including password information. If the materials are located in a library, learners are told whether the items are available by e-reserve or physical reserve and the limitations on these reserves. If your student cohort comprises individuals who may be geographically distant from a library, refrain from relying on physical resources that may be found in only one location.
  10. Special requirements are identified, such as recommended modem speed or Internet bandwidth, hardware (computing speed and storage capacity), software (including an e-mail program and file formats for handling assignments and other attachments), and plug-ins. If learners do not have access to a reliable Internet connection or suitable bandwidth, can they receive videos and other large files via compact discs (CDs), digital video discs (DVDs), or universal serial bus (USB) keys or flash drives?
  11. The estimated time (i.e., hours per week) required for completion of the module and/or course is stated, along with its overall duration. If feasible, a timeline is provided that outlines dates by which specific activities must be completed.
  12. Guidelines for participating in online discussions, also referred to as “netiquette,” are provided, as well as suggestions for handling incoming e-mail, e-mail attachments, viruses, and e-mail filters. If there is an online discussion area or chat room, the procedures or expectations for language, quality of discourse, and length of postings should be delineated. Participants must be informed as to whether they must use their real name or whether they can use a pseudonym during discussions.
  13. Learners are informed about group-work activities, the guidelines for forming groups, grading criteria for group participation and assignments, and their responsibilities as group members. If possible, the grading rubric/grid and/or criteria for each grade are outlined.
  14. The introduction to the course takes into account the learners’ backgrounds, ability levels, and expectations, including their personal learning goals and objectives, or specifies the attributes of the learners for whom the course is designed. Learner attributes may include the level of prior knowledge of the subject matter, experience with related materials, and skills already developed.
  15. The availability of technical support is stated, and links to online technical information are provided. The hours during which technical support is available are clearly identified, including the time zone.
  16. Links to institutional learner support services such as a mathematics help centre, writing centre, library, and counseling services are provided and learners are encouraged to use these resources. Off-site learners should be entitled to the same services provided to on-site users.
  17. Learners are directed to a source for answers to “frequently asked questions” pertaining to online learning. This source may provide information covering many of the items listed above, as well as items related to plagiarism, virus protection, and firewalls.
  18. Learners are informed about their right to privacy and the conditions under which their names or online submissions may be shared with others.
  19. Learners are informed about the institution’s or course provider’s information and technology policy as well as online or e-learning policy which outlines acceptable behaviour and the responsibilities of the learner in upholding the procedures and rules that are included in the various policies.
  20. Learners are informed if information about their course webpage visits, participation in discussions, and other online activities are gathered for institutional research purposes.
  21. Some online courses require learners to create their own online profiles that can be viewed by the instructor and those registered in the course. Thus, learners must be informed about the type and size of documents, files, and folders that they are able to upload and how this operation can be done.
  22. A copyright statement or disclaimer clearly identifies the owner(s) of the course and the source(s) of the material students are about to use. Learners should also be informed about their responsibilities in terms of use, distribution, and access to the copyrighted materials. The ethical and legal ramifications of using copyrighted works should be discussed or links provided to such a discussion.

2.0 Accessibility

How accessible is the course material? Can learners find information quickly?

Has the instructor ensured that the infrastructure and server can handle the number of learners enrolled in the course and the size of the documents or folders students will be working with and saving?

  1. The function of each icon or button is explained and/or is naturally evident to the learners.
  2. A detailed table of contents includes objectives, learning outcomes, topics, and assessments.
  3. Every section of the course or module begins with a preview, overview, or summary.
  4. Every page is linked to the previous page, the start of the module, the beginning of the course, and to e-mail links so that learners may contact instructors and other learners for clarification and discussion. These links must be checked to ensure that they are working and it should be possible for the learners to return to the original spot where they clicked on the link. Some course management systems do not allow designers to place a link on every page, but links may be placed on the screen frame or interface page.
  5. Links within the course are provided to other parts of the course.
  6. Links are provided to special editors (e.g., an equation editor or special language characters) that learners can use when completing their assignments.
  7. Page headers or footers identify where the learner is in the course.
  8. An index lists key words or topics.
  9. A glossary defines uncommon or technical terms used in the course and may provide links to sources of supplementary information.
  10. Consideration is given to learners who may have visual or auditory challenges.
  11. The online course complies with accessibility laws and follows the Principles of Universal Instructional Design (UID).
  12. The material can be accessed conveniently on a handheld device or alternatively, platforms on which the material is intended to be accessed are clearly stated.

3.0 Organization

Is the material organized in such a manner that learners can discern relationships between parts of the course?

  1. The table of contents gives an accurate indication of how the material is arranged. Instead of a table of contents, a graphic organizer or chart may be used as this may help learners see the relationships between topics and modules.
  2. The organization or sequencing of the content is appropriate for the subject matter and the intended audience. For example, content may be sequenced in several ways including chronological, hierarchical or pyramid, utilization-related or step-by-step, spiral, inquiry-related, and generic or isolated blocks.
  3. Units of instruction or topics are divided into subunits or subtopics.
  4. Subtopics are related to main topics.
  5. The organization of components is consistent throughout the course. For example, each module may have the following sections:
  • introduction,
  • objectives,
  • pretest,
  • directions,
  • explanatory text including learning activities such as case studies,
  • suggested answers for learning activities and links to additional information,
  • module summary,
  • self-test,
  • self-test answers,
  • references,
  • additional readings, and
  • module assignment(s).

6.  Required course elements. References and links to other parts of the material or external sources are accurate. The number of external links should be considered carefully as too many links may cause learners to lose their place within the course.

4.0 Language

Is the level of the language used appropriate for the intended participants?

The items below not only apply to written material, but also to video and multimedia content.

  1. The writing or presentation style is clear and direct.
  2. Instructions are stated simply and are easy to understand.
  3. Clear directions are given.
  4. Familiar or common words are used.
  5. A conversational tone employs the second person: you, not the learner.
  6. Verbs are in the active, not passive, voice; for example, Maslow developed the theory, not the theory was developed by Maslow.
  7. Sentences are short.
  8. Paragraphs are brief.
  9. Numbers are used to identify sequential steps in a task or process.
  10. Dots or bullets are used to list items that are not prioritized or sequential. However, when a list of non-prioritized or non-sequential items contains more than eight or ten items, it may be easier for the learners if the items are numbered.
  11. Dots, dashes, and numbers are used consistently.
  12. The tone of the writing or presentation is supportive and encouraging.
  13. Terms are used consistently.
  14. Abbreviations and symbols are defined.
  15. Conventional technical terms relevant to the level of the learner or geographic location are used where appropriate. For example, which term is best for your learner – USB key, thumb drive, flash drive, or portable storage device?
  16. Spelling and grammar are consistent and accurate. If the information is presented in an audio form, the pronunciation of terms is consistent.
  17. The writing or presentation conveys no explicit or implicit bias relative to age, culture or ethnicity, race, gender, or sexual preference. However, biased opinions may be included as examples if they are relevant to the course content. For example, you may require learners to discuss different views of a topic after you have presented opposing or biased views.

Has the course material been edited for grammar, language, and content verification? Content verification can refer to the accuracy, breadth, and depth of the information to be presented to a specific target audience.

5.0 Layout

Does the layout facilitate learning? Is the material attractive and appropriate for the course content and the intended audience?

Note that it may be impossible to design a page that is displayed in the same manner on every computer screen; a simple layout is best if the equipment and software available to the learners are unknown.

  1. The layout is appropriate for the content and intended audience.
  2. Navigational icons or cues are used consistently. Navigation is explained or demonstrated, especially when it may differ from navigation in other courses in the program.
  3. Links to external materials open in a new tab or window. This method is used consistently and explained to students. If this method is not used, students may lose their place in the module and may find it difficult to return to the original page of departure. As the permanence of external materials may be in doubt, one option is to cache the documents and link to the cache.
  4. The typeface is appropriate for the content and common to all programs and computers, such as Times Roman or Arial.
  5. Bold-face type is used sparingly, to highlight important terms, for example.
  6. Capital letters and underlining are not used for emphasis. Underlining is used only for hyperlinks.
  7. Key words are highlighted, especially when they are first used.
  8. Headings and subheadings are used to organize content.
  9. Consideration has been given to the numbering of sections and subsections and to the logical naming of files, so that the name of a file saved locally or visible as a URL gives useful clues about its contents.
  10. The format is uncluttered and includes white space.
  11. Ragged right margins are used or letters are kerned (evenly spaced on the line). The use of ragged right margins may lead to uneven line lengths, but the spacing between words and letters are consistent thus making the words easier to read.
  12. The contrast between text and the background material makes the text legible. If learners are expected to print the materials, the text may be a shade off of deep black so as to save on ink cartridge consumption.
  13. Colour is used effectively. Note that the colours used should not be indicative of any cultural or political bias or meanings. This can be difficult to achieve when the learners may have different cultural backgrounds. However, in most cultures “red” is used to grab attention and/or signal a warning. “Green” is used to signal safety or identify environmental factors.
  14. Graphic elements such as diagrams, tables, and photographs illustrate or clarify information presented in the text. All these illustrative objects are consistently and appropriately captioned.
  15. Illustrations can be seen easily on a computer screen, and image files are used that can accommodate different download speeds. Care is taken to minimize the file sizes so that those with limited bandwidth can obtain the information that they need.
  16. Text explaining a graphic is aligned with the non-textual material.
  17. Labels within a diagram are fixed so that their relative position does not change as the size of the image is altered.
  18. To assist those who may use screen readers, a description of each illustration and graphic is available as an ALT tag. (An ALT tag is an HTML tag that represents an object. It provides alternative text when non-textual elements cannot be displayed.) Note also that if learners use a software reading program, such as Natural Reader, the text should be in a plain format as the software may not read italicized text or text presented in boxes.
  19. Videos have the option of turning on or off closed captioning.
  20. If frames are used in all web pages, their use is consistent.
  21. The material is displayed attractively.

6.0 Goals and Objectives

Are goals and objectives or outcomes provided to outline learning expectations at the beginning of the course and, where appropriate, at the beginning of each module? “Objectives” usually refer to the intentions of the instructor whereas “outcomes” refers to what the learner will, should, or might learn.

If a constructivist approach is used, learners may be asked to select their own goals, objectives, content, learning strategies, resources, and evaluation scheme. If so, the rationale of this self-regulation of learning is provided, as this approach may be new to students in the course.

  1. Goals and objectives cover course content and are related to the program of study.
  2. They are relevant to the subject matter and to the “real world” in which the content may be applied.
  3. Objectives specify learning outcomes related to knowledge, skills, competencies, behaviours, and/or attitudes.
  4. Appropriate action verbs are used in goals and objectives. The accomplishment of objectives should be measurable; therefore, vague words such as “understood” and “realize” are not used. An explanation of how progress against the objectives or outcomes will be assessed is provided.
  5. Objectives state clearly and concisely what must be done, when, and how.
  6. Different levels of outcomes are stated, including those that call for critical thinking and problem-solving skills.
  7. Goals and objectives provide a discernable sequence of expectations.

7.0 Course Content

Is the content appropriate and relevant, given the subject matter and the learners’ backgrounds and abilities? How does the content relate to the current and future lives of the learners?

If you are using a constructivist approach to your online course, you may want to substitute the term “learning experiences” for the word “content” below.

The content is:

  1. Directly related to learning objectives and the relationship is explained to learners.
  2. Complete, preferably providing all the content or learning experiences needed to achieve the learning objectives. Ideally, learners should be encouraged to obtain information from a variety of sources including their virtual classmates. However, if your learners have limited access to the Internet, limited bandwidth, and/or limited access to local educational resources, it may be best to provide the essential learning materials that they will require to achieve the course objectives. This approach is not congruent with the constructivist paradigm that encourages learners to seek out and integrate their own knowledge. Yet, it must be acknowledged that those in rural areas may not have access to diverse resources at a reasonable cost.
  3. Appropriate to the learners’ characteristics (ability and maturity level) and experiences
  4. Comparable and at least equal in rigor to similar on-campus courses
  5. Accurate
  6. Relevant
  7. Current and updated regularly. Documents and other resources clearly indicate the date on which they were last updated, and possibly, their version number.
  8. Broken into small, incremental learning steps
  9. Presented in a logical sequence
  10. Related to other material the learners may have studied or experiences they may have had
  11. Illustrated by examples and/or case studies when new information is presented
  12. Linked to other sources, with reading assignments clearly specified
  13. The amount of content is appropriate for the objectives to be covered and the time available to the learner to cover them. (Frequently, course designers have a tendency to add more content as it becomes available without removing something from the course. Thus, the course increases in size, but the learner only has a limited time to complete it.)

Did a second content expert review the course?

8.0 Copyright of the Resources Used in the Course

If the works of others has been included in the course, has permission been obtained to use them and have they been used according to the specifications of the creator?

Unless the material clearly has an open copyright license, educators should be careful about using the work of others. Even some open copyright licensed material may have restrictions that may require the user to provide attribution to the original creator, may prevent the user from using the material in profit-generating situations, or may inhibit the modification or re-mixing of the material.

  1. Are all quoted materials or resources cited correctly?
  2. Has permission been obtained to use copyrighted material such as articles, texts, images, illustrations, music and/or sound?
  3. Has permission been obtained to link to videos and animations including those on YouTube?

Instructors may need to be aware of what constitutes copyright infringement in an educational setting and how to gain permission to use copyrighted protected materials. Instructors also need to be aware of their rights as it pertains to copyright and what restrictions they may want to place on the use, replication, adaptation, and re-mixing of their work. Instructors should be cognizant of Creative Commons licenses that provide a legal licensing framework enabling creators to retain the copyright for their works yet allow others to copy, modify, and distribute the work.

9.0 Instructional or Learning Strategies and Opportunities for Practice and Transfer

Do the instructional or learning strategies enable learners to learn effectively in a variety of ways and to engage in activities that promote practice and the transfer of skills? Is there a range of different activities the learners may use or must they all do all of the activities? Have allowances been made for differentiation and learner choice in terms of self-regulation?

  1. Instructions or directions are clear and concise.
  2. Learners are told which activities must be performed synchronously and which may be performed asynchronously.
  3. Learners are told whether learning activities are sequential or whether they can be completed in any order.
  4. Learners are informed about their own responsibilities in online learning.
  5. Expectations are clearly specified for participation in collaborative or team-based learning activities.
  6. Procedures for grouping learners for team-based learning activities are specified.
  7. Deadlines are specified, and the consequences of missing deadlines are clearly stated. However, there may need to be exceptions for extenuating circumstances and this should be stated as well as the procedure for asking for an extension of a deadline.
  8. A variety of instructional or learning activities are used to promote interactivity. These may include online discussions, online conferencing, and collaborative assignments.
  9. Illustrations, photographs, animations, and other forms of multimedia are used to present facts and reinforce concepts. Note that if multimedia material is built into a course, learners may need advanced computer hardware and software, as well as a high-speed Internet connection. If receiving multimedia on a CD or DVD is an option, this must be stated, as well as any additional costs involved. Although it is impossible for instructors to specify the delivery time for physical resources such as CDs and DVDs, instructors should remind learners that there will be a delay between when learners request a resource and when they receive it. Thus, learners must plan ahead and order resources well before they think they might need them.
  10. Learners can select activities that are relevant for them, as the activities have been designed with their ages, cultural backgrounds, and experiential needs in mind.
  11. Learners can proceed at a pace that is appropriate for them and can repeat sections as often as they need to.
  12. Activities engage and motivate the learners. Learners must frequently respond to questions, select options, provide information, or contact others.
  13. Activities develop appropriate cognitive, affective, and psychomotor skills.
  14. Activities encourage critical thinking, creativity, and problem solving.
  15. Activities and materials are presented sequentially in order of difficulty.
  16. Learners are encouraged to interact with others and benefit from their experience and professional expertise.
  17. The interaction among learners may be framed within the Community of Inquiry Framework that includes social, teaching, and cognitive presence.
  18. Learners are linked to resources beyond the course material.
  19. The number of activities is sufficient to support learning.
  20. Activities are realistic and appropriate and can be performed with the resources and time available to the learners. Alternative resources should be suggested. For example, if a student is required to use hydrochloric acid in a chemistry experiment, they should also be informed if common household bleach can be used as a substitute.
  21. Frequent opportunities are provided for practice and knowledge transfer.
  22. Self-check questions are provided so that learners can ascertain for themselves the level of progress they have made.
  23. Constructive, relevant, and frequent feedback is provided to promote clarification, elaboration, and transfer.
  24. Summaries are provided throughout the material, particularly at the end of topics, lessons, and modules.

Is the instructor primarily a facilitator of learning, rather than a provider of content? Is the instructor providing opportunities for students to interpret learning, to make connections, and to associate learning with their current and future lives?

10.0 Learning Resources

Are the learning resources accessible, appropriate, and accurate?

  1. Learning materials are appropriate for the learners and the subject matter.
  2. The resource material is accurate, current, and related to the course content.
  3. Lists of learning resources are divided into “required” and “optional” categories.
  4. Various learning resources are used to ensure compatibility with learners’ different interests, abilities, and learning styles. The needs of students with learning or physical disabilities are taken into account.
  5. Multimedia clips, such as audio and video clips, are included. Should a learner not have an Internet connection that supports the file size required, alternate methods of accessing the resources should be provided.
  6. The format of multimedia materials is specified, and a direct link to a required plug-in is provided.
  7. Learning resources reflecting different points of view are provided when appropriate.
  8. If students are required to use external resources, learners are instructed on how to access and use the resource. For example, if learners are required to access the huge Health Canada website for drug product information, learners should be instructed on how to navigate the website for the information they are seeking.
  9. Learners are informed about the acceptable forms of citations that they should use. Although learners may be referred to style manuals such as the Publication Manual of the American Psychological Association or the Chicago Manual of Style, instructors should provide examples of the acceptable formats for frequently referenced resources such as a book, a journal article, and a webpage.
  10. A bibliography or reference list includes a variety of material such as Web links (URLs), books, journals, CDs, and videos.
  11. Links are provided to material within and external to the course.
  12. Learners are encouraged to seek out their own learning resources and to build knowledge with other students. The extent to which this can be accomplished depends on the accessibility to the Internet, Internet bandwidth limitations or download speeds, and costs of these services to the student.

11.0 Evaluation or Assessment

Are the evaluative or assessment activities clear, feasible, relevant, accurate, and congruent with the objectives, content, and practical applications of the content?

  1. Learners are given clear expectations and criteria for credit assignments. Instructors may require learners to submit an outline before learners proceed with their projects. Examples of assignments that meet the criteria may be included for students to review; however, this approach may limit creativity. If examples are provided, exemplars are provided that cover a range of achievement – what does an “A-rated” assignment look like and how does it differ from a “B-rated” assignment?
  2. The number of assignments and their due dates are reasonable and take into account various holidays.
  3. Evaluation and grading procedures are clear and explicit. Rubrics used in grading are provided.
  4. Appropriate links to institutional policies on grading and evaluation are provided.
  5. The relationships between course learning outcomes, evaluation strategies, course assignments, and the real world in which these skills will be used are evident to the learner.
  6. The relationship between individual assignments and the final course grade is clearly specified.
  7. A variety of feasible and content-relevant assignments or evaluative exercises are provided.
  8. The evaluative exercises are relevant to the learners and the career or profession they may pursue.
  9. Sufficiently detailed step-by-step instructions are provided for each evaluative exercise.
  10. Guidelines for submitting assignments are provided.
  11. Students are informed about the criteria that will be used to evaluate their participation in online activities such as discussion groups.
  12. Learners are able to track and evaluate their own progress. Self-tests are similar to the final or formative evaluation instruments they will encounter.
  13. Criteria and procedures for peer review and evaluation are clearly specified if these elements are included in the course.
  14. Learners are informed about the consequences of plagiarism and the failure to properly cite copyrighted material.
  15. Learners are told when they can expect to receive feedback from the instructor and that the feedback they receive will be timely, detailed, relevant, supportive, specific to the assignment, and will nurture reflection and critical thinking. The feedback should be motivationally effective.
  16. Evaluation procedures are congruent with the objectives and reflect any priorities that have been established for the objectives.
  17. Learners are given the opportunity to rate the module and provide feedback which can be used to improve the next iteration of the module.

12.0 Overall

  1. Is there evidence that the course has been piloted and that learners can achieve the objectives of the course? This step is often missed due to expediency. But, should learners experience difficulties that could have been easily addressed following a pilot?
  2. Has the course been reviewed by experts in content and design? Have stakeholders such as learners and employers been involved in the design and review of the course?
  3. Is the course up-to-date – current in both content and technical aspects? When was the course last revised and the URLs updated?
  4. If instructors who did not participate in its development will use the course, can the course be modified? If so, what permission is needed before this modification can occur?
  5. What feedback has been received from learners and course facilitators that could be used to improve the course? Has information collected during the operation of the course been reviewed, interpreted, and applied to the new rendering of the course? How could this information be used for research purposes and further the development of online learning?
After reviewing the above list of items that contribute to effective online courses, carefully select and modify items that best suit your needs for an effective, quality online learning experience. The success of online learning will depend on the quality of its instructional design, the academic and technical support provided to learners and instructors, and the attention and interest instructors give to their learners. It is hoped that this document will help you ensure that online learning is successful and that you avoid problems encountered by instructional designers, course developers, online facilitators, and learners who use your course.

Further Reading

Below are a number of resources that may be helpful when developing and reviewing courses.

Anderson, T. (2008). The Theory and Practice of Online Learning, Athabasca, Canada: Athabasca University Press. http://www.aupress.ca/index.php/books/120146

Barker, K. (2002). Canadian Recommended E-learning Guidelines. FuturEd and the Canadian Association for Community Education. http://www.futured.com/pdf/CanREGs%20Eng.pdf

Boelryk, A. & Hunter, J. (2005). Online Course Review Document, Georgian College, http://www.georgianc.on.ca/staff/ctl/wp-content/uploads/2009/02/CourseReview.pdf

British Columbia Ministry of Education (2010). Standards for K-12 Distributed Learning in British Columbia. http://www.bced.gov.bc.ca/dist_learning/docs/dl_standards.pdf

California State University (2009), Rubric for Online Instruction, http://www.csuchico.edu/celt/roi/

Chao, T., Saj, T., & Tessier, F. (2006). Establishing a quality review for online courses. EDUCAUSE Quarterly, 29(3). http://www.educause.edu/ir/library/pdf/EQM0635.pdf

e-Learning Guild. (2007). Handbook of e-Learning Strategy. http://www.elearningguild.com/content.cfm?selection=doc.817

Frydenberg, J. (2002). Quality standards in e-learning: A matrix of analysis. The International Review of Research in Open and Distance Learning, 3(2). http://www.irrodl.org/index.php/irrodl/article/view/109/189

Illinois Online Network (2010). Quality Online Course Initiative. http://www.ion.uillinois.edu/initiatives/qoci/rubric.asp

International Association for K-12 Online Learning (2010, October), National Primer on K-12 Online Learning, Vienna, Virginia, USA: iNACOL. http://www.inacol.org/research/docs/iNCL_NationalPrimerv22010-web.pdf

International Association for K-12 Online Learning (2011, October), National Standards for Quality Online Course, Version 2, Vienna, Virginia, USA: iNACOL. http://www.inacol.org./research/nationalstandards/iNACOL_CourseStandards_2011.pdf

Koul, B.N. and A. Kanwar (Eds.) (2006). Towards a Culture of Quality. Vancouver: Commonwealth of Learning. http://www.col.org/resources/publications/Pages/detail.aspx?PID=124

Legion, R. (2006). Comparison of the Quality Matters Rubric to Accreditation Standards for Distance Learning, https://confluence.delhi.edu/download/attachments/74055682/Comparison+of+the+Quality+Matters+Rubric+-+Summary.pdf

Moore, J. C. (2005). The Sloan Consortium Quality Framework and the Five Pillars. Needham, Massachusetts: The Sloan Consortium. http://www.aln.org/publications/books/qualityframework.pdf

National Education Association (NEA). (2006). Guide to Teaching Online Courses. Washington, D.C.: NACOL. http://www.nea.org/assets/docs/onlineteachguide.pdf

Organisation for Economic Co-operation and Development, (OECD). (2006). Guidelines for Quality Provision in Cross-border Higher Education. http://www.oecd.org/dataoecd/27/51/35779480.pdf

Phillips, R. A., McNaught, C., & Kennedy, G. E. (2011). Evaluating e-Learning: Guiding Research and Practice. New York & London: Routledge.

Quality Matters, Inter-Institutional Quality Assurance in Online Learning, http://www.qualitymatters.org/

Shelton, K. (2011). A review of paradigms for evaluating the quality of online education programs. Online Journal of Distance Learning Administration, 14(1). http://www.westga.edu/~distance/ojdla/spring141/shelton141.pdf

South African Institute of Distance Education (SAIDE), (2004). Distance Education and Open Learning in Sub-Saharan Africa: Criteria and Conditions for Quality and Critical Success Factors. Vancouver, Canada: Commonwealth of Learning. http://www.col.org/SiteCollectionDocuments/04DEinSSA_CriteriaforQuality.pdf

UNESCO (2007, November 30). Third Global Forum on International Quality Assurance, Accreditation and the Recognition of Qualifications in Higher Education: Learners and New Higher Education Spaces – Challenges for Quality Assurance and the Recognition of Qualifications, Final Report of the Dar es Salaam, Tanzania Forum. http://unesdoc.unesco.org/images/0015/001559/155919e.pdf

University of Wisconsin-Madison, Quality Principles, http://depd.wisc.edu/html/quality3.htm

Web-Based Training Information Center. What Constitutes Quality in Web-Based Training? http://www.webbasedtraining.com/primer_quality.aspx

Wong, A. L. (2007, November). Cross-cultural delivery of e-learning programmes: Perspectives from Hong Kong. The International Review of Research in Open and Distance Learning, 8(3). http://www.irrodl.org/index.php/irrodl/article/view/426/937

Contributors

The contents of this document were reviewed by a diverse group of professionals who work in a variety of geographically-dispersed education and training environments. The list below includes those who were involved in reviewing the original criteria and those who contributed to the revised version of this document in 2011.

  • Wayne Baillie, formerly with the South Shore District School Board, Bridgewater, Canada
  • Maureen Baron, English Montreal School Board, Montreal, Canada
  • Cathy Conroy, Athabasca University, Athabasca, Canada
  • Rod Corbett, Mount Royal University, Calgary, Canada
  • Daphne Crane, Memorial University, St. John’s, Canada
  • Genevieve Gallant, Memorial University, St. John’s, Canada
  • Doug Hamilton, Royal Roads University, Victoria, Canada
  • Judith Johnson, formerly with Grant MacEwan College, Edmonton, Canada
  • Valerie Lopes, Seneca College of Applied Arts and Technology, Toronto, Canada
  • Carmel McNaught, The Chinese University of Hong Kong, Hong Kong, China
  • Sushmita Mitra, National Institute of Open Schooling, New Delhi, India
  • Craig Montgomerie, University of Alberta, Edmonton, Canada
  • Diana Mukami, African Medical and Research Foundation, Nairobi, Kenya
  • Morag Munro, Dublin City University, Dublin, Ireland
  • Wai Kong Ng, formerly with Wawasan Open University, George Town, Malaysia
  • Sunday A. Reju, Polytechnic of Namibia, Windhoek, Namibia
  • Seb Schmoller, Association for Learning Technology, The United Kingdom
  • Bryan Shepherd, formerly with NorQuest College, Edmonton, Canada
  • Ingrid Stammer, Canadian Pacific Railway, Calgary, Canada
  • Val Stewart, formerly with Grant MacEwan College, Edmonton, Canada
  • Sabrina Stotland, Oranim Academic College of Education, Tivon, Israel
  • Michael Szabo, formerly with the University of Alberta, Edmonton, Canada

Clayton R. Wright

crwr77@gmail.com

If you enjoyed reading this article we invite you to join the Association for Learning Technology (ALT) as an individual member, and to encourage your own organisation to join ALT as an organisational or sponsoring member.

Leave a Reply

Your email address will not be published. Required fields are marked *