Guideline E: Learner Assessment and Feedback
1. Learner Assessment questions should be derived from the Learning Activity content and align with the Learning Objectives or outcomes. The questions should encompass a range of difficulty (e.g., easy to more difficult), and should include questions that encourage critical thinking. Providers are encouraged to consider Bloom’s taxonomy (see https://www.bloomstaxonomy.net/) or another assessment framework when developing questions. To assist Providers in applying the information below, and the requirements of Standard 7 two detailed examples are provided in Section 4.
Types of questions:
1.1 Knowledge-based questions are structured to assess the level of understanding the learner has of the facts or information presented. These questions typically have right answers associated with them.
1.2 Application-based or case-based questions are structured to allow for learners to apply what they have learned. These questions move beyond the recall of facts and ask the learner to apply the Learning Activity content to another context or situation, or to use the content to solve problems or interpret data. Although these questions can have right answers associated with them, there is flexibility for learners to present a variety of acceptable responses.
1.3 Practice- or performance-based (authentic) questions or tasks are structured to allow for the learner to apply the Learning Activity content specifically to their professional practice. The questions should utilize examples or scenarios that are relevant to the learners’ work context and allow for them to demonstrate specific skills or competencies. The questions or tasks should mirror real-life scenarios in which the learner will use the content in their work. These questions typically have right answers associated with them, but there is flexibility for learners to present a variety of acceptable responses. Tasks where learners are demonstrating specific skills or competencies will have a standard level of acceptance that learners must meet. An example of performance-based task is the appropriate administration of an injection.
1.4 Reflective questions are structured to create opportunities for the learner to reflect on the applicability of Learning Activity content to their profession, themselves as professionals, and personally. Learners should be encouraged to draw upon their personal and professional experiences to connect with the learned content. This can include structuring the assessment so that participants explore:
1.4.1 The application of ideas or concepts learned in the Learning Activity to their practice or career; and
1.4.2 The integration of these ideas and concepts with their current knowledge and understandings. This results in a deeper understanding of the concept or idea and a changed or enhanced conceptual perspective.
These questions typically do not have right answers associated with them. As these questions are meant to generate thought, a variety of reasonable responses should be accepted.
Assessment methodologies:
1.5 Formal or traditional methodologies are typically completed at the end of the Learning Activity and utilize question types such as multiple choice, short-answer or essay-type. These questions typically have right answers associated with them and a score (e.g., 4/5 four questions out of five are correct) is usually provided to the learner. These types of questions can be standardized and have statistics to support conclusions of learner achievement
1.6 Informal or non-traditional methodologies are typically administered throughout the Learning Activity to check for understanding. The purpose or intent of these questions is to encourage reflection and discussion, and to assess the level of understanding. These question types may also be multiple choice or short-answer questions with feedback provided upon response, but a final score is not provided. Rather, learners may be provided with the opportunity to re-do the question or discussion may ensue around the responses provided. These questions can be criterion- or performance-based measures that are used to inform instruction.
1.7 Formative methodologies are assessments for learning. The purpose or intent of these assessments is to monitor the progress of learners to identify gaps in learner understanding, and provide timely, specific feedback to learners to improve their understanding of the content. Formative assessments can be used continually throughout the Learning Activity by providing descriptive feedback to the learner. It can include a variety of different activities such as observing the learner completing an activity or task, having students complete self-evaluations or self-assessments, group work or group activities, voting, think-pair-share, ungraded quiz or poll, etc.
1.8 Summative methodologies are assessments of learning. The purpose or intent of these assessments is to provide information on how well the learner understood the learned content. These assessments could appear periodically throughout a Learning Activity or at the end, and are typically graded. For example, in an online module, learners would complete a section and then answer two short-answer questions before they can proceed. The correct responses and feedback are provided to the learner, and an overall score is provided to the learner at the end of the activity. Summative assessments usually have standards for grades associated with them. Examples include final projects, presentations, exams, etc.
2. Examples: The following are two of numerous possible examples of how learning activities can be designed to meet the requirements of CCCEP’s standards and the guidance provided above. These are provided only as a means of demonstrating how a program could be designed and by no means are encompassing of all possible approaches.
2.1 Example 1 – How the Guideline and Standards could be applied in an online Learning Activity.
Online module on professionalism:
Module begins with overview of module (objectives, etc.) and introduces topic of professionalism. A brief overview is provided from a few literature sources. A reflective question appears: Describe someone who you consider to be a professional? Once the learner submits their response, the learner is thanked for their response and are encouraged to keep this person and their traits in mind as they proceed. The learner delves into more content looking at the traits of professionals. Another reflective question appears: Did anything change in your perception of a professional? Was anything confirmed? Once the learner submits their response, the learner is thanked for their response and are encouraged to keep their response in mind as they proceed. Learner continues to learn more about professionalism. An application-based question appears – learners read a news article on a pharmacist who has committed a crime. They are asked to identify what elements of the pharmacist’s behaviour are professional and which are not. They then compare their answers to those provided. Feedback is provided that explains the answers. The next section has a case-study question for learners to work through. Answers and feedback are provided after each response. Learner responses are not graded.
In this example, there are reflective and application-based questions used (maybe authentic as well depending on how the case-study questions are constructed). The methodology for the questions was formative and informal. Feedback was provided after the questions either to prompt further reflection and connection with content or to explain the answers. In this manner, the feedback is timely, aligned with the assessments, and constructive and educational.
2.2 Example 2 – How the Guideline and Standards could be applied in a live virtual Learning Activity
Webinar learning new device:
After introduction and overview of objectives, the presenter begins with asking how many learners have used blood glucose meters (BGM) before through a voting website. Students respond and the presenter sees that only half of the audience has used one before. The presenter then gears their explanations throughout the rest of the presentation to that level – explaining in more detail for the inexperienced and asking those with experience, to describe what they have done in the past or how they approached various scenarios. After the presenter has demonstrated how to use the BGM, he asks the learners to practice on themselves. With cameras on, he is able to observe the learners and provides feedback to them as they are practicing. Feedback includes complimenting those who are executing the technique correctly and providing constructive feedback to those who need correction to their technique. At the end of their presentation, the presenter opens a quiz on a student response system and asks the learners to respond. The questions are multiple choice and review the main points on how to use the device. The presenter goes over the answers at the end of the quiz, explaining the correct responses and answering any questions the learners have.
In this example, the presenter used practice- or performance-based tasks and knowledge-based questions (could also be application-based questions here as well). The methodology for the questions was formative and informal. To take this to a summative assessment, the presenter could evaluate the performance of each learner at the end of the webinar. To include formal assessment in this, the presenter could have them write an exam consisting of ten short-answer questions. Feedback was provided to prompt further reflection on and correction of the learners’ technique, and explanation of the answers. In this manner, the feedback is timely, aligned with the assessments, and constructive and educational.
