Niagara College Canada

How do I create an authentic online course assessment?

Steps to create an authentic online assessment

While traditional tests and quizzes generally ask that students recall information, authentic assessment engages students in a higher level of learning, where they are actively participating in situations that often mirror realistic situations. For example, if a student is in a business marketing course they could develop a marketing plan for a business in the community.  The assessment focuses not only on performance over memorization, but also acts as a learning tool for students rather than a measurement exercise.

To create an authentic assessment Jon Mueller outlines four key steps. (Source: http://jfmueller.faculty.noctrl.edu/toolbox/howdoyoudoit.htm)

1. Outline your objectives

Following the direction of backward design your first step will be to consider what you want your students to know or do following a specific class or lesson. For example, your objective might be that students apply three customer service strategies when dealing with a difficult client within a call centre environment. A good objective is measurable and observable, which will enable you to move to the next step.

2. Determine the activity or tasks

The next part involves identifying what activity you want to create that will enable the student to demonstrate their subject matter expertise. The activity should mimic, or be as authentic as possible, to a real world situation. This is where the learning activity becomes meaningful to the student as opposed to an exercise in memorization. As they’re doing the activity they can relate it to their lives. Using the above objective, the activity could be a role play in a simulated call centre (using Blackboard Collaborate), or you could provide a link to a video and ask students to discuss whether it was a suitable interaction or not.

3. Specify the criteria

For this step, you will think about what good performance looks like. What are the characteristics of good performance? What criteria will prove student understanding and what does the final product look like? For example, if the objective (as above) is to “Apply three customer service strategies when dealing with a difficult customer within a call centre environment”, one criteria may be “The student paid attention and responded to what the customer said, and not how it was said.” Once you have the criteria, you can develop the rubric.

4. Develop the rubric

When developing the rubric you will consider two or more levels of performance for each criteria. For instance, your criteria might be fully demonstrated, partly demonstrated or did not demonstrate. When using the rubric, you will include feedback to the student, which will enable them to reflect on their current level of understanding and help them to improve in the future.

Other Considerations:

Scaffolding

Authentic assessment is often linked with a number of opportunities to practice. You should ensure that you scaffold opportunities within the course so that students can gain ongoing feedback and make adjustments as they learn.  Doing this will also give students time to adjust to this new type of assessment, particularly where they may be used to more traditional forms such as quizzes or tests.

Length of time

Authentic assessments can take considerable time to plan and to grade. As a result, you are encouraged to develop a rubric and to ensure you give yourself enough time to review  students’ submissions.

How do assessments and Universal Design for Learning (UDL) tie in together?

Tying in UDL and assessments

In addition to providing different avenues through which students can demonstrate what they’ve learned, there are a number of other UDL factors you might think about when developing online assessments.

1. First, it’s important to make sure that the assessment helps students meet their learning outcomes.  Ask yourself: Are my learning objectives/goals clear? Does my assessment reflect and measure the intended learning goals, or are there additional components or skills that are also being measured by my assessment?  For example, if you were teaching a course on Contemporary Agricultural Issues in Canada, one of your objectives might be “Describe critical environmental issues impacting staple crop farmers in Canada.”  If you were to assign students an essay identifying and describing these issues, your assessment would be measuring students abilities to achieve the objective, and also their abilities to communicate in writing.
2. Second, once you’ve thought about outcome and objective alignments, it is important to consider whether you might be able to offer students some choice in the ways in which they express their knowledge.  So, extending the example above, if you are asking students to write an essay about the issues affecting Canadian farmers, are you most interested in their identification of these issues or are you also interested in assessing their writing skill? If your focus is more topical – the issues affecting Canadian farmers – then you might provide students with choices in how they express their knowledge.  Perhaps they could choose to submit an essay, an infographic, a video or a set of Powerpoint slides.
3. Third, consider the format of your assignment descriptions and supporting materials (such as rubrics or checklists).  Can you offer these materials in multiple formats to support understanding by diverse learners?  For instance, in addition to a written assignment description and rubric, you might also create a short video describing the assignment and talking students through the components of the rubric.

For more information, CAST provides 10 excellent tips to consider when developing assessments.

Four ways to present choice in assessments

When you’re developing assessments, there are a number of ways you can provide students with choice.  First, you can give them choice within an assessment. For instance, they can choose to write one long essay answer or answer four short essay questions.  Second, they can have choice between assessments. For example, they can submit a paper or do a presentation. Third, they can have choice in the media they use, and might choose to create a video or complete a PowerPoint presentation.  Fourth, you can allow choice in grading. For example, you can ask them to complete four out of five assessments. (Source: https://alludl.ca/create/assessment/choice-of-assessment/)

What are the possibilities and constraints for online testing?

Possibilities with online testing

While it takes time to develop and implement online testing, once it’s set up it offers a variety of benefits for both you and your students.

Access to test questions

With online testing you can develop banks of questions.  Blackboard test questions offer the ability to add numerous content options such as links, images, videos and documents.  You can also use test banks developed by publishers or other instructors, and can move tests between different course sections or between courses from semester to semester.

Instant feedback

As an instructor Blackboard enables you to provide feedback directly into the system and to control when students have access to it. Depending on how you set up grading, you can give students immediate access to marks or enable them to make changes to assignments  by allowing more than one opportunity to submit their work. As well, Blackboard’s Grade Centre can provide a quick picture of student progress and how individual students measure against the group. With this information you can evaluate how well your instruction is working and modify it based on student progress.

Flexibility and accessibility

Students can plan when they complete assignments/assessments based on their specific time zones or when they perceive there to be fewer interruptions. Having choice encourages students to be self-directed in their learning and create strategies for their own self-motivation.

Control

As an instructor you have control over how tests are presented and when they are provided throughout a term. For instance, you can quickly offer different versions of tests once they are added to Blackboard and you can control how students use tests, i.e. preventing students from moving ahead to other questions, if necessary. As well, with all grading in Blackboard there is less chance for human error, which can put your mind at ease and reduce the amount of time you spend reviewing grades. Any test errors can be amended in the system and student tests can be automatically graded to account for any changes.

Challenges with online testing

An important component of a test is its validity. If a test is valid then it is measuring what it is intended to measure. However, validity can be taken away when factors that have nothing to do with a student’s “ability, skill, trait, or domain of knowledge” affects their test score. These factors are called “construct-irrelevant” factors.  (Source: http://www.dartmouth.edu/~ajordan/papers/Ferrier%20et%20al.%20(2011)%20-%20Construct-Irrelevant%20Variance.pdf) For online testing, some of these factors include student motivation, cheating and test anxiety. Although there is no certain solution, it is helpful to understand these factors and where you may implement some strategies.

Student motivation

It can be difficult to identify whether poor performance is related to lack of understanding, lack of motivation, or both. However, there are some characteristics tied to students who disengage in test situations. For instance, students are usually less engaged when it comes to low-stakes exams where the personal consequences are perceived as small. Additionally, a student can be less engaged where test items are more challenging. (Source: https://www.tandfonline.com/doi/full/10.1080/20004508.2018.1490127)   Aside from these characteristics, it is helpful to know when a student is aiming to do their best, or when they are not. For instance, perhaps you’re noticing a decline in performance by a student who is normally a high performer. What is the reason? Is there lack of motivation? Or is it that the content is presenting issues?

Academic integrity

As with any testing environment there are ways that students can cheat, particularly for online tests or homework assignments. Is a student’s score based on what they know or based on how they cheated? For online testing, it is challenging to know whether a student is using their materials or if they are doing the test with help.  There’s no way to reduce cheating 100 percent, but there are strategies you can consider. For instance, you can randomize questions, mix objective and subjective questions, and put a time limit on the test. For written essays, you can take advantage of anti-plagiarism software, like SafeAssign. This website outlines a number of tips for promoting academic integrity for both online tests and assignments within the Blackboard learning environment.

Test Anxiety

Anxiety can be experienced as a trait or arrives within a particular moment or as a result of a situation or event.  It can have a significant impact on a student’s ability to complete a test or perform well. For students who experience anxiety as a trait or disorder, there may be circumstances when they request accommodations to alleviate their anxiety.  For instance, providing untimed or extended testing situations, or answering a long answer question through a bulleted essay. For students who experience anxiety in a testing situation there are ways that you can help to prepare for and reduce their anxiety. For example, over the term, you can provide low stakes practice tests that mimic the higher stakes summative assessment. There has been considerable research on anxiety and how it is correlated with perceived control, such that where an individual feels they have more control, they can manage their stress response more effectively. (Source:https://www.frontiersin.org/articles/10.3389/fpsyg.2016.01415/full). You can provide your students with more control by staggering tests times to allow more time to prepare, as well as giving students choice in how they answer questions or what questions they want to answer.