Supporting innovation in teaching, learning and learner development at Roehampton

eLearningMeet: online exams for Journalism

Screenshot of the first question in the mock test.

Screenshot of the first question in the mock test.

Due to the many other wonderful presentations, there wasn’t time for this one at the eLearningMeet, so this post covers what I intended to talk about.

In 2015 the module Reporting Britain, convened by Kate Wright, held its first online exam. The technology was already widely in use at the university for in-class tests, but this exam was a first for two reasons. It was the first exam held by the Journalism department, and also the first official online examination as part of the university exam period, which required the coordination of several departments across the university. The exam went very smoothly, and feedback from staff and students was positive.

If you’re thinking of setting up an online exam for your module, these are the steps to follow:

  1. Write your exam. People often think online tests must only be multiple choice, but there are actually many different question types available such as cloze, calculation, short answer, and essay (needs manual marking).
  2. Talk through the requirements with your eLearning Adviser. The eLearning team can create the test in Moodle, or train you so that you have the control. They can help you liaise with other departments that need to be involved such as Registry (they handle the scheduling, arrange invigilators and help set up the room) and IT (who get the computers ready on the day).
  3. Set up a mock exam to get students comfortable with the format. In this case, 57 students sat the final exam. Of those, 53 students had completed the mock exam in their own time (though it must be noted that only 17 attempted the essay and not all seriously). None of the students entitled to extra time attempted the test version.
  4. On the day of the exam, give the briefings. eLearning services has a template for a powerpoint to be read to students that goes over how the exam works, and explains what to do each step of the way. There is also a more advanced version for invigilators, so they can cope with any technical problems that may arise.
  5. Start the exam! In this case, the students were divided between two IT suites, and one group started slightly later, so a time adjustment was needed for those students. eLearning and IT staff were on hand to deal with student queries during the exam, and the minor problems that arose were easily dealt with. Multiple choice questions are shuffled during the exam, so students can’t cheat by glancing at each other’s screens. Students do have full access to the internet. eLearning services is looking into an exam browser which would limit that access, so they would only be able to use Moodle.
  6. After the exam is finished, most types of questions are marked automatically, so there may be no marking at all! Some question types like the essay require manual marking, but Kate reported that the process was easier than anticipated, so tutors were able to turn around grading before a tight exam board deadline.

So the online exam process is not difficult to set up and run, but is it also educationally sound? Since this was the first exam run for this module, comparison with paper exams is not possible. However students did achieve higher results than on the previous coursework assessment. Moreover a wealth of statistics can be obtained which helps tutors to target their teaching for the following year. For example you can see the facility index, which calculates how difficult a particular question is for this cohort of students.

The module also made use of the Moodle Grades facility to release preliminary results before the exam board. This is usually not useful to students, as it has to be set up to correctly calculate the overall module mark, but it made sense to do so since all the assessment for this module was online. It’s possible to have the gradebook make sophisticated calculations, such as ignoring formative work, using different weights for categories and subcategories, dropping the lowest marks of a category, accounting for work that was not submitted, and so on. Conveniently for administrators, spreadsheets can be downloaded with the full module results.


Leave a Reply