Put simply computer-assisted assessment is
“The application of computers to assessment processes, including delivery of tests,”JISC 2010, p.56.
Why would I use technology to aid question-based exams or tests?
- Tests can be delivered on or off campus
- Questions can be randomised and selected at random from a pool (e.g. 10 questions randomly selected from 30)
- Questions can be adaptive with the next question released based on the previous answer
- Questions can include multimedia such as images, audio, video or virtual simulations
- Questions can be marked and a score calculated automatically
- This can aid with reduced marking where all questions or most are automatically marked
- Feedback can be provided instantly to the student based on their answers
- Questions can easily be reused for another purpose or the following year
- Statistics can provide you with information about questions which students commonly answer incorrectly helping to guide the creation of further learning materials and activities
How do I use technology to do this?
The starting point for delivering a computer-assisted assessment is to decide on whether the test is formative or summative. It is incredibly important to identify if a test is summative as this generally entails a greater amount of work, such as validating questions, testing created questions for scoring errors, checking delivery options and booking labs.
Here at the University of Derby the main tool used to deliver tests is the Tests tool in Course Resources.
There are three main steps required to deliver a computer-assisted assessment:
- Design the questions
Whether you are looking to deliver a formative or summative assessment the first place to start is with creating the questions and associated feedback. We currently use the Course Resources (Blackboard) Test tool to deliver these assessments as this provides a wide variety of question types.
Writing good questions for tests can be tricky but there is lots of advice available on the web on how to write assessment questions and a useful guide to start with is ‘Designing effective objective test questions: an introductory workshop‘ from the Computer Assisted Assessment Centre at Loughborough University.
- Author the test
Authoring of a computer-assisted assessment is done through the tests tool in Course Resources (Blackboard) environment. This can be either undertaken by yourself or with the involvement of the TEL team.
- Deliver the test
Summative Delivery
Summative computer-assisted assessment is normally delivered under exam conditions as an alternative to the end of module paper based exam. The student’s marks will count towards the final mark/grade for that module or programme. This can be more time consuming than a formative assessment because there are a number of tasks to be done in advance of the delivery date. For more information on this please contact the TEL team.
Formative Delivery
Formative computer-assisted assessment is normally delivered throughout the module as an aid to learning. It can be used as a revision aid or a diagnostic tool and provide students with instant feedback into what they are doing correctly and incorrectly.
Resources:
- Tests, surveys and pools
- Questions types
- Question Pools
- Building a pool
- Test and survey results
- Information for Invigilators
References
JISC. (2010) Effective Assessment in a Digital Age: A guide to technology-enhanced assessment and feedback [online], Bristol, HEFCE. (accessed 26 March 2015).
I have been told to view the test results in the ‘Grade centre’. Is this in PeopleSoft?
No. The Grade Centre in Blackboard (found in the Control Panel) and the Grade Centre in PeopleSoft are completely separate. If after students take a test you need to enter their grades into PeopleSoft, then you will need to do this manually.
Can I carry out a practice version of the test?
There are two options. When adding another academic to your sandbox, it is possible to choose whether they are added as a tutor or a student. In this way you could test it with one of your colleagues, and then easily move the test into the desired module when you’re happy with it. The alternative is simply to turn on all of the feedback options – score, submitted answers, correct answers, and possibly feedback – so that after taking the test you get all of the information.
Why do some image types do not work with the hotspot question?
The standard internet file types – PNG, JPG and Gif – have been tested and work in both Internet Explorer and Firefox. If you encounter a problem with a specific file then please contact the TEL team for assistance.
How can I provide students with immediate feedback after answering a question?
Unfortunately this isn’t possible; as you’ve seen the feedback is only presented after the whole test has been completed.
When viewing results, why is there a green square with question mark instead of a score?
This means that the test has been submitted, but that your input is needed before a score can be given. There are a couple of question types that cannot be automatically marked (essay, short-answer, and file upload). If you use these question types then you will need to manually mark them by viewing the attempt and entering a score. This symbol will also appear if you have set a time limit, and the student has exceeded it.
When viewing results, why is there a paper and pencil icon instead of a score?
This means the student has not clicked on the final submit button. If you are certain that the test should have been finished, e.g. you set the test to only be available for a limited time and that has now passed, then you will need to force the submission yourself. When you view the attempt there will be a button at the top to submit the answers on behalf of the student, and the score will then appear in the grade centre.
When I download the results, why is there a lot of unnecessary information?
Apart from the student’s details the only columns that are usually interesting are the ones that have the heading “Answer #” or “Auto Score #”, and the rest can just be deleted.
How can I add images to my tests?
In the ‘Test Canvas’, where you select which question types to add, there will be a button in the top-right for Question Settings. By clicking on this button you will be taken to a page where you can turn on the option to include images (section 2, tick the boxes as required). Now when you create or edit a question, there will be an option to include 1 file or image underneath the question or answer text. For images you may want to change the ‘Action’ drop-down option to ‘Display the image within the page’.