How Important is Use of Technology in Different Parts of the Assessment Process

  • Assessment Creation
  • Digital evidence
  • Remote invigilation

Use of Technology in Different Parts of the Assessment Process

Assessment Creation

which is about obviously designing and providing assessment tasks and activities the second is about candidates generating digital evidence and the third is about remote invigilation so we’ll start on the first of those and just have a little chat about assessment creation now the first thing i want to just touch on is to begin thinking about using technology in formative

assessment so this is more the assessment which is used to help and support learning so it’s not a summative end loaded assessment process which is more formal in nature it is more that assessment for learning type formative using technology in formative assessment can be a very easy and effective way to engage learners helping them and you to measure their progress and

also to prepare them for digital summative assessment it’s worth bearing in mind that formative assessment tasks should be designed to reinforce and complement the learning so digital formal assessment works best when it’s embedded in that digital teaching and learning so the formative assessment should complement the type of learning that the candidates are doing it also

naturally follows that the summative assessment should also reflect and and be characteristic of the formative assessment and also the teaching and learning so my point there is it will be wrong to just go through a more sort of traditional face-to-face approach to teaching and learning and then suddenly to ask the candidates to undertake remote summative assessment because

it’s something they won’t necessarily be used to so it’s a good idea to drop the use of technology into the learning and teaching process certainly into the formative assessment process and then candidates should be much more comfortable with that before it’s then dropped into the summative assessment the other point to make is there are plenty of online assessment creation tools

available for little or no outlay which are well suited particularly to formative assessment again it’s quite easy to create formative fun engaging assessments which will motivate candidates and can be used throughout that teaching and learning process to measure attainment and improvement and to give you and the candidate extra confidence they are in fact ready to move forward

to summative now whether we use it for formative or summative assessment digital use in assessment is always easier by questioning than it is by observation particularly where complex equipment is required by the learner to generate evidence so if we’re looking at using questions to assess somebody’s knowledge that’s easier for us to use technology in the assessment process

than it is to observe performance particularly where specialist equipment is used is needed by the candidate because they may not have access to that specialist equipment and certainly i think that’s one of the issues that a lot of centers are struggling with at the moment is how to take candidates through practical assessment generally the sort of knowledge and understanding side is easier

to to move to using technology in assessment than that sort of practical side is but there are there are ways that centres are moving very wisely and very sensibly to trapping observation through use of technology as well certain subjects will lend themselves more naturally to the use of technology which is why i was saying earlier you need to think about the qualifications

that you’re offering for example much of the evidence for subjects like computing is is likely to very naturally lend itself to electronic nature electronic evidence by its nature so it’s already compatible with that move to remote assessment so do think very carefully about the different qualifications and subject areas you’re thinking about using technology in in terms of when we get to

summative assessment it is very important that whatever assessment type you use whether it’s questioning or observing or product evaluation and whatever the subject area the assessment task must always follow sqa’s principles of assessment now you’ll probably be aware that we have principles of assessment they are that assessment should be valid in other words it’s

appropriate for the assessment methods and instruments that have been applied it’s reliable, it’s equitable it’s fair and it’s practicable now i’m not going to go into those principles and details but i do always whenever i get the opportunity strongly encourage you to read sqa’s guide to assessment and to encourage you to encourage your colleagues to have a read of the guide to

assessment as well it’s a really useful guide and will certainly help you design and deliver really effective assessments to your learners and it will help you to pick out some of the issues about using technology in that assessment process so the easiest way to

get to that is just go to any search engine and type in sqa guide to assessment and you’ll find it’ll come up as the probably the top outcome of the search and you can look at that document you can download it and you can spread it around your colleagues and

some centres will take extracts of the guide to assessment and will embed that in their staff induction programs and in their staff training programs and in standardization etc so i would strongly advise you have a little look at the guide to assessment and really use that because it is a very very helpful

Digital evidence

now digital evidence generally well let me start by saying is it’s probably easier to gather collect and control the storage and the use of evidence in a classroom setting whereas if you are gathering collecting and controlling and storing evidence which is digital and generated remotely that’s a little bit more of a challenge so here are five things for you to think about in terms of candidates

generating evidence and we’ll come back to a couple of these in a few minutes so there are things that you need to take care that when you’re watching out for assessment that has been generated digitally and remotely the first which i see a couple of you already or a few of you have already alluded to in the chat box is that you have to be pretty sure in fact very sure that the evidence

that’s been generated has been created by the learner who is claiming ownership of the evidence and that authentication of the candidate can be proved the one benefit of electronic evidence is that there are tools available to check for plagiarism you’ll have heard of some of them turnitin is one of them that’s popularly known but there are many other products out there so it is one way

that you can check to make sure that candidates have not plagiarized evidence from other places you also need to do everything you can to deter learners from cheating rather than to focus solely on detecting malpractice and this can be done as simply as requiring learners to complete a declaration that the evidence is their own when they submit that evidence to you it sounds very

simple but the majority of candidates solely both simply by being asked to declare that the evidence is theirs will think very carefully about submitting evidence which isn’t theirs it’s also important you cover this in candidate induction if you are using remote evidence production or generation so that candidates are very clear as to what your expectations are in this thing now it’s

quite obvious that you won’t be able to see the candidates performing activities or generating evidence at first hand but it might be useful once they’ve generated the evidence and submitted to you that evidence you then have a quick meeting with them to ask them a couple of questions about what they did and how they did it in some cases it might even be appropriate to ask them to

provide a short presentation of their work followed by a couple of questions this isn’t to overly assess so it isn’t to increase the assessment burden but it does help you to be more confident that the work presented to you is their own we know that candidates can sometimes get external help from tutors or from parents or from friends or whatever now sometimes that work will be sort of

coached before it’s provided and i think the use of some gentle probing questions once the assessment evidence is submitted will help you to understand whether the candidate has generated that evidence themselves primarily by whether they understand what they have submitted you can of course use remote invigilation to ensure authenticity and we’ll come on to that in a little bit

more detail in a minute or two another thing to think about is is the evidence that the candidate is sending you digitally is it received securely by the appropriate designated center staff so do you know that the candidate knows exactly who to send that evidence to and does that person who’s receiving the evidence know where to store that evidence and how to to process it in effect

linked with that how are you sure that the evidence that’s been submitted can’t then be altered by others learners must be able to protect or lock their evidence before they submit it and or assessors should be able to lock that evidence once it is received from candidates now lots of e-portfolio systems allow that and and facilitate that but if you’re not using something like an e-portfolio

system you do have to think about how you freeze that evidence so it can’t be altered in the future you also need to think about where you’re going to store that candidate evidence in terms of being securely stored in a restricted access file throughout the assessment process and also to make sure that you are storing that evidence in line with sqa’s evidence retention requirements

you need to think about who has access into the the storage folders for this evidence and who doesn’t have access into these storage folders you also need to make sure that wherever you are storing this evidence the evidence is protected so it prevents unauthorized access to learners personal details you really don’t want to be sharing details of candidate email addresses or

candidate contact details because these are absent-mindedly attached to to evidence that’s being sent in so you need to be very careful you are protecting candidates personal details when they are submitting work to you so there’s a few things to think about

just in terms of of digital creation and digital evidence generated by candidates we’re going to move now on to what is probably the the more meaty issue for most of you that of

Remote invigilation

so let’s have a little look at that we don’t just to be absolutely clear in sqa we don’t advocate the use of particular brands of remote invigilation there are many remote invigilation tools offered out in the marketplace some because generally because american origins will refer themselves as remote proctoring tools rather than remote invigilation tools invigilation and proctoring are the

same thing it’s just proctoring tends to use that as an american word whereas in in the uk we tend to use invigilation so i’m not going to talk to you about particular remote invigilation tools what i’m going to do is to challenge you with sort of three different questions and then we’ll go on to talk about how you might meet these three questions the first which you’ve already started when

you’re chatting through this is how are you going to confirm that the candidate is who they say they are that issue of personation in other words if i say i am matthew mccullagh when i come on to to generate digital evidence how can you be sure that it is actually me and not my twin brother for example how do you prove that i haven’t paid somebody to take my place in an exam or

whatever the second question is how are you going to prevent the assessment material that you are required to manage as confidential from being copied and finding its way into the public domain that’s easy to do in a classroom you hand out the question papers you take them back in at the end of a session but it’s much more difficult to do if you’re putting those questions or

the assessment task up on a screen or if candidates are able to download them somehow onto their own software or hardware rather and so that’s an issue about how you prevent from the confidential assessment materials which you should be managing from becoming publicly available the third question is how are you going to uphold the required assessment conditions and by

that obviously i mean if you look at your unit specifications and you look at your assessment strategies it will identify where for example you should be using closed book assessment techniques and if we are required to use close book technique and then how

to ensure that candidates aren’t using additional help in terms of open books or collaborating with other people in the room or whatever so let’s have a little look at those in a little bit more detail in terms of authenticating candidates i think the first thing to

say is it may not be a major issue particularly where the candidates are well known to you in the center because let’s face it if you’re sitting on a teaching an hnd course for two years for example or a part-time course over a number of months and you see

these candidates again and again either face-to-face or on-screen you will recognize them you form a relationship with them you’ll get into chat mode with them it may be very easy for an invigilator to know whether the candidate is who they say they are when they’re on screen simply because they are familiar with that individual but it also may be important to look at things like identity

cards and some sort of physical matriculation you can buy if you buy in things like online invigilation there are things like retina scans et cetera that can be used as well and again the more complex the method of of checking authentication and generally the the more expensive it’ll be but you do need to have a think about how you are going to ensure that the candidate is who they say

they are requiring the same identity evidences you’d use in the center but use the the the video link to check the possession of an identity card is one very simple way you you can use i wouldn’t necessarily say you have to go to things like full facial recognition

software unless it’s for really high stakes assessments or where candidates are just really not well known to the centers in terms of how you ensure the continued security and integrity of the assessment context so in other words how to make sure that doesn’t go

out into the public domain you would probably want to consider using software that allows access to the assessment content only once the assessment session has begun so that candidates only get that that screen opening up once the session has started and

ideally this could be linked to a timer that automatically limits the duration of the session or at very least there should be an audit trail of timings but if you restrict the time the candidate has available to complete the task they will probably be more focused on

completing the task rather than trying to copy the task out for other people you can also use software to lock down the desktop in effect this puts the device in what we call kiosk mode so it blocks access to other applications such as screen grab while the

assessment is running mainly for desktops and laptops obviously more challenging if people are using personal devices such as

tablets or mobile phones but certainly if you’re able to lock down the desktop then that that is very helpful and you might say to

candidates that they can’t use tablets and or mobile phones and therefore they have to use desktop for laptop you can also use a camera setup that gives the remote invigilator a full and continuous view of the candidates screen and their immediate working

environment one note of caution i would put there we are all talking to each other remotely here have a look around the room

you’re in now i’m actually doing this from my daughter’s bedroom now that’s fine if i was sitting an assessment and an invigilator said to me lift up your laptop and show you around the the room you’re in to make sure there’s nobody else in there it might be

recently comfortable to show you around the the surroundings of this room but if i’m in a one-bedroom bedsit or if i’ve gone up to

my bedroom as a young learner because it’s the one place in the house i can get some peace and quiet or i’m caring for somebody

or there’s lots of things happening there may be many reasons why i’m just not comfortable lifting up my camera and showing you around the room that i’m in so again we need to be careful that we’re not putting people in awkward positions and we’re not

breaching privacy rules or laws by asking people to show us that so and you know there is a place for using cameras as in remote

invigilation but i think they have to be very well thought through okay in terms of how you uphold the assessment conditions

some of those making sure that you are if it says it’s closed book that there is closed book and again that desktop lockdown

lockdown is a very good way so if you can put that in in kiosk mode so it stops the candidate being able to access other applications again a single camera view on the candidate and their immediate environment can check for other devices and other resources not being used in the absence of this you you could or should look into a camera setup that allows invigilators to clearly

see the screen or the device and the candidate and the immediate environment but that as i say has some some risks of it as well now in terms of online and vigilation we produced some guidance recently in december.

Post Your Thoughts