This is a complete transcript of the SULS panel discussion for online exams held in Semester 1, 2020. The panel was moderated by Natalie Leung (Vice President (Education) 2020) and consisted of Professor Rita Shackel, Dr Kevin Walton, Professor Peter Gerangelos and Dr Gemma Turton.
Questions (click on the links to jump to each section):
Do you have any advice on turning the lecture or tutorial notes into appropriate exam notes?
How is preparing for an online exam different to preparing for an in-person exam?
How do you think the reading time should be used in an online exam/open book exam?
Do you have any tips on typing exams? I often feel more fluent when writing with pen and paper.
Would the expected standard of answers for a two-hour exam be different to that of a two-day exam?
What should I do to prepare for an open book exam?
GT: My advice for an open book exam is to largely treat it like you would a close book exam. You aren't going to have time to be making an enormous amount of reference to your materials, and you certainly don't want to be looking at any of the topics or any of the statutes for the first time in the exam. Even though it's an open book exam, you should be as prepared as you would be if it was a closed book exam, and think of your materials as a support, rather than the primary resource that you're going to use.
RS: I agree entirely. I think one of the key messages for this semester with the online exams, is to be organized. Students will need to be really mindful of time management, given the fact that there are word limits imposed on the short release take-home exams.
PG: Yes, I would concur with everything that's been said, and particularly the point about treating the open book exam in the same way as you would a sit down exam. In other words, you should be preparing for it in a very similar way. I think the only difference with an open book exam is simply that you don't have to rote learn case names etc., so you can focus on the substantive content of the material. Don't think that you can leave it until the day and just rely on your notes; you won't be able to. And if other courses are like mine, when we are aware that students have access to materials, sometimes the questions can be a little bit more challenging, in order to avoid students being able to look up an answer. I would be preparing for it as if it were a closed book exam, by understanding case principles and how they apply, identifying issues and so on. We should also add, students should become familiar with the scenarios and the facts of the cases you've been studying and also do revision problems that you've been given and practice that under time conditions.
KW: I just want to pick up on something that Peter said, and that is although you don't need to memorize the case names, you're going to need a really good set of notes. You don't want to be rifling through your notes, trying to find the name of the case, while you're doing the exam. You should be making notes that you can refer to quickly in the exam and that would be a crucial part of preparation.
Do you have any advice on turning the lecture or tutorial notes into appropriate exam notes?
PG: The examination notes should contain a clear statement of legal principle, with reference to the relevant cases, which goes back to Kevin's point. The idea of exam notes is to have a set of principles that are a full summary of the lecture and tutorial notes in order to also assist you to remember the more nuanced points; but if there are important nuanced points, perhaps you might want to put them precisely in the exam notes. That’s what I often say to students. There is an art in putting maximum meaning into minimum words, and that's the exercise with exam notes. Now some students say, ‘I took your advice and I used minimum words when they seek feedback on an assignment’ and I say, well yes, but unfortunately you forgot the first bit which is putting maximum meaning. Whatever you state must have the complete statement of the legal principle. That's what the exam notes are there for, to jog your memory, with respect to principles that hopefully you will have internalized as if you were preparing for a closed book examination. You can’t put everything down on the exam notes. They presuppose deep study and deep knowledge. Often you surprise yourself, if you have been doing that deep study, deep preparation and deep thinking, at how that material or knowledge you've gained does come out when you see a very cleverly framed set of exam notes.
GT: A couple of other tips that I might suggest is that when you're making those notes, make sure that you are referring to the course as a whole, rather than the specific questions that were asked in a tutorial. I think students sometimes frame their notes in terms of the issues that came up in the tutorials, and obviously it's not possible for us to have covered everything about a topic within the tutorial. So make sure that your notes are wider than that they addressed the course as a whole, and perhaps when you're making your notes about the cases, think about why you're using cases in your answers. Is the case a key case that establishes a legal principle? Is it a case that creates an exception or limits or expands that principle or is it a case it's more illustrative of how that might be applied to facts? And think about why you might use particular cases in your answers, and let that shape how we take notes on them as you're condensing those notes down.
How is preparing for an online exam different to preparing for an in-person exam?
KW: I think it's implicit in a lot of what we've been saying, there's not really much difference in the preparation you should be doing. Obviously, just before the exam, you're not going to be getting on the train or the bus to come to the University campus. But in terms of the substance of the exam, your preparation is going to look no different. We've already made the point that you shouldn't be relying heavily on notes in the exam anyway, so although you might have more space than you would have in the exam hall, doesn't mean you need to fill the space.
RS: I agree entirely with Kevin's point. And I think that is one of the key messages coming out of everything that has been said so far, that in terms of the static side of exam preparation, not all that much is really different. I think where the difference arises is perhaps in the physical preparation and also what you do during the exam, and even some of that is not all that different. But I do think that in the conduct of the exam, students will need to be much more alert to issues around time management, and obviously also alert to some of the technical aspects of accessing and submitting an exam online.
GT: In an in-person exam, you're used to having a time limit; and now there will be word limits on some exams, if not all. It's important to make sure that you are aware of the word limit before you start. If you've got more of an extended-release take-home exam, plan when you're going to actually complete the exam: are you going to leave yourself time to go back and proofread it, and make sure that you've built all of that in before the submission deadline?
PG: Prepare a nice space for yourself. Make sure that you've got everything ready to go before the day. Perhaps a cup of coffee or whatever else you might need and put up a notice on your door that you're doing an exam so that people don't disturb you. Really make sure that everything is working well technically, well before the exam period so that it doesn't hit you on the day and create a massive panic.
KW: Just as you would in an in-person exam, you should be putting your phone away. It shouldn't be next to you while you're doing the exam, it's going to distract you. But also you have no reason to have it because you're not going to be communicating with anybody during the exam and certainly not about the exam itself. Just as you're going into the exam hall and there are certain things that you shouldn't have with you, you shouldn't have that device sitting next to you.
GT: And I would also put it on airplane mode because I know, for example, I get notifications on my watch. If a message has come through on my phone and I would find that distracting. As well as putting my phone completely out of reach, I would put it on airplane mode so that there is no unexpected distraction at all.
KW: And turn off notifications on your laptop as well.
How do you think the reading time should be used in an online exam/open book exam?
PG: In the same way that you would do in an in-person exam. Even though we cannot enforce reading time, because you're in charge of the time yourself, I do think it's important that you spend 20 to 30 minutes planning your examination. I think that the initial period of looking at the exam problems, making an initial assessment of what you're going to attempt and initial issues is how you should use the time. Once you see what you need to do, attempt to allocate a number of minutes to each question, in order that you are able to complete the examination on time. The reading time is a time of deep reflection to identify issues, key facts, etc. before you actually start answering the question. If you just start immediately, you will get bogged down in your writing and you will miss things. That’s the essential answer.
RS: From some of the emails I received from students, there is a particular concern around how to manage word limits. Perhaps adding to Peter's suggestions around planning and allocating your time in the exam, there may also be some benefit to thinking about how you're going to allocate word limits so that you can keep track of those word limits as you're progressing through your exams.
If I am going over the word count of an online exam, what should I do? Should I keep writing and cut later or stop and start cutting?
KW: Peter has provided some great advice on this, which is "maximum strategy of the maximum meaning and the minimum words". We've set the word limit in such a way that you should have plenty of words in order to answer the question. And what you shouldn't be doing is quoting slabs of a judgment, or sections in legislation; that's a waste of the words that you've been given. You should be mindful as you're planning your answers; treat each part of the answer as something quite separate, and be aware that if you have gone over the limit in one part, you're going to have fewer words to answer other parts. Try and avoid that as you're going and always have in your mind Peter's advice that you're trying to be pithy and capture the crucial principles and rules in as few words as possible. Unless there is a particular inspiration that's very important, there's no need to be reproducing sentences.
RS: Also resist the temptation to cut and paste from other word documents from the electronic version of your notes, for example. Cutting and pasting into your exam risks, first, taking you over the word limit, and secondly, opening you up to plagiarism. So students should think really carefully about cutting and pasting into their exam answer.
Generally speaking most exams have specified an overall word limit. And that word limit has been made very clear in the cover sheet and will be strictly adhered to. When that word limit has been reached, the marker stops reading.
PG: When you are preparing, spread out the words evenly between the questions that you've been asked, as you're doing each question. Once you're reaching a particular self-allocated word limit, then you should start self-restraint to not go over too much. The word limit is ample in order for students to be able to, not just have a solid answer, but an excellent answer. One way to assist you in this word limit scenario where we're applying word count is in the preparation for examination notes. Now that you have the time, state the key principles or as many as you think is appropriate in very pithy language, but very fulsome in meaning. You won't be able to do that in the examination. Part of preparing for exams is being able to do that and not be very long-winded in the statement of principles.
GT: If you're trying to determine how many words you want to allocate to specific parts of the question, look out for whether marks allocated to parts of the question have been indicated by the examiner, because that's obviously a guide to how many words you want to be spending on each question.
Are students expected to include a record of the work count, or can the work count be viewed by examiners without students stating it?
RS: The word limit can be independently checked and will be checked and verified through Turnitin.
If we are running out of time in the exam, should we be covering every issue briefly, or a couple in more detail?
PG: As many issues as you can briefly, I think, although it depends on the examiner. One of the things that the mark is based on is the identification of issues. It's far easier to get from naught to 50 than it is to go from 50 and above. So if you get a couple, in detail, and you've missed five other issues, it's very difficult to get a pass; whereas if you identify five issues and deal with them briefly, it's indicative to the examiner that you know what you're talking about, you just haven't had the time to go through it all. Then, you might find that you actually do get to the 50% mark on that question. Applying that across the examination as a whole, it's far better to answer all the questions. Say, if there are five questions, to answer four really well and not answer question five, you'll get absolutely zero and that last question does affect your others. It's better to attempt everything. If you do run out of time, just even in dot points, identify as many issues as you can. Then if you still have time, start to go into more detail with a couple. That's very general advice. It really depends on the examination, but I think that's probably a general principle that most of us tend to agree.
KW: I agree. And just as with the word limit, we've been generous with the time limit as well, in recognition of all these complications of sitting online exams. And so, as with the word limit, you should have plenty of time to produce, not just a pass answer but even an HD answer.
How can I get into a mindset that is similar to being in an exam room? It’s often very easy to get distracted at home.
GT: I would say treat it as if you're going into an exam room. When you go into the exam room, you take your pens etc. in and you've got everything you need. Make sure you've got everything you want before you sit down to do the exam so that you're not distracted by getting up to get a drink and things like that. It helps to define your workspace. I know not everybody has an ideal workspace at home. Personally I don't either, I'm working at my dining table, and so I've got a load of my textbooks and I've marked out a desk space with those textbooks. They're like a little book barricade. That means that my eye-line is surrounded by work things, even though I'm in my dining room and kitchen. I think in terms of a workspace, doing the best with the space you have is the tip that I would give.
PG: If students are anything like I was before an exam, I didn't really need much to get me into the mindset, as it were. I would think ‘I'm doing a law exam’, and that really puts the pressure on and I think most of you will already be there in terms of your mindset. Just don't be complacent. Because you're at home, it takes away that little bit of anxiety, which I think is always a good thing to spur you on; not so much that it's debilitating, but just that sense of seriousness to help you. Another point that I make to students every year is to be kind to yourself. The first thing that you will notice when you see the factual scenario in front of you, you immediately think ‘I haven't got a clue what this is talking about’ and you start to panic. My advice is to allow yourself to panic for ten, twenty seconds and then say to yourself ‘well that's enough, I've got that out of my system’ and then you'll see, if you've prepared sufficiently, that your own ability, your intelligence, the fact that you've picked things up by osmosis more than you realized, will kick in. Just acknowledge that element of anxiety or panic at the start but allow yourself to experience that and then give yourself a time limit, and then get on with it. A positive mindset is key, which says, ‘I can do this’. It's not a pep talk, it's a statement of truth. The mere fact that you're in law school indicates a very high level of intelligence. Just let that intelligence come to the fore. Don't let panic or anxiety debilitate you in any way.
Do you have any tips on typing exams? I often feel more fluent when writing with pen and paper.
RS: Be really aware about what settings you have in place, in terms of autocorrect. I've been quite horrified lately, to discover the ways in which my autocorrect has actually been set up and has been working and what default words it's using. That's just a very practical example of what to be aware of when you are typing.
GT: I would add that, some advice that we've already given which is that you should be practising with past exam questions where possible. And so when you're practising with those, set yourself the same conditions that you will when you're doing the exam and practice typing them. That's a habit that you start to get into while you're studying, rather than doing it for the first time in the exam. If you're more comfortable with pen and paper, you've got your reading time and you might want to be jotting down a few notes or a structure for your answer, and then you type your answer. If actually sketching something out on paper helps, then perhaps use that during your reading time.
Questions from the floor:
What is the significance of the extra time that has been provided in assessments? Assuming that we don't have any internet issues or uploading and downloading issues, how should we be using this extra time?
RS: The examiner has determined what time should be set for completing the particular paper in a holistic way. The exam duration has factored in the reading time that students would require, as we've been discussing, and has also made account for the fact that this is an online exam, and students may experience some technical difficulties. There's an allowance for the time that it would take to access the paper, the time that it would also take to finalize your written submission, and upload, and there is an allowance for the possibility that something might go wrong in that process as well. It's useful for you in your planning phase, to think about how you're going to allocate your time and ensure that you do allow sufficient time to deal with those technical aspects.
KW: We've given students plenty of time in order to do the uploading. If you only leave five minutes for uploading, you could get unlucky and experience some problem in that last five minutes. If the worst happens - you'll be given these instructions on the cover sheet - there is an email address to which you should send your answer as soon as possible, once you are able to re-establish internet connection. We've thought about the concerns that students will have about sitting exams in this way, and we try our best to put in place processes that would deal with that. That takes me back to a previous question as well about getting into the mindset, not being distracted during the exam. You could be in shared accommodation and there's noise in the place that you live in. You've done your best and yet there are these distractions. In the event that happens, students are able to apply for special consideration. There are a number of issues that might arise through no fault of your own, and the university has tried to anticipate those. If the worst comes to the worst in the exam, there are processes to go through and adjustments and accommodations that can be made.
RS: I totally agree with everything that Kevin has just said, but I would like to emphasize that not allowing yourself enough time to upload your exam of itself isn't a justification, or a basis for special consideration. Be mindful that it is your responsibility to ensure that you allow adequate time to submit your paper online. So don't be tempted to leave it until the very last couple of minutes, that's a free take home message. Factor that into your planning, and allow adequate time, assume that there might be a glitch.
In relation to online exams, what would be the impact of referencing on plagiarism? What style of referencing are examiners expecting?
PG: It depends on each exam and I'm sure you'll get instructions if it were to the contrary. It's just what would happen in a normal examination. I'm not expecting any 'referencing' at all in the examination, unless perhaps there is a direct quote that is particularly important, you may want to put the judge’s name or the case. But certainly very lenient as far as that's concerned.
RS: I would emphasize that it will be very important for students to familiarize themselves with the exam cover sheet for your unit of study. That exam cover sheet, which has very clean and very explicit instructions, will be provided to you well ahead of your scheduled exam. A very simple piece of advice is to read your exam cover sheet very carefully, and be very familiar with what you're being asked to do in that particular unit of study.
Is it recommended to print out the questions?
PG: That depends on individual preference; I certainly would, but really it depends on each person. It might be useful just to have it there on the side, rather than on the screen and you're going from one screen to another.
RS: I would just like to highlight that you have an obligation to ensure that the exam remains confidential. And that you also have an obligation not to share the exam or any questions within the exam, either in hardcopy or electronically.
Would the expected standard of answers for a two-hour exam be different to that of a two day exam?
PG: Yes, because you have the two days. If you're doing an examination in three or four hours, the examiner will of course, take it on board that you have a very limited time and the expectation would be lower. But again it all depends on the examination, I should say. But yes, there would be a slightly different standard if you've got a short release examination, as opposed to a longer one.
GT: Check the instructions for your exam; if it's a longer one, there may be referencing requirements. Also check the instructions because you're going to be told whether you are expected to take those full two days or whether it is a short exam that you are being given more flexibility as to when you complete it.
RS: Just building further on that, with some of the longer release extended take home exams, you may find that the approach toward limits is different. So, once again, do ensure that you read the instructions, very carefully because those instructions will give you information about what is expected of you in relation to that particular paper.
Do you have any tips you would recommend for students to make the best use of past papers when they're unable to obtain feedback for their responses?
KW: One thing you could do is, you can share practice answers with your friends doing the same subject, and give one another feedback. That would be a useful thing to do.
GT: You may want to see whether the generic feedback for those questions has been made available. Not model answers, but generic feedback. If that's the case then, my advice would be to answer one practice question, look at how your answer compares with that feedback, take that feedback on board, and then go and answer another one. I've seen students who will try and write four answers, and only then look at the feedback, and it's not a developmental process.
RS: I think that's great advice. I would, again, use this as an opportunity to caution students on how any model responses, or general answers that have been previously circulated to students, are being used in the context of a take home exam. Students need to be really careful about not lifting sentences or parts out of the model answers or general responses and using them in their response in a take home exam, because that is plagiarism.
PG: I wanted to share two things. The first is from my experience over the years. There are some students that I come across that say they have been relying on notes that have been circulating or model type answers. It seems that they spend a lot of time doing that, to the detriment of their own reading of cases. If your whole preparation is just gleaning answers from circulating notes or other people's responses, you're not reading the cases and letting the principles sink in. And who's to say that what other people wrote are correct? It is often the case that if you're just repeating past errors or repeating past quality. Have confidence in yourself. If I can just make one point, and this is purely personal relating to my view of things, people may disagree - but sometimes students are just so focused on getting the top marks in an examination, that their actual legal education is lost or diminished. You've got to balance that with true learning and getting away from this earlier secondary school type emphasis on maximizing marks. Of course I'm not suggesting that good grades are not important, but not at the expense of deeper learning.
The second thing is, from my experiences being in charge with the integrity unit in the faculty. Look, be very careful with passing around your documents to others. Sometimes it does lead to a form of academic dishonesty. And the person that is giving out their own work, which another person then plagiarizes even or especially after it's been marked from previous years, can find themselves also in a lot of trouble. So please be very conscious of that. No one is stopping you from collaborating before an examination in the sense of discussing points, or perhaps sharing your own responses to questions and that sort of thing, but be very careful, don't cross the line. And I think a warning should be put out there, that a lot of trust is being put in people doing a take home exam. And, you know, we're trusting that people will not use their mobile to phone a friend in order to assist them in answering a question. So, it's a question of personal honor coming to the fore here. One final remark; one of the things I never understood was when students try to cheat. I always thought that an examination is one test to yourself, of how well you do, don't you really want to know that? As opposed to gaming the system to maximize a result, what's the point of that? And I can guarantee you, from my years in practice, a day will come that you will not be able to do that, and the whole facade comes crumbling down. So, I'm not just saying this for the sake of honesty, per se, which it really should be, but also the practical consequences of dishonesty - not just being caught out, but also the effect it has on your actual ability and the way you can perform in the future. I do apologize it's a bit long winded of me, and a bit personal, but I just wanted to make those remarks.
RS: Just to add to Peter's comments, I would really encourage students to reflect on what has been included in the cover sheet by way of the combined statement, and to really reflect on what that statement entails on the part of each and every student in terms of academic integrity. I think this is a wonderful opportunity to think about those types of issues, and to think about what that means in terms of how you conduct yourselves in the context of these online take home examinations.
Thank you to Professor Rita Shackel, Dr Kevin Walton, Professor Peter Gerangelos and Dr Gemma Turton for sharing your experiences with us and taking the time to share your insight.
GT: Good luck with your exams!