TeachLearn:Assessment

electronic marking
eMarking: Providing electronic feedback for electronic assessments in Moodle

Feedback is an important component in providing motivation for students in their learning. With an increasing amount of student work being submitted electronically, and therefore an absence of any physical artefact, ways to successfully handle the marking and feedback of these virtual assessments need to be developed.

The wiki describes techniques that were developed initially to mark CD-ROM based assessments, and have been extended to assessments submitted in a learning management system (Moodle) to be accepted, marked, and feedback returned to the students fully electronically.

Leave me a message
Ok. I presented this at the EIT Teaching & Learning Conference (3 October 2008). If you want to leave a comment you can hit the Edit link and add it here!

in the beginning

 * Assessments paper based (offline)
 * Could mark on the paper itself, or
 * Had a paper marking schedule
 * Submitted on Floppy disk -> CD ROM
 * Multimedia so also needed headphones!
 * Spreadsheets
 * DipTec (individual student on each sheet and multiple workbooks)
 * Others (also used to manage attendance)
 * Each assessment has 3 sheets 1. Marking schedule, 2. Results grid, 3. cross tab (for pasting into Moodle)

Hardware

 * Dual monitor
 * Laptops (have set up already.
 * PCs trickier (usually need another graphics card with dual monitor support – and may need to disable onboard one)
 * Widescreen – doesn’t really show student work.
 * Different screen sizes
 * Managed with a screen background that shows monitor sizes.
 * Download full image


 * Setting up a dual monitor
 * Windows XP: Configuring multiple monitors

Assessment type: Research (htm or doc)



 * Student submission: Either word (doc) or html file
 * Student skills: Word (or htm) file creation, ability to zip files, ability to upload to Moodle
 * Response: eMarked document, comments
 * Marker skills: Create assessment, download files, Track changes in Word, Spreadsheet, upload marked doc, cut & paste from Spreadsheet to Moodle

Method


 * 1) Students upload doc (or htm) file as zip file
 * 2) Marker: Downloads (zip) file(s)
 * 3) If htm - load into Word
 * 4) Open Spreadsheet
 * 5) Turn on track changes, mark document (with comments into S/S)
 * 6) Cut and paste results into Moodle

Problems


 * 1) uploading to wrong student !!
 * 2) students email address institute default (so they never read it!)

A sample document

A Word document to play with

Assessment type: Document including hand drawings


Figure StuSubDoc: Showing Word document that includes a scanned image, Word text and Word drawing tools

Student skills: Ability to scan of digital photograph

Observation


 * improved quality of student work.

Assessment type: Multimedia presentation (e.g. Adobe Flash)
Problems


 * 1) No artifact that you can actually annotate

How would you mark this??

Michaels really basic Flash Joke Book

The spreadsheets
Assessment based

Class based

The skills


 * 1) Basic skills, plus
 * 2) Named ranges




 * 1) VLookup (including an IF formula to prevent 0 showing up)



Sample Spreadsheet

Example XL Spreadsheet used for marking

Student support

 * Providing forms (they are not paper based so dont format them as such!!)
 * If available an electronic white board can assist in drawings

Student submissions
File formats
 * 1) When single file indicate they do not need to archive
 * 2) zip files - use Windows [Right click] + Send to > Compressed zip folder (found "advanced" zip programs like WinZip and 7Zip created problems particularly when sub-folders are used.
 * 3) rar archive use 7Zip (Figure 7ZipRAR)

Some students found it difficult to find the upload link. (Now with the inability to change the colours of the text links they all look the same so it is hard to make them obvious) .. so provide a Drop box at the top (an assessment worth 0 marks).

Photographing images (in one instance a student used a cell phone set to a very small size - so images barely readible)

Lots of files of multiple types when they were pushed for time just photographed/scanned images, created a few doc files then stuck them into a folder and zipped) (Figure 7ZipRAR)- difficult to mark.

Remember not all students have Wintel computers (some have Apples) so you may encounter "weird" files (e.g. __MACOSX )

File formats encountered: Archive (Zip, RAR) Word (doc, docx), PowerPoint (xls, xlsx), Adobe (PDF), Images (gif, jpg, png) - so need applications to view them all!



Figure 7ZipRAR: Showing RAR Archive in 7Zip with multiple file types (Word (docx), image (jpg), Excel (xlsx))

Providing feedback
One common problem is how do you reduce the number of time students ask "When will we get our results back?" My solution has been to provide a simple Marking progress graph



This is really easy to do. The bar chart is just a series of colons for those done, and fullstops (.) for those to be done. e.g. Research [5/24] :::::...................

Then all you do is:
 * highlight the ":"s and make the text colour AND the background colour dark blue
 * highlight the "."s and make the text colour AND the background colour aqua (light blue)

(Of course you could use colons for both - with different text/backgrounds)

Overall observations

 * Marking still takes time
 * Marking in small chunks difficult - have to open up all documents (S/S, student work) - and can be affected by your PC performance
 * Plagerism
 * Easy to check .. type a phrase into google... but really slows the marking speed.
 * Students have difficult creating unique file names!!
 * I ask for yyyyLastName_Topic e.g. 2008Verhaart_MyTopic.doc
 * Upload problems
 * Need to cater for: Different versions of Word (doc, docx), have not had ODF yet
 * Need to be able to unzip. rar files appear (can use 7Zip)
 * Students rename their files prior to upload and lose the extension.
 * Students zip files are missing folders/sub-direcories (They use programs like ZipCentral where they need to click include folders/subdirectories).
 * More difficult to manage than paper based as:
 * Need to be near a computer

Online marking for NCEA Qualifications will not occur the near future in New Zealand at least. While it has been introduced in the Britain and the USA with moderate success reasons outlined by the NZQA will not see it here soon.("NZQA says no," 2011)

Reference NZQA says no. (2011)retrieved from http://tvnz.co.nz/national-news/online-marking-fails-nzqa-test-4358734 on Sept6, 2011.