Crowdmark collaborative grading (integration with Brightspace)
What is Crowdmark?
Crowdmark is an online collaborative grading and analytics platform that helps educators, working alone or in teams, to more effectively evaluate student work. The platform allows for easy distribution and collection of student work, offers tools for the assembly and management of grading teams, and provides an online workflow that streamlines grading and the delivery of rich feedback to students.
What are the benefits to using Crowdmark in your course?
Integrated with D2L Brightspace
Graders collaborate with ease and are able to see results and comments of other graders
Digital implimentation of marking reduces the need to carry physical papers
Efficiency gains in exam administration, grading, data recording and analysis
1. Click the Create assessment button on the top right corner.
2. A window will pop up where you can title your assesssment. Click the Create assessment button once you are finished.
In the type menu, you will be asked to select which type of assessment would you like to create.
Administered assignments are for monitored tests or exams written in person. The assignments are created with Crowdmark assessment templates, and then printed and administered by an instructor. Then, the assignment is scanned and uploaded to Crowdmark for grading.
Assigned assignments are completed and submitted by students independantly, such as regular assignments or take home exams. The assignment is emailed to the student and then the student must either complete the assignment digitally and submit it as a file or print and complete it on paper and then scan and upload a pdf.
Checkmark the options/features that you would like to add to your assessment.
Double-sided printing allows for the assessment sheets to be printed double-sided. If you will not be printing this assessment double-sided, please uncheck the box. Crowdmark needs this information to decide the placement of the QR codes in the printable file.
The following features are in beta and may not work reliably, so we do not reccomend using them yet.
Enable automated matching will add a region to the cover page of your template for students to write first name, last name and student ID number. Crowdmark will then scan the page to automatically identify the student.
Include multiple choice will add a 40-question bubble sheet to the end of each student’s booklet, then grade it automatically. Alternatively, we reccomend that you check out using Akindi for your courses bubble sheet exams.
An assessment template is the questions booklet that you will be printing and handing out to students. This menu allows you to upload a PDF of the assessment you have pre-created. Either drop your PDF into the box or click on the button to upload your PDF. This page can be formatted however you like, and can be any size — including 8.5" x 11", legal, A4, portrait or landscape format. Make sure to leave room under each question for the student to write out their answer.
After you upload your template, Crowdmark will add a unique QR code to each page and generate a printable PDF file containing a booklet for each student.
It is important that you leave some space between the top of the page and the question, the QR codes are placed in this region.
Once you upload your assessment template, you can review the pages with generated QR codes. If you are happy with the layout, click Next to continue with the setup. If not, click Try again to upload a different file.
During this step, be sure to verify that the QR codes aren't accidentally covering any of the questions.
In the questions menu, you will need to define the locations of each question (and subquestion) on your template. Crowdmark will automatically add a question label and a points label on each page of your assessment template where it identifies a question. The system will only detect 1 question per page, but you can manually add questions to include multiple.
If the automated system has missed a question, click Add question on the right and then drag the question on the page to change its location. You can also drag to reposition the existing question labels.
You may also edit the maximum score allocated for each question by editing the Max. points box on the side.
Once you are satisfied with the question label placements, click Save and Continue located at the bottom of the page.
In the students menu, you are able to import the students in your courseshell from D2L Brightspace to Crowdmark. Click on import students from Brightspace and then review their details. Once finished, click Enroll students.
In the generate menu, click on Generate PDF to generate PDF files for the assessment. These files can be printed yourself or sent to Ricoh to be printed for you.
Once you have downloaded the PDF files, click Finished! Go to Dashboard.
Set the due date and time of your assignment. Students will be able to submit and edit their submissions up until the time set.
Automatically set the lateness penalty per day to be deducted from the final mark your teaching team allocates to the student.
Allowing group submissions allows students to make their own groups and submit per group instead of individually. These groups can be set manually by an instructor, however students will still be able to modify them themselves.
You can write an optional assignment description to be displayed at the top of the page. This statement should include any general instructions about the assignment, but no specific questions.
You can create as many questions as you need by pressing the blue Add another question button at the bottom of the page.
The label is the identifyer for each question. By default Crowdmark labels each question using the Q1, Q2, Q3 prefix, but this can be modified as every question has a unique identifier. It is not reccomended that these labels contain words and should be kept as short and consise as possible.
The max points are the highest score the grader will be able to allocate to each question.
The Question Content window is where you will write in the question for the student to answer. This window can use special formatting (click on Formatting tips for instructions) and can include attachments by clicking on Attach Files.
When you are finished, click Save and continue.
Press the blue button in the centre of the screen to automatically load your D2L Brightspace student roster into the assignment.
You can then look over the student roster to make sure everything is correct and then click Enroll students.
Go to your assignment dashboard by clicking on Courses in the purple left-hand sidebar, then the name of your course, and then on the assignment name. Then, click Students from the sidebar on the left and then Edit Groups in the top right corner of the Students page.
You can edit groups by dragging and dropping them from the Unassigned students section on the left into groupings on the right.
Clicking Distribute to students will send out an automatically generated email with a link to the assignment for students to read the questions and submit their responses.
Clicking Overview grid will give you more information on who has submitted the assignment so far.
Navigating to the Students tab will provide you with an overview of student access to this assignment.
Applying Individual Accommodations
From the Student tab, search for a student then click on the link beside their name under the "Submission status" column.
A window will appear that includes accommodation options. You can choose to modify any of the follwing three options for this student:
a) Customize the due date: Select a due date and time for the individual student.
b) Customize lateness penalty: You can adjust the lateness penalty for an individual at any time, before or after the submission.
c) Customize time to complete: If the assessment is timed assessment, you can enter the number of hours and minutes this student will have to complete it.
You can print your Crowdmark assessment yourself using any regular printer. Please note that since Crowdmark supports different sizes of paper, the printer you use must support the size of paper you created your assessment on. Most printers default to regular 8.5x11" paper which is the same as the default page size in most writing applications such as Word and Google Docs.
Please Note: Getting your assessments printed and scanned by Ricoh will still require you to go through the Exam Matching process.
Toronto Metropolitan University has an agreement with Ricoh to take care of Crowdmark printing, binding, and scanning, as well as pick-up, delivery, and drop-offs. To do this:
Send your assessment/exam files to Ricoh by emailing them to email@example.com. This is the file you generated at the end of the assessment creation process.
To arrange for Ricoh’s printing/scanning pick-ups and deliveries, fill out the request form linked below. When you submit the form, Ricoh staff will receive an email providing the details of the pick-up/delivery arrangement (be as specific as possible when specifying the pick-up/delivery location). Printing/Scanning and uploading usually takes 24 hrs.
You can take an image of the completed booklets yourself by using a phone camera or regular scanner, and upload them as either individual files or packaged pdfs. The photos don't need to be labelled or in any specific order, as every question has its own QR code that identifies the question number and the package it is from.
Go to your assignment dashboard by clicking on Courses in the purple left-hand sidebar, then the name of your course, and then on the assignment name. Then, click on Upload booklets from the right-hand sidebar.
Drag and drop or click on the click here button to upload your files. You can upload multiple files at once. Wait until they have all uploaded and all of your pages have gone from the yellow pages pending indicator to the green pages completed indicator. Any issues will be indicated in red.
Make sure the green "Complete booklets" number eventually reaches the number of students that sat at the assessment.
If you choose to get your assessments printed by Ricoh in the printing stage, they will pick up the exams at the time you scheduled on the Google Form you submitted and will scan them for you. You will get an email when your exams are uploaded, and can move on to Exam Matching.
Go to your assignment dashboard by clicking on Courses in the purple left-hand sidebar, then the name of your course, and then on the assignment name.
Click on Match booklets on the right-hand sidebar under Administration.
Search for the student's name, email, or student number. Regardless of which one you use to search, the results will show up as the student's Ryerson email.
Click on the correct email displayed on the drop-down list.
Once you have verified that the user is correct, click Next Unmatched.
Once you have matched all the booklets to a student, the page will tell you that all uploaded booklets have been matched. It will save your changes automatically, so you are free to start the grading process.
Note: This step is not recommended as it requires scanning booklets twice, but can be useful if you're already going around the room and taking attendance during an exam.
You can match the assessments while the students are writing their assessments by scanning each paper's QR code and searching in the app for the student's name that they have hand written on the cover sheet or have displayed in front of them on a student ID.
To set up your account to use the Exam Matcher app, go to your My Account settings on the Crowdmark desktop site by clicking the abstract coloured icon in the top right corner of your screen. This icon may be a different colour for you, or may be a custom photo if you have already set it yourself.
Enter your full name in the Full Name text field.
Click Update my settings to save your changes.
Return to your assignment dashboard by clicking on Courses in the purple left-hand sidebar, then the name of your course, and then on the assignment name.
Click on Match with Exam Matcher App on the right-hand sidebar under Administration.
Click Generate token.
Take note of the number displayed on the screen under the most recent row of the Token column. Tokens can expire, so make sure to use the most recent one or generate a new one if neccesary.
Download the app by tapping on the Apple App Store or Google Play Store app on your phone, search for Exam Matcher, and then open the app once it has completed downloading.
This tutorial shows screenshots for the Apple iOS version of the app, but the Android version should work the same.
Enter your full name, making sure it matches what you entered in the first step on the desktop version of the site.
This can either be left blank or set to the room number your assessment is being written in. It will not affect your login.
Enter the token as it is displayed on the desktop site.
Click Sign in.
You will be prompted to enable Crowdmark's access to your device's camera. Tap OK.
Scan the first assessment's QR code. This can be the cover sheet, or the code at the top of any supplemental pages.
Search for the student's name, email, or student number in the app. Regardless of which one you use to search, the results will show up as the students Ryerson email.
If you've made an error, tap Unmatch this student and try again.
If the results are accurate, tap Match another and repeat the process for all the assessment sheets.
The system will automatically save between each scan. When you are finished, close the app and continue to the desktop version of the site. When your exams are all scanned and uploaded, they will be ready to be marked there.