Hello all, I am starting from scratch on an "automated assignment Grading system" and I believe mturk would be a good solution to perform the actual work of grading... But I'm here for your feedback and wisdom.
1. Would mturk workers be consistently able to accurately grade student assignments if they have the original lesson, the assignment instructions, the rubric and/or answer key and the student submission?
2. How likely am I to consistently get quality reasons for point deductions?
3. What should I include in my HIT to ensure quality, consistency and accuracy?
4. Assuming student submissions are complete (all requirements filled) and it takes no more than 5 minutes to check answers, what should be the HIT value for each assignment?
I am looking forward to your replies!
1. Would mturk workers be consistently able to accurately grade student assignments if they have the original lesson, the assignment instructions, the rubric and/or answer key and the student submission?
2. How likely am I to consistently get quality reasons for point deductions?
3. What should I include in my HIT to ensure quality, consistency and accuracy?
4. Assuming student submissions are complete (all requirements filled) and it takes no more than 5 minutes to check answers, what should be the HIT value for each assignment?
I am looking forward to your replies!