This page is written for graders, TAs, and other instructors.
It is a draft/work-in-progress, so if you have any ideas, let me know!
Open source, local, and/or self-hosted systems:
- ssh/remote capable, open, local, transparent, debug-capable, git compatible
- https://submitty.org/ (RPI's overly-fancy, overly-GUI system, also does grading)
- self-hosted, web-based student interface, not debug-capable
- A system for containing executed code
Pre-reading on Git (if you need it):
Services/software for managing bulk student repositories (this may overlap with grading functionality too).
- https://github.com/redkyn/assigner (Keep it local. This is what we use here at MST!)
- https://github.com/redkyn/assigner/blob/master/TUTORIAL.md (the tutorial to read)
- This is pre-installed in the VirtualBox OVA I distribute (below), so you don't need to install yourself if you use it.
- Actually read the documentation for assigner in-full before trying to use it.
Here I detail both:
1) Student perspective
2) Faculty perspective
#!/bin/bash # In a terminal, make and cd into whatever directory you want to use as the superdirectory, such as mkdir CS-1500-assignments/ cd CS1500-assignments/ # Using a web browser, sign into https://git-classes.mst.edu with campus credentials. # Click on the class group for your class # Find your git repo in the web interface git clone: https://git-classes.mst.edu-example-assignment # Head into the repository cd example-assignment # make your changes, do assignment vim examplecode.py # add, commit, push to remote git add whateveryouchangedoradded.py git commit -m "commit message" git push # In web browser, check https://git-classes.mst.edu to make sure your changes are up! echo "Yay, I'm done, and know my grade instantly!"
Some work is done:
- just once ever,
- some once per semester, and
- some is for each assignment
Generate an SSH key on the computer you're going to assign on,
and then copy it into the git-classes web-interface.
In git-classes web-interface, generate an access token,
and save it somewhere safe (you'll need it later for assigner init).
In Canvas web-interface, generate an access token,
and save it somewhere safe (you'll need it later for assigner init).
#!/bin/bash # create super-folder on your local computer for each class: e.g., mkdir 2019-FS-CS1500-A # cd into class folder: e.g., cd 2019-FS-CS1500-A # keeps the student code in a sub-folder mkdir student_submissions # create git-classes group named the same (or preferably let assigner do it just below). assigner init # If you did not create the group manually on gitlab, let assigner do it for you now. # Find your course assigner canvas list # Import students into config file assigner canvas import classnum section # where classnum is from the output of previous command, and # where section is section-letter, if any, else, A
#!/bin/bash # Head into class folder cd 2019-FS-CS1500-A # Make new assignment, and it's grader partner, and clone the empty repos assigner new assignment01 assigner new assignment01-grader git clone whateverlinkassignernewgeneratedabove git clone whateverlinkassignernewgeneratedabove-grader # head into the grader repo cd assignment01-grader # write the solution to your assignment # write the unit tests and CI for your assignment, checking they all pass # Head back to class folder cd .. # Copy your solution into the student repo cp -r assignment01-grader ../assignment01 cd ../assignment01 # break/remove your solution (or parts of it) from the solution # check the CI/tests fail # Head back to class folder cd .. # Assigning actually generates student repos from your template assigner assign assignment01 # --branch main # Opening adds them as developers assigner open assignment01 # You can check on them in bulk: assigner status assignment01 # When you need to close the assignment, gives students reporter rights only: assigner lock assignent01 # Pulls all the student repos assigner get assignment01 student_submissions/ # If you grade locally, then run your autograder, # modifying the student repos in student_submissions/ assigner commit -S assignment01 "message to students" student_submissions/ -u -a '*' # --branch main # -S assumes you have a gnupg2 key with an email that matches the listed public email account in gitlab web-interface. # Pushes edits to student repos back up: assigner push assignment01 student_submissions/ # --branch main # Just shows the scores, if present, in results.txt in student repo assigner score all assignment01 # You can use gitlab CI/CD to generate the results.txt, # or push it back manually # pushes scores from results.txt file in repo to Canvas, # if assignment is named the same as a canvas assignment assigner score all --upload assignment01
If you need to make changes it the middle of an assignment's release
#!/bin/bash # lock so that your edits don't collide with students, and create remote conflicts assigner lock assignent01 # get the student code assigner get assignment01 student_submissions/ # --branch main # edit assignment # assumes you have a gnupg2 key with an email that matches the listed public email account in gitlab web-interface. assigner commit -S assignment01 "message to students" student_submissions/ -u -a '*' # --branch main # push back assigner push assignment01 student_submissions/ # --branch main # give students developer access back assigner unlock assignment01
Pre-reading on testing and debugging (not required):
Classes:DataStructuresLab:Content (section on Unit tests)
Check out our auto-grading suite,
and design your own Gitlab-CI based auto-graded assignments much more easily.
- Write a script for students to run, as a sanity check, even if you don't give out all the tests, which gives them a feedback file, easily readable.
- Write modular small unit tests, which are to be wrapped twice: once at the project level, and once at the multi-student level
- Base the core design pattern of your first wrapper/grader around a single-repo single-student script, meant to test one instance of the whole project.
- That project-wrapper script should compute the grade.
- Then, after that whole-project grading script is done, write a cross-student wrapper, which should be the most minimal of all, just running the project grader for each student.
- Validate modular unit tests, which return/exit code to parent bash process: (0 pass, 1 fail)
- Write a bash script which runs all unit tests for the project, so students can run this locally.
- Mimic that bash script in a .gitlab-ci.yml file, which kick-starts CI for the repository, passing if all return codes are 0, and failing if any are not 0.
- Alternative 1: Grab the number in each student feedback file, tabulate it, and upload to canvas (easily automated).
- Alternative 2: Pushing to Canvas from repositories: in development now. Email me if you want to check it out.
To determine whether students copied code from each other, or the internet,
there are several options:
My crude copy-checker is much easier to run than the following two:
Moss does pretty well, and compares to an online, and bigger database than just students against each other. It is however, a bit of a pain to run.
How to make sure all students are running in an environment compatible with the tests you write?
I recommend using a Fedora Security Labs virtual machine (XFCE desktop).
We provide installation scripts for configuration in each assignment.
I created a simple docker setup.
It is included in the template for each grade.sh example:
It's pretty simple.
There are 4 files:
Does nothing but echo that you should install docker.
Is the template upon which the docker_build.sh runs.
You should edit this per-assignment.
Actually builds the container.
You can tweak this, depending on needs.
Runs the container from the given repo folder.
From a student perspective:
# Install docker # Open one terminal git clone https://your_repo cd your_repo/ ./docker_build.sh ./docker_run.sh
This terminal now runs a fully configured Linux environment to run your auto-grader, test, etc.
It shares a folder with the current directory (your repo),
so that you can edit in your host, and run in the container.
Rather than pay for, or even worse, rent, Clickers, or pay for Kahoot... try these:
- Use our Canvas-based (Kahoot mimic), Cahoot: https://gitlab.com/classroomcode/cahoot
- Check out this very cool QR-code based CRS: https://get.plickers.com
For students learning branching and looping, generating flowcharts from code can be quite helpful.
(Flowcharts are also known as CFGs:
#!/bin/bash # install pip3 install py2cfg --user # generage flow-chart py2cfg yourcode.py # debug py2cfg yourcode.py --debug # generate a diffable file for grading py2cfg --diffable yourcode.py yourcode.diff
I highly recommend https://zulipchat.com as a platform for students to ask questions of you, each other, graders, tutors, etc.
https://www.discourse.org/ is pretty nice too, but not quite as seamless, quick, and efficient as Zulip, and a slightly different model, more targeted at public-facing organizations.