ClassroomCode

____________________________________________________


This page is written for graders, TAs, and other instructors.
It is a draft/work-in-progress, so if you have any ideas, let me know!



Code grading and/or learning platforms

Open source, local, and/or self-hosted systems:


Managing bulk Git repositories

Pre-reading on Git (if you need it):
Classes:DataStructuresLab:Content:03-VersionControl


Services/software for managing bulk student repositories (this may overlap with grading functionality too).


Local/open:


Assigner

Here I detail both:
1) Student perspective
2) Faculty perspective


1) Student work-flow example

#!/bin/bash

# In a terminal, make and cd into whatever directory you want to use as the superdirectory, such as
mkdir CS-1500-assignments/
cd CS1500-assignments/

# Using a web browser, sign into https://git-classes.mst.edu with campus credentials.
# Click on the class group for your class
# Find your git repo in the web interface
git clone: https://git-classes.mst.edu-example-assignment

# Head into the repository
cd example-assignment

# make your changes, do assignment
vim examplecode.py

# add, commit, push to remote
git add whateveryouchangedoradded.py
git commit -m "commit message"
git push

# In web browser, check https://git-classes.mst.edu to make sure your changes are up!
echo "Yay, I'm done, and know my grade instantly!"


2) Faculty work-flow example

Some work is done:

  • just once ever,
  • some once per semester, and
  • some is for each assignment

a. Work done once ever (or until access tokens expire)

Generate an SSH key on the computer you're going to assign on,
and then copy it into the git-classes web-interface.


In git-classes web-interface, generate an access token,
and save it somewhere safe (you'll need it later for assigner init).


In Canvas web-interface, generate an access token,
and save it somewhere safe (you'll need it later for assigner init).


b. Work done once per semester
#!/bin/bash

# create super-folder on your local computer for each class: e.g., 
mkdir 2019-FS-CS1500-A

# cd into class folder: e.g., 
cd 2019-FS-CS1500-A

# keeps the student code in a sub-folder
mkdir student_submissions 

# create git-classes group named the same (or preferably let assigner do it just below).
assigner init
# If you did not create the group manually on gitlab, let assigner do it for you now.

# Find your course
assigner canvas list

# Import students into config file
assigner canvas import classnum section
# where classnum is from the output of previous command, and
# where section is section-letter, if any, else, A


c. Work done once per assignment
#!/bin/bash

# Head into class folder
cd 2019-FS-CS1500-A

# Make new assignment, and it's grader partner, and clone the empty repos
assigner new assignment01
assigner new assignment01-grader
git clone whateverlinkassignernewgeneratedabove
git clone whateverlinkassignernewgeneratedabove-grader

# head into the grader repo 
cd assignment01-grader
# write the solution to your assignment
# write the unit tests and CI for your assignment, checking they all pass
# Head back to class folder
cd ..

# Copy your solution into the student repo
cp -r assignment01-grader ../assignment01
cd ../assignment01
# break/remove your solution (or parts of it) from the solution
# check the CI/tests fail
# Head back to class folder
cd ..

# Assigning actually generates student repos from your template
assigner assign assignment01 # --branch main

# Opening adds them as developers
assigner open assignment01

# You can check on them in bulk:
assigner status assignment01

# When you need to close the assignment, gives students reporter rights only:
assigner lock assignent01

# Pulls all the student repos
assigner get assignment01 student_submissions/

# If you grade locally, then run your autograder, 
# modifying the student repos in student_submissions/
assigner commit -S assignment01 "message to students" student_submissions/ -u -a '*' # --branch main
# -S assumes you have a gnupg2 key with an email that matches the listed public email account in gitlab web-interface.

# Pushes edits to student repos back up:
assigner push assignment01 student_submissions/ # --branch main

# Just shows the scores, if present, in results.txt in student repo
assigner score all assignment01
# You can use gitlab CI/CD to generate the results.txt, 
# or push it back manually

# pushes scores from results.txt file in repo to Canvas, 
# if assignment is named the same as a canvas assignment
assigner score all --upload assignment01

If you need to make changes it the middle of an assignment's release 

#!/bin/bash

# lock so that your edits don't collide with students, and create remote conflicts
assigner lock assignent01

# get the student code
assigner get assignment01 student_submissions/ # --branch main 
# edit assignment

# assumes you have a gnupg2 key with an email that matches the listed public email account in gitlab web-interface.
assigner commit -S assignment01 "message to students" student_submissions/ -u -a '*' # --branch main

# push back
assigner push assignment01 student_submissions/ # --branch main

# give students developer access back
assigner unlock assignment01

Testing and grading

Pre-reading on testing and debugging (not required):
Classes:ComputationalThinking:Content:10-DebuggingTesting
Classes:ComputationalThinking:Content:19-TestingFrameworks
Classes:DataStructuresLab:Content (section on Unit tests)


https://www.fuzzingbook.org/


A generalized auto-grading suite: grade.sh

Check out our auto-grading suite,
and design your own Gitlab-CI based auto-graded assignments much more easily.
https://gitlab.com/classroomcode/grade-sh


General guidelines to create an assignment and grading suite

  • Write a script for students to run, as a sanity check, even if you don't give out all the tests, which gives them a feedback file, easily readable.
  • Write modular small unit tests, which are to be wrapped twice: once at the project level, and once at the multi-student level
  • Base the core design pattern of your first wrapper/grader around a single-repo single-student script, meant to test one instance of the whole project.
  • That project-wrapper script should compute the grade.
  • Then, after that whole-project grading script is done, write a cross-student wrapper, which should be the most minimal of all, just running the project grader for each student.

Using GitLab continuous integration for unit testing

  • Validate modular unit tests, which return/exit code to parent bash process: (0 pass, 1 fail)
  • Write a bash script which runs all unit tests for the project, so students can run this locally.
  • Mimic that bash script in a .gitlab-ci.yml file, which kick-starts CI for the repository, passing if all return codes are 0, and failing if any are not 0.

Grade extraction from repos to Canvas

  • Alternative 1: Grab the number in each student feedback file, tabulate it, and upload to canvas (easily automated).
  • Alternative 2: Pushing to Canvas from repositories: in development now. Email me if you want to check it out.

Cheat/copy checking

To determine whether students copied code from each other, or the internet,
there are several options:


My crude copy-checker is much easier to run than the following two:
https://gitlab.com/classroomcode/copy_checker


Moss does pretty well, and compares to an online, and bigger database than just students against each other. It is however, a bit of a pain to run.
https://theory.stanford.edu/%7Eaiken/moss/


Rensselaer's submission system is called Submitty (https://submitty.org/) as discussed above.
Their plagiarism detection tool is called Lichen (See https://github.com/Submitty/Lichen for Lichen;


Working environment

How to make sure all students are running in an environment compatible with the tests you write?


Pre-reading:
Classes:DataStructuresLab:Content:00-VirtualMachines


Standardized virtual machine

I recommend using a Fedora Security Labs virtual machine (XFCE desktop).
We provide installation scripts for configuration in each assignment.


Docker working environment

I created a simple docker setup.
It is included in the template for each grade.sh example:
https://gitlab.com/classroomcode/grade-sh/cpp-example
https://gitlab.com/classroomcode/grade-sh/bash-example
https://gitlab.com/classroomcode/grade-sh/python-example
https://gitlab.com/classroomcode/grade-sh/rust-example
It's pretty simple.
There are 4 files:


docker_install.sh
Does nothing but echo that you should install docker.


Dockerfile
Is the template upon which the docker_build.sh runs.
You should edit this per-assignment.


docker_build.sh
Actually builds the container.
You can tweak this, depending on needs.


docker_run.sh
Runs the container from the given repo folder.


From a student perspective: 

# Install docker
# Open one terminal
git clone https://your_repo
cd your_repo/
./docker_build.sh
./docker_run.sh

This terminal now runs a fully configured Linux environment to run your auto-grader, test, etc.
It shares a folder with the current directory (your repo),
so that you can edit in your host, and run in the container.


Classroom Response Systems (Alternatives to: Clicker, Kahoot, etc.)

Rather than pay for, or even worse, rent, Clickers, or pay for Kahoot... try these:


Generating flowcharts from code

For students learning branching and looping, generating flowcharts from code can be quite helpful.
(Flowcharts are also known as CFGs:
https://en.wikipedia.org/wiki/Control-flow_graph


Check out our project, which turns arbitrary python code into a colorful CFG:
https://gitlab.com/classroomcode/py2cfg
https://py2cfg.readthedocs.io/en/latest/ 

#!/bin/bash

# install
pip3 install py2cfg --user

# generage flow-chart
py2cfg yourcode.py

# debug
py2cfg yourcode.py --debug

# generate a diffable file for grading
py2cfg --diffable yourcode.py yourcode.diff

Communication platforms for students, TAs, graders, help Q&A, etc.

I highly recommend https://zulipchat.com as a platform for students to ask questions of you, each other, graders, tutors, etc.


https://www.discourse.org/ is pretty nice too, but not quite as seamless, quick, and efficient as Zulip, and a slightly different model, more targeted at public-facing organizations.


Backlinks: index:ResearchDevelopment