Another gradeR Vignette

Written on May 22nd , 2022 by Christopher Kinson

Taylor Brown developed the gradeR package for R. His original vignette explains foundations of using the package. I am writing this vignette to supplement aspects of the original because I had questions. I explored answers for my questions and thought it might be useful to others.

The big picture is that the gradeR package can be thought of as a localized autograder (automatic grading of .R files on one’s own computer). With some additional work and licenses, it can be connected to online systems such as Gradescope and GitHub. This can make it a versatile and powerful tool, especially for large courses that incorporate computing or programming technology with or without teaching assistants, course assistants, or hourly employees.

Let’s take a look at some of the questions I had while learning about this helpful package. If you want to follow along and try out these ideas yourself, you can download the another-gradeR-vignette.zip, extract it, and set your working directory in R or RStudio to “another-gradeR-vignette”.

setwd("~/another-gradeR-vignette")

Can students submit .Rmd files?

Yes. gradeR automatically grades R code within .R or .r files. As someone who enjoys using notebooks and R markdown for reproducibility and checking student answers to questions about data wrangling and programming, I did not want to give up using .Rmd files for my courses. After searching online for possibilities, there was one: knitr::purl(). The purl() function within the knitr package is interesting in that it can convert .Rmd documents to .R documents through extracting the .R code from within that .Rmd file. It searches for R code chunks and removes all the code chunk scripting to keep the raw R code. It’s remarkable!

If you have created a main gradebook (see course-semester-gradebook.csv) file for storing students’ grades in your course, then you can loop through those students in order to extract the R code from their .Rmd files.

# import course-semester-gradebook.csv (official one for the course)
library(tidyverse)
netids <- read_csv("course-staff-resources/course-semester-gradebook.csv") %>%
  select(NetID)

# extract R code from .Rmd files
for (i in 1:nrow(netids)){
  knitr::purl(input = paste0("student-homework/student-homework00/homework00-", netids[i,], ".Rmd"),
            output = paste0("student-homework/student-homework00/homework00-", netids[i,], ".R"), 
            documentation = 0)
}

Above, I am repeating the extraction of R code chunks for each student in my roster. The netids object is a vector of student IDs for students in your roster; only these students’ files exist in your submission directory. The input= and output= arguments refer to file locations, while the documentation= argument controls what to do with the text and markdown syntax that is not R code. See the help documentation for details on these purl() arguments.

Can the instructor compute the total score per student?

Yes! While the original vignette describes using gradeR interactively, I know that it will be difficult to remember the flow of code each time I need to compute grades for all students per assignment. Thus, I will suggest a separate R file (see homework00-gradeR-script.R) that contains all the code mentioned in the original’s interactive section. One thing Brown didn’t mention was what to do now that the grades object has been created. Are we done after that?

The grades object (created within homework00-gradeR-script.R) is structured as \(n\) rows and \(p\) columns, where \(n\) represents the number of .R files within the submission directory, and \(p\) represents the number of test_that() functions used in the homework00-instructor-solutions.R file. The values in those \(p\) columns are either 1 (correct) or 0 (incorrect). In a typical assignment, I would likely setup the test_that() functions to correspond to a unique problem/question in the assignment with names that make sense. In this way, we know exactly which column is referring to which problem/question in the assignment and its point value. Then after all problem/questions are automatically graded, we can run a row sum on that grades object. Since I do teach about coding in the tidyverse, I’ll use tidyverse code in this row sum example.

library(tidyverse)
# to add specific columns for easier grade distribution
grades2 <- as_tibble(grades) %>%
  rowwise() %>%
  mutate(TotalPoints =  sum(c_across(problem01:problem06)))

grades2
## # A tibble: 3 × 8
## # Rowwise: 
##   id     problem01 problem02 problem03 problem04 problem05 problem06 TotalPoints
##   <chr>      <dbl>     <dbl>     <dbl>     <dbl>     <dbl>     <dbl>       <dbl>
## 1 stude…         1         1         0         0         1         1           4
## 2 stude…         1         1         1         1         1         1           6
## 3 stude…         1         0         1         0         1         1           4

Recently, I learned that the rowwise() function operates like the group_by() function in that it doesn’t do anything explicitly. It works behind-the-scenes storing how the process for the next data wrangling steps will affect the tibble currently in the pipeline.

What kinds of problems/questions can I write?

Given that test_that() requires R code for the automatic grading, it might seem limiting in the kinds of questions we as instructors may want to ask students in assignments. But maybe this just means we need to be more creative. Seems to me that any type or style of question can be written and automatically graded.

Multiple choice and True/False

An example of a multiple choice problem/question is Problem #6 in the homework00-instructor-solutions.R. An example of a True/False problem/question is Problem #2.

Problem: Choose a single letter that represents the most correct choice, then create an R vector named prob06 with the lower case letter chosen in quotes. For example, prob06 <- "e". This R code should be written in the code chunk of the Answer section below. This code chunk should be executable and able to be evaluated.

  • Question: In which of the following cases is the file name in bold Homework01Kinson2 written?

  1. pascal case

  2. snake case

  3. camel case

  4. kebab case

  5. none of these

Multiple answer

An example of a multiple answer problem/question is Problem #4 in the homework00-instructor-solutions.R

Problem: Choose all letters that represent the correct choice(s), then create an R vector named prob04 with the lower case letter(s) chosen in quotes. If choosing more than one lower case letter, the elements in the vector prob04 need to be separated by a comma. For example, prob04 <- c("e","f"). This R code should be written in the code chunk of the Answer section below. This code chunk should be executable and able to be evaluated.

  • Question: Which of the following cases are discussed in this course?

  1. pascal case

  2. snake case

  3. camel case

  4. kebab case

  5. none of these

Fill-in-the-blank and Exact answer

I omit a specific fill-in-the-blank problem/question, but I recognize this style as an exact answer problem/question. Examples of exact answer problems/questions include: Problems #1 and #3.

Problem: Create an R vector named prob01 that contains the following 3 elements (in this order): first name, last name, and netID with each element enclosed in quotes. For example, prob01 <- c("Chris","Kinson", "kinson2"). This R code should be written in the code chunk of the Answer section below.

Short answer (1 complete sentence) and Long answer (3 or more complete sentences)

I omit a specific short answer problem/question, but I recognize this style as a version of a long answer problem/question. An example of a long answer problem/question is Problem #5.

Problem: Create a character-type vector of length 1 in R named prob05 with at least two complete sentences that answer the following question(s). This R code should be written in the code chunk of the Answer section below. This code chunk should be executable and able to be evaluated.

  • Question: In your own words, describe the best way(s) for you to learn new things. What specific things do you require for you to retain information? How frequently do you refer back to notes or textbooks when learning new information?

Can we connect this to GitHub?

Yes, especially if you are the owner of a GitHub organization (org for short), where your students are members of that org. In this sense, GitHub can be considered as your Learning Management System (LMS). All that’s needed is to connect your local machine to your GitHub account such that it is authorized for your account. Then run code from the command line.

We can even use R script to run git operations such as:

for (i in 1:nrow(netids)){
url <- paste0("git clone ", "https://github.com/course-semester/", netids[i], ".git",
               " ", netids[i])
  shell(url)
}

I hope to share more on GitHub as an LMS at a later date.

Feel free to share!

You may also enjoy...