STAT 440 Fall 2024 Unofficial Syllabus

Written on September 22nd , 2024 by Christopher Kinson

STAT 440 Statistical Data Management

3/4 Credit Hours - Major Elective

Sections 1UG/1GR

Fall 2024 - Syllabus

Table of Contents

Course Description

Statistical Data Management (STAT 440) is a focused data wrangling course that aims to cover various types of data storage, manipulation, cleaning, and extractions and to apply these methodologies in R. This means that students must have a laptop that they bring to class each day. This course is not a traditional lecture format. Most lecture content will be distributed in video form at the beginning of the semester. Notes are reproducible documents that include examples and code. Videos and notes should be read and understood before (and outside of) class. The classroom space and time will be for application, practice, and assessment of course content in those videos and notes. The expectation is that students will gain competency in exploring, organizing, designing, creating, storing, cleaning, wrangling, sharing, and using data, all of which are commonly done prior to data analysis. Critical and creative thinking and efficient coding will be encouraged. Concepts covered in this course will build upon each other. Thus, students can expect all assessments to be cumulative. An autograder is used to grade assignments. Students should be sufficient and comfortable in R prior to beginning this course. The RStudio offers reproducible documentation with Markdown syntax which will support long-term learning opportunities. Git propels students’ capacity for collaboration and organization as well as version control of documentation and files.

Learning Objectives

These learning objectives are important because they connect the physical know-how with the technical knowledge of the course.

  • Students will assess effectiveness, organization, and intent from a published data set

  • Students will explore data sets of various types

  • Students must design well-organized, clean data sets for the purpose of data analysis

  • Students will present data management work in a reproducible document file using Markdown syntax and R code chunks. No local data files will be utilized.

  • Students must demonstrate critical thinking and creativity through asking questions about a given data set

  • Students must be able to explain and summarize data wrangling code

  • Students will share and discuss data management ideas, coding snippets, and other thoughts to aid in meaningful dialogue

  • Students must recall important data management concepts

  • Students will reflect on their own learning of data management principles

  • Students will build data wrangling tools, apps, and dashboards and store all work using git, a version control software.

  • Students will collaborate on lab assignments.

  • Students will reproduce and replicate data visualizations.

Course Staff

  • Instructor - Christopher Kinson (kinson2@illinois.edu)

profile picture of Christopher Kinson

Course Specifics

Course Website

The course website is https://github.com/illinois-stat440. This course is operating as an organization named illinois-stat440 within GitHub. Students should bookmark or save the link above in their browser for future use, because it contains access points to all repositories, course materials including notes, assignments, and lecture videos.

Prerequisites

The prerequisites for this course are the following:

  • A laptop (not a netbook) with most up-to-date versions of R and RStudio installed. If using a netbook or Chromebook, please setup an RStudio Cloud account.

  • STAT 400 or STAT 409

  • Operating knowledge of computers such as locating a file, creating a directory, saving a file, compressing a file, extracting a compressed file, keyboarding, and fundamental troubleshooting

  • Operating knowledge of R such as understanding various object types, mathematical and logical operators, and value types and their coercion, as well as creating user-defined functions and fundamental R troubleshooting

Meeting Schedule and Expectations

  • For section 1UG/GR, the class meets at 10:00 am - 10:50 am in Room 32 of the Psychology Building on Mondays, Wednesdays, and Fridays.

  • There will be asynchronous lecture videos with links posted on the Course Website (see in videos directory of the course_content repository). See the Instructional Activities section below for more details.

  • All students are expected to fully participate in class regularly.

  • All students are expected to do the following before coming to class each week: read the course notes, practice coding the examples (code chunks) within the course notes, watch the lecture videos, and complete assignments.

  • Course content - syllabus, notes, videos, and weekly schedules, discussion board (as Discussions tab) - will be found on the Course Website via the course_content repository. Do check the course_content repo often for updates and announcements. Students are encouraged to clone and pull the course_content repository daily if accessing it remotely via git.

Office Hours

Any and all times listed in this document are in current US Central Time. Take care to adjust clocks when daylight savings time occurs.

Office hours will be remote. If a student has a specific question, but cannot attend the office hours, then that student should post their question in the discussion board (see Discussions tab of the course_content repo). If a student wants one-on-one assistance from the course staff at an alternative time, then that student should email the course staff in order to schedule a Zoom meeting.

Textbooks

There is no required textbook, but students may find the texts below to be helpful. These are all free and accessible to students for further reading. The Instructor may refer to certain sections of these texts in the course content. The asterisk * means these are accessible from the University Library as E-books.

R

  • *Data Wrangling with R. Boehmke. Springer Cham. http://www.library.illinois.edu/proxy/go.php?url=http://dx.doi.org/10.1007/978-3-319-45599-0
  • *R for Data Science. Wickham and Grolemund. O’Reilly Media, Inc. https://learning.oreilly.com/library/view/-/9781491910382/?ar
  • Mastering Shiny. Wickham. https://mastering-shiny.org/

Markdown

  • The Markdown Guide. Cone. https://www.markdownguide.org/getting-started

RMarkdown

  • RMarkdown Cheat Sheet. RStudio. https://rstudio.com/wp-content/uploads/2015/02/rmarkdown-cheatsheet.pdf

Git and GitHub

  • Happy Git and GitHub for the useR. Bryan et al. https://happygitwithr.com/

Software

The course requires students to already have a fundamental and operational understanding of R. It is recommended that students with no familiarity in R understand that this course will not discuss fundamental and operational usage of R.

Calendar

Below is a calendar of topics and tentative assignment deadlines. Any deadlines apply to all sections of STAT 440 unless otherwise noted.

Week 2024 Dates Topics Covered
01 08/26 - 08/30 Introduce the course and software - RStudio, RMarkdown, Git, and GitHub, Git and GitHub tips, Autograder pitfall examples
02 09/02 - 09/06 LABOR DAY on September 02 (no class or office hour), Autograder pitfall examples, Git and GitHub tips, Loops and conditional execution, Apply family of functions, Vectorization
03 09/09 - 09/13 What is data, Structures of data, delimiters, and file extensions, Accessing and importing data, Exporting data, Handling dates and times
04 09/16 - 09/20 Assigning objects, Arranging data, Reshaping data, Data expansion, Data reduction
05 09/23 - 09/27 Regular expression and string manipulation, Summarizing data, Combining data
06 09/30 - 10/04 Data visualization using tidyverse
07 10/07 - 10/11 SQL queries and sub-queries
08 10/14 - 10/18 Midterm Exam 01 on Wednesday October 09, Reassessment for Midterm Exam 01 on Friday October 11, Focus on Midterm Exam
09 10/21 - 10/25 Validating data, Cleaning data using tidyverse
10 10/28 - 11/01 Accessing and importing data via web scraping, Accessing and importing data via an API
11 11/04 - 11/08 ELECTION DAY on November 05 (no class or office hour), Shiny apps and dashboards
12 11/11 - 11/15 Data exploration using curiosity and creativity exercises
13 11/18 - 11/22 Midterm Exam 02 on Wednesday November 20, Reassessment for Midterm Exam 02 on Friday November 22, Focus on Midterm Exam
14 11/25 - 11/29 FALL BREAK (no class or office hour)
15 12/02 - 12/06 Interview with a data worker, Data workers and responsibilities, Discussion about careers, graduate school, and the future post-STAT440, Focus on Final Exam
16 12/09 - 12/13 READING DAY - December 12 (no more classes or office hours), Interview with a data worker, Data workers and responsibilities, Discussion about careers, graduate school, and the future post-STAT440, Focus on Final Exam
17 12/16 - 12/20 Final Exam for section 1 UG/GR at 8:00 am - 11:00 am on Thursday December 19

Grading Breakdown

12 Piloted Practice Assignments: 12 points total (1 point each)

  • pp01-pp16 are weekly comprehensive assignments due on Mondays (before class starts). No student receives credit for pp01, pp02, pp03, and pp14. Credit is earned for pp04-pp16, excluding pp14.

2 Lab Assignments: 8 points total (4 points each)

  • Lab01 series begins Week 04 (September 16) and ends Week 07 (October 11). Each student will be an in-person in-class driver only once in this series. Labs take place on Wednesdays and Fridays in this series.
  • Lab02 series begins Week 09 (October 21) and ends Week 12 (November 15). Each student will be an in-person in-class driver only once in this series. Labs take place on Wednesdays and Fridays in this series.

2 Midterm Exams: 20 points total (10 points each)

  • Midterm Exam 01 is a 50-minute (student’s section’s class start and end time) in-person exam (October 16). It contains conceptual and applied problems.
  • Midterm Exam 02 is a 50-minute (student’s section’s class start and end time) in-person exam (November 20). It contains conceptual and applied problems.

1 Final Exam: 30 points total

  • Final Exam is a 3-hour remote exam containing conceptual and applied problems. This exam schedule is in alignment with the University Final Exam Schedule (see https://registrar.illinois.edu/courses-grades/final-exam-schedule-public/ for more information). Adjust your schedules and travel plans accordingly. Any undergraduate requests for a conflict exam will be denied.

Course Total Points: 70 points

Final Letter Grades

When computing final grades, students can add up their scores on the assignments. The resulting sum will determine which letter grade they earn when the course is completed. There is only one $+$ letter grade in this course. All other letter grades are without $+/-$. Points are not rounded.

Lower bound Upper bound Letter grade
70.000 points 80.000 A+
63.000 points 69.999 points A
56.000 points 62.999 points B
49.000 points 55.999 points C
42.000 points 48.999 points D
0.000 points 41.999 points F

Instructional Activities

Students should read the course notes, watch the lecture videos, and attempt the assignments. If or when students get stuck, then they should ask questions in the i) Discussions Board, ii) Office Hours, or iii) via email (preference in this order). In addition to lecture videos and office hours, the following activities and tools will be useful for students.

Course Notes

The course notes perform the duty of a textbook for this course. Yes, there is a lot of information in the notes, but it is useful to read it for the important parts and return to it for details after attempting the assignments.

Discussions

This discussion board, which exists as a tab on the course_content repository, is one of the best ways to communicate with classmates and course staff. Questions can be seen quickly and receive a rapid response. Students are encouraged to use this board, but there is no requirement to participate in the discussion board.

Do use the board to openly discuss ideas about the course such as questions about content, deadlines, notes, data, etc. If a student specifically wants the course staff to respond, then student should use the mention @staff when posting in the board. The things discussed here should be of a non-personal and non-private matter. If student has a personal or private matter to discuss with the Instructor, please send an email to kinson2@illinois.edu. Additionally, the conversation in the discussion board should be respectful of people’s differences and cannot be used to speak negatively about anyone or harm anyone.

The course staff will view and respond to content on Tuesdays and Thursdays (at a minimum). The course staff can be expected to spend at least 30 minutes on Tuesdays and Thursdays monitoring the Discussions.

Grade Disputes

Please email the Instructor (kinson2@illinois.edu) with your requests and disputes within 7 days (i.e. one week) of your grade being returned for a specific assignment.

Assignments

Piloted Practice Assignments

These are guided comprehension assignments to be completed by each student as an individual before the class begins on Mondays. There are 4 ungraded piloted practice assignments and 12 graded piloted practice assignments for the semester. The shortname for these assignments is pp. They are numbered by their corresponding week number. For example, pp04 corresponds to week 04 piloted practice and is due on Monday of Week 04.

These assignments are to be submitted in main branch of the student’s repository inside of the piloted-practices directory, which exists within the illinois-stat440 organization in GitHub. See Course Website for web links. These pp’s are graded for completion, not correctness. However, the code must follow reproducibility guidelines and must not contain executable errors or warnings.

Lab Assignments

These are in-person lab sessions that contain 4 problems and are due at the end of the class period on Mondays and Wednesdays by the “driver” for that day. The course is located in an iFLEX classroom, which allows for ease of communication, collaboration, and displaying content on large screens at stations. Each week (beginning in Week 04), randomly chosen students (“drivers”) will complete the lab assignment seated at specified stations, while the remaining students (“passengers”) at the stations will help the drivers by giving them ideas and advice on how to complete the problems. Drivers must submit their labs. Passengers are not allowed to type on driver’s laptops. All students must bring their laptops to class each day. Lab assignments are intended to push students to apply concepts covered in the course notes and lecture videos and encourage students to work together as a team in a limited amount of time. Each student must complete two lab assignments as the driver by the respective lab due dates. A schedule will be given to students displaying exactly when each student is to be driver and which station number the driver should sit. Students should not deviate from this schedule. Late submissions of lab assignments will not be accepted. There will be no make-ups for any missed labs. This policy applies to any students who add the course late to their registration.

Midterm Exams

Each midterm exam contains 10 problems and is in-class and in-person. The exam will be closed notes, closed book. Each midterm file must be submitted on the Course Website via a student’s repo. The midterm exam will be graded for correctness and completeness. Students can expect the midterm exam to include a structure similar to lab assignments with a possible mixture of completion, correct/incorrect, and open-ended questions. Late submissions on any Midterm Exam submission will not be accepted or graded. There will be no make-ups for any missed Midterm Exam submission.

There is one Reassessment Midterm Exam for each original Midterm Exam to give students who struggle with the original Midterm Exam submission a second opportunity to review course content and improve their score: Reassessment for Midterm Exam 1 and Reassessment for Midterm Exam 2. The Reassessment Midterm Exams take place on the Friday following the original Midterm Exam. The Reassessment Midterm Exam grade may be used to replace the original Midterm Exam grade, for the respective exam number. For example, a student’s Reassessment for Midterm Exam 1 grade of 10 points may only be used to replace the grade of the original Midterm Exam 1 grade of 0 points. All exams (original and reassessment) are in-person exams. For students who do not wish to take the reassessment, they may use the reassessment exam date as a study at home day.

Final Exam

The final exam is a remote exam by retrieving and submitting via Git/GitHub containing one problem per point value. The exam will be open notes, open resources, and take up to 3 hours to complete. The final exam will be graded for both correctness and completeness. Students can expect the final exam to include a structure similar to labs and the midterm exams with a possible mixture of completion, correct/incorrect, and open-ended questions. Students should not expect solutions to be provided for the final exam. Late submissions on any Final Exam will not be accepted or graded. There will be no make-ups for any missed Final Exam.

Autograder

The code we write in this course must be reproducible - verifiable by any computer running the same exact code and receiving the same exact result as the original source. It is important that code does not contain executable errors and warnings. Submitting code with executable errors and warnings shows that a student is not following one of the course learning objectives. Submitting error-producing code also shows that there is no regard for what reproducibility means. There is an autograder used in this course to grade assignments. The autograder is not forgiving. It will scan the entire file and check for base R executable errors and warnings as well as grade the assignment for correctness and completion. Objects created at the top of the file which are overwritten at the bottom of the file will be considered incorrect by the autograder. When the autograder detects a base R executable error or warning, it will stop grading the student’s submission and assign a grade of 0 for the assignment.

To follow reproducible coding guidelines and prevent executable errors and warnings, be sure to do the following (in no particular order):

  1. Save the file with the correct name. Your netid should replace anything saying ‘netid’.

  2. Save the file in the correct location. pp** assignments belong directly in the piloted-practices directory of your repo on the main branch. All lab** assignments belong directly in the labs directory of your repo on the main branch. All exam** belongs in the exams directory of your repo on the main branch. Any sub-directories within these directories is inappropriate.

  3. Within a code chunk, explicitly write code that attaches or loads a package using either library() or environment call package_name:: if you use a package to produce your result.

  4. Change your RStudio Global Options’s General Tab such that:
    • Restore most recently opened project at startup is not checked.
    • Restore previously open source documents at startup is not checked.
    • Restore .RData into workspace at startup is not checked.
    • Save workspace to .RData on exit is Never.
    • Always save history (even when not saving .RData) is not checked.
  5. Change your RStudio Global Options’s Code Tab such that Under the Saving section:
    • Always save R scripts before sourcing is not checked.
    • Automatically save when editor loses focus is not checked.
    • When editor is idle is Do nothing.

  6. Restart your R session. This can be done in RStudio using the Session > Restart R. After clicking this, if your session still shows objects in the Environment, then click Session > Terminate R > Yes. Terminating the R session effectively does the same thing that restarting the R session should do: detach any packages and remove all objects in the global environment giving you a new session.

  7. After beginning a new session, execute and run all your code to ensure there are no executable errors or warnings. Some warnings are specific to a package which may not cause R executable errors or warnings.

  8. Comment out any erratic code using the hashtag symbol #. Doing so will prevent the autograder from executing it. This is useful if you don’t know how to correct your errors or warnings before the deadline.

  9. Comment out or remove any install.packages() in your code chunks.

  10. Do not work with local files or any files that exist only on your computer. Data objects and data files must be accessed with URLs (provided by the Instructor within the assignment).

University Specifics

Disability Accommodations

To obtain disability-related academic adjustments and/or auxiliary aids, students with disabilities must contact the course instructor and the Disability Resources and Educational Services (DRES) as soon as possible. To contact DRES, student may visit 1207 S. Oak St., Champaign, call 333-4603, e-mail disability@illinois.edu or go to the DRES website at http://disability.illinois.edu/

Academic Integrity and Generative Artificial Intelligence Tools

It is expected that all students abide by the campus regulations on academic integrity http://studentcode.illinois.edu/article1_part4_1-401.html. Intentional violations of academic integrity can be found at http://studentcode.illinois.edu/article1_part4_1-402.html and include, but are not limited to, copying any part of another student’s assignment and allowing another student to copy any part of student’s own assignment.

Generative artificial intelligence tools are relatively new to the general public and can be useful in learning and studying. If students use generative AI tools in this course, do so outside of class as a means of studying and learning accurate information relevant to this course’s content. Students should not use generative artificial intelligence tools as a means to perform (on graded assignments and submissions) in this course. Doing so is an intentional violation of academic integrity.

Safety Protocol

We have been asked by Public Safety https://police.illinois.edu/emergency-preparedness/run-hide-fight/ to share the following information in case of weather or security emergencies. See the links:

Sexual Misconduct Policy and Reporting

The University of Illinois is committed to combating sexual misconduct. Faculty and staff members are required to report any instances of sexual misconduct to the University’s Title IX and Disability Office. In turn, an individual with the Title IX and Disability Office will provide information about rights and options, including accommodations, support services, the campus disciplinary process, and law enforcement options.

A list of the designated University employees who, as counselors, confidential advisors, and medical professionals, do not have this reporting responsibility and can maintain confidentiality, can be found at https://wecare.illinois.edu/resources/students/#confidential. Other information about resources and reporting is available at https://wecare.illinois.edu.

The Last Word

The Instructor reserves the right to make any changes considered to be academically advisable. Any changes will be announced in class and on the Course Website. It is the student’s responsibility to attend the class and keep track of the changes.

Feel free to share!

You may also enjoy...