Introduction: Removing Time and Space as Barriers to the Pursuit of a Terminal Degree

In 2005, then-California State University Chancellor, Charles Reed, and Senator Jack Scott, worked together to propose and pass legislation that allowed CSU campuses to begin offering doctoral degrees in Educational Leadership (Ed.D.). Until about four years ago, these were traditional doctoral programs housed on site at particular CSU campuses.

Together with CSU Fresno, CSU Channel Islands made history when it began to offer the first fully-accredited, online Ed.D. program in the CSU, making it possible for geographically-dispersed, working professionals to earn an Ed.D. from the CSU system. The first cohort of students from the Collaborative Online Doctorate in Educational Leadership (CODEL) will graduate in May 2019!

Online courses are different from face-to-face, “on-ground” courses in interesting and complicated ways, and this paradigm becomes even more complicated at the program level. An example of this shift is the administration of doctoral qualifying examinations, which recently occurred for Cohort 2 on January 12, and were facilitated completely online.

What are Qualifying Exams?

Qualifying Exams (QE) are a type of assessment that doctoral students traditionally take, after their first two years of coursework in fundamental disciplinary theory and methods, in order to advance to doctoral candidacy. A passed QE signifies certain requisite skills and knowledge, and that the now-doctoral candidate is ready to propose and execute a full dissertation study. What the QE looks like, how it is administered, and what kind of experience students have tends to depend on the institution, often the specific campus, and sometimes the department and/or chairperson of a student’s doctoral faculty committee.

The on-ground doctoral program at CSU Fresno consists of an exam day, where students show up to a common location, receive their exams, and then have a certain amount of time to complete their written responses. Because CODEL is an online program, with students dispersed across the state (and, in some cases, the country), the administration faced the challenges of facilitating this process without requiring a common physical location. Together with the Teaching and Learning Innovations team, CODEL designed and executed a workflow leveraging technology to overcome barriers of time and space. The image below lists some of the features from the on-ground QE and how they were adapted for synchronous-but-remote, online implementation.

 

Here is how the current iteration of the CODEL Qualifying Exam works:

Before the Exam

• Students receive instructor-prepared study guides following each of their core doctoral courses.
• 4-6 weeks prior to the QE, eligible students are presented with a common, hypothetical, case study scenario that contextualizes their thinking about the exam questions with which they will be presented on exam day.
• On the first day of January, in the new calendar year, students are invited to join the CODEL Qualifying Exam space in CI Learn (Canvas). This site contains important pre-exam protocol (like signing and submitting certain forms), day-of-exam procedures, and even a sample test for students to familiarize themselves with the format and feel of the exam, which will be administered via Canvas.

Exam Day

• On the day of the exam, all students and proctors log in to an automatically-recorded Zoom meeting with video turned on. Participants are instructed to mute their microphones and to log any questions or activity in the Zoom chat, which is attended by proctor. For example, if a student takes or returns from a personal break, this information is recorded and time-stamped in the chat in order to document during-exam activity. Any technical issues or personal emergencies are also documented in this way in order to ensure that adequate exam time is restored to any individual who encounters an emergency.
• The exam, itself, is administered in Canvas. It is set up to open and close at certain times and to randomly assign three questions from a larger question bank to the students. All three randomly-assigned questions are displayed on a single page, so students may answer the questions in any order.
• Using the Collaborations feature in Canvas, a temporary Google Doc is shared with individual students so they may draft their exam responses before copying and pasting their responses into the Canvas quiz/exam.
  

Exam Scoring & Reporting

• Following the exam, student responses are anonymized and student names are replaced with unique ID numbers. The responses are then pasted into a Google Doc that is housed in a corresponding course folder. These private course folders are shared with three exam readers (a primary, secondary, and optional third reader to reconcile differing scores).
• Using a rubric, the readers score the exam responses assigned to them and then report their scores via an online AirTable form. Like a Google Form, the AirTable form intakes data, like unique Exam ID, Reader Name, Course Exam Question, Score, and Comments, and then organizes that data in a back-end spreadsheet that can be sorted by certain fields. This way, the QE administrator can view student scores, follow up with certain readers, and notify students of their exam status.
• Students are notified via email of their passing or need to retake certain questions following this initial round of score reporting.

Each year, the administration will work to improve the workflow and leverage the resources and technologies available to offer the best possible experience of this already high-stakes (and high stress) exam. The partnership with Teaching and Learning Innovations is integral to that effort, and we look forward to continued collaboration in support of our future Educational Leaders.