MOOCs as a Massive Laboratory: Opportunities and Challenges

Paul Diver, Statistics
Ignacio Martinez, Economics

How do students actually learn and achieve? Researchers find it difficult to get inside students’ learning processes in order to show how students learn and what most affects their learning. Are actions prior to or during the course of an educational experience most important for learning? Students’ innate abilities? Other factors?

With the advent of Massive Open Online Courses (MOOCs) whose enrolled students can reach into the tens of thousands, researchers have an unprecedented opportunity to analyze student learning more deeply and accurately by studying complex data sets regarding student enrollment, demographics, and achievement, which analysis of these courses is able to provide. This project explores the opportunities and challenges that MOOCs are generating for research.

The researchers involved in this project analyzed two MOOCs, a 6-week course on “Foundations of Business Strategy” and a 14-week course on “The Modern World: Global History since 1760.” They tracked the numbers and percentages of students initially enrolled in the course who took and/or passed each quiz or exam and who received a final statement of accomplishment indicating they had passed all course requirements. Challenges that come with analyzing this data include students’ failure to fill out pre-course demographic surveys, as well as the computational and statistical methods needed to handle large amounts of data (from courses with 84,377 and 46,575 students enrolled, respectively).

Even with these challenges, the project is beginning to produce intriguing results suggesting possibilities for analyzing student learning going forward. These include some interesting correlations that will motivate future research. For instance, students who watch the lectures between their first and last attempts improve more than those who do not. The same is true for reading the forums. Across quizzes and courses there is a consistent negative correlation between procrastination and achievement. Using IP addresses to geolocate students, the team found evidence that students from the United States procrastinate significantly less than students from the rest of the world. The team is in the process of merging the data from these two courses in order to compare and correlate results.