Penetrating the black box of time-on-task estimation

Type: Evidence | Proposition: C: Uptake | Polarity: | Sector: | Country:

There is a large body of research suggesting that the amount of time spent on learning can improve the quality of learning, as represented by academic performance. The widespread adoption of learning technologies such as learning management systems (LMSs) has resulted in large amounts of data about student learning being readily accessible to educational researchers. One common use of this data is to measure time that students have spent on different learning tasks. Given that LMS systems typically only capture times when students executed various actions, time-on-task measures can currently only be estimates.

This paper takes five learning analytics models of student performance, and examines the consequences of using 15 different time-on-task estimation strategies. It finds that choice of estimation strategy can have a significant effect on the overall fit of a model, its significance, and the interpretation of research findings.

The paper concludes

  1. The learning analytics community should recognize the importance of time-on-task estimation and the role it plays in the quality of analytical models and their interpretation
  2. Publications should explain in detail how time on task has been estimated, in order to support the development of open, replicable and reproducible research
  3. This area should be investigated further in order to provide a set of standards and common practices for the conduct of learning analytics research.

This was selected as the best paper at the Learning Analytics and Knowledge conference 2015.

Citation: Kovanovic, Vitomir, Gasevic, Dragan, Dawson, Shane, Joksimovic, Srecko, Baker, Ryan S, & Hatala, Marek. (2015). Penetrating the black box of time-on-task estimation. Paper presented at LAK '15, Poughkeepsie, NY. DOI 10.1145/2723576.2723623 | Url: