Crowd-sourced learning in MOOCs: learning analytics meets measurement theory

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

Evidence of the Month: April 2015

This paper reports findings from a study of MOOC log file data relating to a large University of Melbourne MOOC that ran in 2013. The study investigated a series of hypotheses: (i) there is skill involved in using forums to learn (ii) MOOC participants use forums differently as they progress from novice to expert in this skill (iii) this progression is reflected in log file data (iv) log file data can be used to measure a learner's skill in learning through forums. The study provides provisional support for each of these hypotheses.

The paper also sets out how skill in the use of forums for learners can develop. The proposed framework identifies how learners at each level are likely to view knowledge, how they are likely to view forums in the context of learning, and how they are likely to use MOOC forums. The framework has five levels:

  • Level 1: Novice, dependent learning
  • Level 2: Beginning, independent learner
  • Level 3, Proficient, collegial learner
  • Level 4, Competent , collaborative learner
  • Level 5 Expert, learning leader.

For example, at Level 1: 'Learning is about consuming stable knowledge in a domain, comprised mainly of cognitive understanding or skill; seeks efficient transfer from authoritative or reputable sources; follows procedural guidance from teachers, relinquishing responsibility for learning process; calibrates performance on formal assessments and accepts standards inherent in them. Forums are essentially social adjuncts to courses; the information is  possibly unreliable and misleading. Never visits forums.'

The relationship between achievement levels on the course studied and level on this scale was apparently strong. Of those rated at expert level 5, 78% received at least a pass score and 67% a distinction score. However, of those rated at level 1, less than 1% received a pass score.

Citation: Milligan, Sandra. (2015). Crowd-sourced learning in MOOCs: learning analytics meets measurement theory. Paper presented at the Learning Analytics and Knowledge (LAK15), Poughkeespie, NY, USA. | Url: