Affect and behaviour detectors were used to estimate student affective states and behaviour based on analysis of tutor log data. For every student action, the detectors estimated the probability that the student was in a state of boredom, engaged concentration, confusion, or frustration. They also estimated the probability that the student was engaging in off‐task or gaming behaviours.
Boredom during problem solving was negatively correlated with performance
Boredom during scaffolded tutoring was positively correlated with performance
Confusion during problem solving was negatively correlated with performance
Confusion during scaffolded tutoring was positively correlated with performance
Engaged concentration was associated with positive learning outcomes
Frustration was associated with positive learning outcomes
Gaming the system was associated with poorer learning,
A unified model was used to predict student standardized examination scores from a combination of student affect, disengaged behaviour, and performance within the learning system. This model achieved high overall correlation with standardized exam scores.
Building on this work could produce positive evidence for ‘Proposition A: learning analytics improve learning outcomes’ and ‘Proposition B: Learning analytics improve learning support and teaching’.
This paper investigates the correspondence between student affect in a web-based tutoring platform throughout the school year and learning outcomes at the end of the year, on a high-stakes mathematics exam. Affect detectors were used to estimate student affective states based on post-hoc analysis of tutor log-data. For every student action in the tutor, the detectors provided an estimated probability that the student was in a state of boredom, engaged concentration, confusion, and frustration, and estimated the probability that they were exhibiting off-task or gaming behaviours. Boredom during problem solving was found to be negatively correlated with performance, however, boredom was positively correlated with performance when exhibited during scaffolded tutoring. A similar pattern was seen for confusion. Engaged concentration and, surprisingly, frustration were both associated with positive learning outcomes.
Comment from Martyn Cooper: This paper reports an investigation of the relationship between affective states of students as noted in online tutor logs and the performance in summative high stakes exams. An interesting result was that students who were bored or confused while answering the main problems, tended to do poorly on the test; however, boredom and confusion on scaffolding were associated with positive performance on the test. It is a good example of using analytics to assess students’ attitudes while learning. However, it is just a study with a single cohort over one year in a single limited learning environment used in maths education. The intention is to work towards better support for tutors to intervene appropriately given the students’ affective states and their contexts.
Citation: Z.A. Pardos, R. S. J. D. Baker, M. O. C. Z. San Pedro, S. M. Gowda, S. M. Gowda, (2013). Affective states and state tests: investigating how affect throughout the school year predicts end of year learning outcomes, In: Third International Learning Analytics and Knowledge Conference (LAK13), 8-12 April, Leuven, Belgium. | Url: http://dl.acm.org/citation.cfm?id=2460296
Abstract: Predicting student’s performance is a challenging, yet complicated task for institutions, instructors and learners. Accurate predictions of performance could lead to improved learning outcomes and increased goal achievement. For that reason, prediction of performance is acknowledged as one of the major objectives of Learning Analytics and Educational Data Mining research. In this paper we explore the predictive capabilities of student’s time-spent on answering (in-)correctly each question of a multiple-choice assessment quiz, along with student’s final quiz-score, in the context of computer-based testing. We also explore the correlation between the time-spent factor (as defined here) and goal-expectancy. We present a case study and investigate the value of using this parameter as a learning analytics factor for improving prediction of performance during computer-based testing. Our initial results are encouraging and indicate that the temporal dimension of learning analytics should be further explored.
Overview from the Hub: The preliminary results from this research indicate that temporal learning analytics – that is to say the cumulative duration of time that a student takes to make correct or incorrect answers on a multiple choice quiz - have a statistically significant capability of predicting actual performance (almost 62% of the variance). In their single case study of 96 high school students the report finds that if a learner spends more time to answer correctly then they are more likely to score higher (positive effect of 0.60) and if they spend more time answering incorrectly then it appears they are likely to score lower (0.29). They also report a third finding: that there is a positive effect of (learner pre-stated) goal expectancy on correct answers and a negative effect on the total time spent making incorrect answers. The proposed use of this technique once researched further is as a potential replacement for traditional grading activities where determining grades act as effort overload.
Citation: Papamitsiou, Z.K., Terzis, V. and Economides, A.A. (2014) Temporal Learning Analytics for Computer Based Testing. The 4th International Conference on Learning Analytics and Knowledge. March 24-28, 2014. Indianapolis, USA. http://lak14indy.wordpress.com/lak-2014-conference-program-schedule/