Tag Archives: ev-evaluation

Evidence is primarily quantitative evaluation and/or feedback data. No control group included.

Type: Evidence | Proposition: B: Teaching | Polarity: | Sector: | Country:

Abstract: Understanding the behavior of learners within learning applications and analyzing the factors that may influence the learning process play a key role in designing and optimizing learning applications. In this work we focus on a specific application named “1x1 trainer” that has been designed for primary school children to learn one digit multiplications. We investigate the database of learners’ answers to the asked questions (N > 440,000) by applying the Markov chains. We want to understand whether the learners’ answers to the already asked questions can affect the way they will answer the subsequent asked questions and if so, to what extent. Through our analysis we first identify the most difficult and easiest multiplications for the target learners by observing the probabilities of the different answer types. Next we try to identify influential structures in the history of learners’ answers considering the Markov chain of different orders. The results are used to identify pupils who have difficulties with multiplications very soon (after couple of steps) and to optimize the way questions are asked for each pupil individually.

Citation: Taraghi, B., Ebner, M., Saranti, A., Schön, M. (2014): LAK '14, March 24 - 28 2014 | Url: http://dx.doi.org/10.1145/2567574.2567614

Type: Evidence | Proposition: B: Teaching | Polarity: | Sector: | Country:

An early intervention solution for collegiate faculty called Course Signals is discussed. Course Signals was developed to allow instructors the opportunity to employ the power of learner analytics to provide real-time feedback to a student. Course Signals relies not only on grades to predict students’ performance, but also demographic characteristics, past academic history, and students’ effort as measured by interaction with Purdue University’s learning management system. The outcome is delivered to the students via a personalized email from the educator to each student, as well as a specific colour on a traffic signal, to indicate how each student is doing. The system is explained in detail, along with retention and performance outcomes realized since its implementation.

The study showed that students who began at Purdue University in autumn 2007, 2008  or 2009 and participated in at least one Course Signals course were retained at rates significantly higher than their peers who had no Course Signals classes but who started at Purdue during the same semester.

The paper also argues that students who had two or more courses with CS were consistently retained at rates higher than those who had only one or no courses with Signal. However, bloggers have since pointed out that the longer students stay at Purdue, the more likely they are to be on two courses using CS, so the figures in this area need to be re-examined in future.

Citation: Arnold, Kimberley E, & Pistilli, Matthew. (2012). Course Signals at Purdue: Using Learning Analytics To Increase Student Success. Paper presented at the LAK12: 2nd International Conference on Learning Analytics and Knowledge (30 April - 2 May), Vancouver, Canada.

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

Abstract: Learners often think math is unrelated to their own interests. Instructional software has the potential to provide personalized instruction that responds to individuals’ interests. Carnegie Learning’s MATHia software for middle school mathematics asks learners to specify domains of their interest (e.g., sports & fitness, arts & music), as well as names of friends/classmates, and uses this information to both choose and personalize word problems for individual learners. Our analysis of MATHia’s relatively coarse-grained personalization contrasts with more fine-grained analysis in previous research on word problems in the Cognitive Tutor (e.g., finding effects on performance in parts of problems that depend on more difficult skills), and we explore associations of aggregate preference “honoring” with learner performance. To do so, we define a notion of “strong” learner interest area preferences and find that honoring such preferences has a small negative association with performance. However, learners that both merely express preferences (either interest area preferences or setting names of friends/classmates), and those that express strong preferences, tend to perform in ways that are associated with better learning compared to learners that do not express such preferences. We consider several explanations of these findings and suggest important topics for future research.

Overview by the Hub: This study investigates whether personalising maths questions using names of learners friends impacts learning (names provided by learner). They found that honouring the preference given by the learner had mixed results. It is speculated that there may be a ‘conciensous’ factor in operation – that those who set preferences are generally more concisous learners. The authors conclude by suggesting that the personalisation may need to be made more clear to learners or done more frequently (so they appreciate it) but also ‘that the use of personalisation features appears to be associated with improved outcomes.’


Citation: Fancsali, S. and Ritter, S. (2014) Context Personalization, Preferences, and Performance in an Intelligent Tutoring System for Middle School Mathematics. The 4th International Conference on Learning Analytics and Knowledge. March 24-28, 2014. Indianapolis, USA. http://lak14indy.wordpress.com/lak-2014-conference-program-schedule/