The widespread adoption of Learning Analytics (LA) and Educational Data Mining (EDM) has somewhat stagnated recently, and in some prominent cases even been reversed following concerns by governments, stakeholders and civil rights groups. In this ongoing discussion, fears and realities are often indistinguishably mixed up, leading to an atmosphere of uncertainty among potential beneficiaries of Learning Analytics, as well as hesitations among institutional managers who aim to innovate their institution's learning support by implementing data and analytics with a view on improving student success. In this paper, we try to get to the heart of the matter, by analysing the most common views and the propositions made by the LA community to solve them. We conclude the paper with an eight-point checklist named DELICATE that can be applied by researchers, policy makers and institutional managers to facilitate a trusted implementation of Learning Analytics.
Journal writing is an important and common reflective practice in education. Students' reflection journals also offer a rich source of data for formative assessment. However, the analysis of the textual reflections in class of large size presents challenges. Automatic analysis of students' reflective writing holds great promise for providing adaptive real time support for students. This paper proposes a method based on topic modeling techniques for the task of themes exploration and reflection grade prediction. We evaluated this method on a sample of journal writings from pre-service teachers. The topic modeling method was able to discover the important themes and patterns emerged in students' reflection journals. Weekly topic relevance and word count were identified as important indicators of their journal grades. Based on the patterns discovered by topic modeling, prediction models were developed to automate the assessing and grading of reflection journals. The findings indicate the potential of topic modeling in serving as an analytic tool for teachers to explore and assess students' reflective thoughts in written journals.
This paper presents an analytics dashboard that has been developed for designers of interactive e-books. This is part of the EU-funded MC Squared project that is developing a platform for authoring interactive educational e-books. The primary objective is to develop technologies and re- sources that enhance creative thinking for both designers (authors) and learners. The learning material is expected to offer learners opportunities to engage creatively with mathematical problems and develop creative mathematical think- ing. The analytics dashboard is designed to increase authors' awareness so that they can make informed decisions on how to redesign and improve the e-books. This paper presents architectural and design decisions on key features of the dashboard, and discusses the evaluation of a high- fidelity prototype. We discuss our future steps and some findings related to use of the dashboard for exploratory data analysis that we believe generalise to similar work.
Since LAK2015 an increasing number of researchers are taking learning design into consideration when predicting learning behavior and outcomes across different modules. Learning design is widely studied in the Higher Education sector, but few studies have empirically connected learning designs of a substantial number of courses with learning behavior in Learning Management Systems (LMSs) and learning performance. This study builds on preliminary learning design work that was presented at LAK2015 by the Open University UK. In this study we linked 151 modules and 111.256 students with students' behavior (
Differences in learners' behavior have a deep impact on their educational performance. Consequently, there is a need to detect and identify these differences and build suitable learner models accordingly. In this paper, we report on the results from an alternative approach for dynamic student behavioral modeling based on the analysis of time-based student-generated trace data. The goal was to unobtrusively classify students according to their time-spent behavior. We applied 5 different supervised learning classification algorithms on these data, using as target values (class labels) the students' performance score classes during a Computer-Based Assessment (CBA) process, and compared the obtained results. The proposed approach has been explored in a study with 259 undergraduate university participant students. The analysis of the findings revealed that a) the low misclassification rates are indicative of the accuracy of the applied method and b) the ensemble learning (treeBagger) method provides better classification results compared to the others. These preliminary results are encouraging, indicating that a time-spent driven description of the students' behavior could have an added value towards dynamically reshaping the respective models.
One-digit multiplication errors are one of the most extensively analysed mathematical problems. Research work primarily emphasises the use of statistics whereas learning analytics can go one step further and use machine learning techniques to model simple learning misconceptions. Probabilistic programming techniques ease the development of probabilistic graphical models (bayesian networks) and their use for prediction of student behaviour that can ultimately influence decision processes.
We analyse engagement and performance data arising from participants' interactions with an in-house LMS at Imperial College London while a cohort of students follow two courses on a new fully online postgraduate degree in Management. We identify and investigate two main questions relating to the relationships between engagement and performance, drawing recommendations for improved guidelines to inform the design of such courses.
When used effectively, reflective writing tasks can deepen learners' understanding of key concepts, help them critically appraise their developing professional identity, and build qualities for lifelong learning. As such, reflecting writing is attracting substantial interest from universities concerned with experiential learning, reflective practice, and developing a holistic conception of the learner. However, reflective writing is for many students a novel genre to compose in, and tutors may be inexperienced in its assessment. While these conditions set a challenging context for automated solutions, natural language processing may also help address the challenge of providing real time, formative feedback on draft writing. This paper reports progress in designing a writing analytics application, detailing the methodology by which informally expressed rubrics are modelled as formal rhetorical patterns, a capability delivered by a novel web application. This has been through iterative evaluation on an independently human-annotated corpus, showing improvements from the first to second version. We conclude by discussing the reasons why classifying reflective writing has proven complex, and reflect on the design processes enabling work across disciplinary boundaries to develop the prototype to its current state.
In this paper we present a learning analytics conceptual framework that supports enquiry-based evaluation of learning designs. The dimensions of the proposed framework emerged from a review of existing analytics tools, the analysis of interviews with teachers, and user case profiles to understand what types of analytics would be useful in evaluating a learning activity in relation to pedagogical intent. The proposed framework incorporates various types of analytics, with the teacher playing a key role in bringing context to the analysis and making decisions on the feedback provided to students as well as the scaffolding and adaptation of the learning design. The framework consists of five dimensions: temporal analytics, tool-specific analytics, cohort dynamics, comparative analytics and contingency. Specific metrics and visualisations are also defined for each dimension of the conceptual framework. Finally the development of a tool that partially implements the conceptual framework is discussed.
Computer-based writing systems have been developed to provide students with instruction and deliberate practice on their writing. While generally successful in providing accurate scores, a common criticism of these systems is their lack of personalization and adaptive instruction. In particular, these systems tend to place the strongest emphasis on delivering accurate scores, and therefore, tend to overlook additional variables that may ultimately contribute to students' success, such as their affective states during practice. This study takes an initial step toward addressing this gap by building a predictive model of students' affect using information that can potentially be collected by computer systems. We used individual difference measures, text features, and keystroke analyses to predict engagement and boredom in 132 writing sessions. The results from the current study suggest that these three categories of features can be utilized to develop models of students' affective states during writing sessions. Taken together, features related to students' academic abilities, text properties, and keystroke logs were able to more than double the accuracy of a classifier to predict affect. These results suggest that information readily available in compute-based writing systems can inform affect detectors and ultimately improve student models within intelligent tutoring systems.