Tag Archives: Learning analytics

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

This article concerns the evaluation of the social capital of the European teachers participating in the eTwinning Portal activities, run by European Schoolnet and characterised by more than 160,000 registered teachers from 35 countries, involved in more than 19,000 projects (2010). This evaluation has been performed by using the Social Network Analysis approach.

The authors found that some correlations can be found "between social network analysis measures like degree and betweenness centrality as well as the local clustering coefficient, activity statistics about usage of eTwinning and the quality management of European Schoolnet".

For the analysis of eTwinning network data, three Learning Analytics tools have been developed:

  • eVa (eTwinning Network Visualization and Analysis), a network visualization and simple analysis tool.
  • CAfe (Competence Analyst for eTwinning), an SNA-based competence management and teachers' self-monitoring tool.
  • AHTC (Ad Hoc Transient Communities) services, which involve users into question-answer activities on the eTwinning Portal.
Citation: M.C. Pham, Y. Cao, Z. Petrushyna, R. Klamma. "Learning Analytics in a Teachers' Social Network" (2012). Proceedings of the 8th International Conference on Networked Learning 2012, ISBN 978-1-86220-283-2 | Url: http://www.lancaster.ac.uk/fss/organisations/netlc/past/nlc2012/abstracts/pdf/pham.pdf

Type: Evidence | Proposition: A: Learning | Polarity: | Sector:

In this paper, the authors describe very well the issues and the opportunities of management of informal learning at workplace based on technology. Contents of this paper are derived by the experience of TRAILER project.

Authors identify informal learning as the most prominent way of learning at workplace (more than 70% of the total learning), and this is why scientific community and policy makers are trying to explore new approaches and technologies to formalize it.

However, "tracking" informal learning with technology raises some issues:

  • Systems can be rejected by users.
  • Difficulty in promoting the adoption of the approach in day-to-day professional activities.
  • The use of a catalogue of competences for informal learning classification can be seen as a major barrier.

These issues can be partially mitigated by using a more pragmatic approach, in particular by integrating systems with the applications which already formed part of the users working environment.

The authors discuss about the definitions of formal, non-formal and informal learning, suggesting that the distinction among them is more related to the presence or absence of a management process.

The introduction of this management process allows to reach two conditions:

  • The validation of informal learning ensures that individuals are not only assessed on their formal qualifications, but also given credit for the whole range of learning which they have achieved in their lives.
  • Whatever the intentions of their designers may be, competence based systems and systems for the validation of informal learning both inevitably extend the reach of educational management into areas to which it did not previously have access.

Hwever, according to authors and TRAILER project experience, "a high priority in supporting informal learning should be the avoidance of additional management processes".

In this scenario, Data and Learning Analytics could represent a solution, considering that "if it were possible to simply monitor people’s activities and deduce their capabilities from this, then the need to validate items of informal learning might disappear. There are increasing signs that this may be possible".

In the second part, authors describe the structure and functioning of a prototype of a Learning Analytics-based system for informal learning tracking.

Citation: F.J. García-Peñalvo, D. Griffiths, M. Johnson, P. Sharples, D. Sherlock. "Problems and opportunities in the use of technology to manage informal learning". 2014, Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM '14), pp. 573-580. DOI=10.1145/2669711.2669958 http://doi.acm.org/10.1145/2669711.2669958 | Url: http://dl.acm.org/citation.cfm?id=2669958

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

This paper talks about the introduction of a tool, named Network Awareness Tool, for the investigation of informal learning at workplace. The importance of tools like this is high, considering that "Informal learning is an important driver for professional development and workplace learning. However [...], there is a problem when it comes to making it a real asset within organizations: Informal learning activities are mostly invisible to others, sometimes the learners themselves might not even be aware of the learning that occurs. As a consequence informal learning in organizations goes undetected, remains off the radar of HR departments and is therefore hard to asses, manage and value".

In order to test the tool, a taget group has been selected, composed by teaching professionals working in school organizations. The tool is based on different theories:

  • Networked Learning Theory: "Networked Learning Theory is an emerging perspective that tries to understand learning by asking the question how people develop and maintain a ‘web’ of social relations used for their own and reciprocal learning and professional development".
  • Social Network Theory: "Social Network Theory asserts that the constitution of a network may influence the accessibility of information and resources and that the social structure may offer potential for the exchange of resources". The structure and the dimensions of a social network can be analysed by Social Network Analysis.
  • Social Network Analysis: "According Social Network Analysis a network consists of nodes and ties. Nodes are the individual actors within a network and ties are the relationships between the actors. The impact of the structure of social networks can be studied on three levels: first the positions people have in a network (individual dimension), the relational level (ties dimension) and finally the overall network structure (network
  • Social Capital Theory: this theory concerns "the relational resources embedded in social ties and how actors interact to gain access to these resources".
  • Communities of Practice: "the collective advancement of knowledge and the development of shared identities comes together in the community aspect of social learning, which we base on the well known concept of communities of practice".
  • Individual demographics: this is an important aspect to be taken into accounts, considering that "age and years of experience can also have an impact on teachers’ professional development. Senior employees tend to take less initiatives in their professional development".

Thanks to Learning Analytics functionalities, Network Awareness Tool can depict the "actors" involved in the social network and to track the quality and the nature of ties between actors. Also the contents of ties can be tracked using meta-tags. Using Social Network Analysis, Network Awareness Tool can design the social network structure and density, indicating to HR department what topic are particularly relevant for informal learning at workplace.

Citation: B. Schreurs, M. De Laat. 2012. "Network awareness tool - learning analytics in the workplace: detecting and analyzing informal workplace learning". In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK '12). ACM, New York, NY, USA, 59-64. DOI=http://dx.doi.org/10.1145/2330601.2330620 | Url: http://dl.acm.org/citation.cfm?id=2330620

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

In this paper, the authors describe a conceptual model for the analysis of informal learning in online social networks for workers, and in particular for health professionals. This work sector has been selected due to its particular characteristics, considering that Staying up-to-date and delivering best evidence-based care is crucial for these professionals, and that they need to be lifelong learners as medical knowledge expands and changes rapidly.

In this environment, "Online social networking (OSN) provides a new way for health professionals to communicate, collaborate and share ideas with each other for informal learning on a massive scale. It has important implications for ongoing efforts to support Continuing Professional Development (CPD) in the health professions. However, the challenge of analysing the data generated in OSNs makes it difficult to understand whether and how they are useful for CPD".

The paper explores three approaches for the analysis of OSN: Content Analysis (CA), Social Network Analysis (SNA) and the most innovative one, the Social Learning Analytics (SLA), as a sub-field of Learning Analytics.

The described conceptual model tries to merge the CA and SNA approaches, considering also a survey to evaluate the learning outcome, instead of real clinical data. This model is divided in three sections:

  • Learning interactions: is focused on studying the structure of interactions and the level of and influential factors associated with engagement.
  • Learning process: involves the examination of cognitive presence, social presence, facilitation presence and learning presence.
  • Learning outcome: measures the  social value for online community members, in terms of valued activities, gained knowledge, changed practice, improved performance and redefined success.
Citation: Li, X; Gray, K; Chang, S; Elliott, K; Barnett, S, A conceptual model for analysing informal learning in online social networks for health professionals., Stud Health Technol Inform, 2014, 204 pp. 80 - 85 | Url: https://minerva-access.unimelb.edu.au/handle/11343/43110

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

This paper describes the analytical section of Lear-B (Learning Biosis), a first research prototype to address some of the challenges inherent to workplace learning, such as its informal aspect, the need to support workers in their self-regulatory learning (SRL) processes, the importance of collective and social sharing of the workplace knowledge and its "on-demand" and contextual characteristics.

The authors stress in particular the role of the workplace community in the development of knowledge and the necessity of each individual to develop a correct and useful Self-Regulatory Learning (SRL) approach. These aspects highlights some of the characteristics that a support IT tool should provide, such as:

  • Collection of learning–related contributions and their re-aggregation, analysis to create further new knowledge, as well as sharing results to users.
  • Identification of learning needs for individuals and self-setting of personal learning goals.
  • Monitoring and comparison of users learning progress.
  • Share of users learning experiences.
  • Necessity to integrate data from different tools and services that are used by workers in their everyday working.

In this prototype, Learning Analytics plays an important role: according to the authors, Learning Analytics "allows for the organization to better align its learning objectives with those of its employees by knowing about their learning practic-es; it supports users’ SRL processes by providing them with the necessary input from the social context of the workplace; and it enhances the motivation of individuals to take part in learning and knowledge building activities and sharing their experiences by providing them with feedback from the collective."

The paper describes then the internal environment of Lear-B (which integrates also wiki, social networking and bookmarking functionalities), indicating how the analysis of the platform is albe to enhance knowledge tracking, workers engagement and individual and collective learning improvements.

Citation: M. Siadaty, D. Gašević, J. Jovanović, N. Milikić, Z. Jeremić, L. Ali, A. Giljanović, M. Hatala. Learn-B: A Social Analytics-enabled Tool for Self-regulated Workplace Learning | Url: http://dl.acm.org/citation.cfm?id=2330632&dl=ACM&coll=DL&CFID=768069737&CFTOKEN=86753107

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

Learning analytics makes extensive use of trace data from learners interacting with Learning Management Systems (LMS), and one of the most common uses is to derive an estimate of the time each learner spent on task, that is, engaging in particular learning activities. The authors of this paper note that while this approach is widely used, the details are often not given, and the consequences of choices made in generating these measures are not explored.

They present two experiments exploring different measures of time on task, one using data from a fully-online course at a Canadian university, and another using data from a blended course at an Australian university.

They found that time-on-task measures typically outperformed count measures for predicting student performance, but more importantly, identified that the precise choice of time-on-task measure "can have a significant effect on the overall fit of the model, its significance, and eventually on the interpretation of research findings".

Given their findings, they argue that there are implications for research in learning analytics: "Above all is the need for more caution when using time-on-task measures for building learning analytics models. Given that details of time-on-task estimation can potentially impact reported research findings, appropriately addressing time-on-task estimation becomes a critical part of standard research practice in the learning analytics community. This is particularly true in cases where time-on-task measures are not accompanied by additional measures such as counts of relevant activities."

They call for researchers to recognise the importance of time-on-task measures, for fuller detail to be given of the measures used, and for further investigation.

(This paper is an expanded version of an earlier paper at LAK15, which reported only on the data from the Canadian university: Kovanovic, Vitomir, Gasevic, Dragan, Dawson, Shane, Joksimovic, Srecko, Baker, Ryan S, & Hatala, Marek. (2015). Penetrating the black box of time-on-task estimation. Paper presented at LAK '15, Poughkeepsie, NY. DOI 10.1145/2723576.2723623.)

Citation: Kovanović, V., Gašević, D., Dawson, S., Joksimović, S., Baker, R. S., & Hatala, M. (2015) Does Time-on-task Estimation Matter? Implications on Validity of Learning Analytics Findings. | Url: https://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4501

Type: Evidence | Proposition: A: Learning | Polarity: | Sector:

This paper is a preliminary study on how learning analytics can support tracking and measurement of informal learing, which represents a large part learning at the workplace.

The paper is in tight relationship with a previous evidence identified by this Hub (please , see http://evidence.laceproject.eu/?evidence=new-framework-informal-learning-workplace) about the development of Social Semantic Server (SSS). In this paper it is highlighted how SSS can generate semantically-enriched Actor-Artifact Networks (AANs) to describe the relationships among actors and artifacts in different learning contexts.

To simply  determine how an AAN can be generated in SSS, a preliminary study was developed using SSS and two of its applications, Bookmarker (a Chrome extension that allows users to submit bookmarks and tags to the SSS while browsing) and Attacher (a WordPress plugin that integrates a blog editor to the SSS and displays a tagcloud that includes tags registered in the SSS).  In this study, ten students of a training course for future teachers were asked to bookmark web resources they considered relevant (using Bookmarker) and to write "a blog post about their reflections of the subject using WordPress and Attacher to browse the bookmarks published in the social semantic infrastructure".

Results show that three AANs can be easily identified, helping the teacher to analyze the learning of the students and to understand their behavior. This study describes a first example on how the SSS can integrate the data from different applications and coherently combine it to support learning analytics at the workplace.

Citation: A. Ruiz-Calleja , S. Dennerlein, V. Tomberg, K. Pata, T. Ley, D. Theiler, E. Lex. Supporting Learning Analytics for Informal Workplace Learning with a Social Semantic Infrastructure. (2015)10th European Conference on Technology Enhanced Learning, EC-TEL 2015, Toledo, Spain, September 15-18, 2015, Proceedings, pp 634-637 | Url: http://link.springer.com/chapter/10.1007%2F978-3-319-24258-3_76

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

This literature paper briefly anticipates and outlines a project about integrating labour market information in a learning analytics goal-setting application, with the aim to help student to develop skills that are currently requested by the labour market.

Authors highlight the absurd contemporary trend that characterise labour market, with high levels of youth unemployment together with difficulties of companies to find candidates with the right job skills.

In the developed IT solution, abour market data will be analyzed to extract information that may impact student planning. This will lead to a "goal-setting program in which students can specify goals (e.g., learning goals as subgoals of more distant career goals)" and have access to "relevant labour market information, view progress and success indicators, and receive recommendations from course advisors".

According to authors, "this research expands the impact of learning analytics beyond the educational setting by helping students to navigate the education-to-employment pathway using a goal-setting application" characterized by analytical methods joint with an examination of learning constructs and educational theories.

Citation: V. Kobayashi, S.T. Mol, G. Kismihók (2014). Labour Market Driven Learning Analytics. Journal of Learning Analytics, 1(3), 207-210 | Url: https://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4194

Type: Evidence | Proposition: A: Learning | Polarity: | Sector:

This paper focuses on the necessity of theories and empirical data for the development of Learning Analytics in all the educational fields.

In a dedicated section, authors describe the issues that Learning Analytics are facing when it is applied to workplace learning, stating that "the current focus has been on formal education settings".

One of the main issues highlighted by authors is that "workplace learning is contextually embedded into the workplace tasks and most commonly occurs through informal opportunities", which are hardly catched and monitored by Learning Analytics.

Authors stated also two important differencec between workplace learning and formal learning:

  1. Upskilling and lifelong learning in the workforce are not necessarily associated to formal qualifications.
  2. Just-in-time learning that occurs during daily work is considerabily different from the formal learning sessions that characterized formal learning.
Citation: S. Dawson, N. Mirriahi, D. Gašević. (2015). Importance of theory in learning analytics in formal and workplace settings. Journal of Learning Analytics, 2(2), 1–4. http://dx.doi.org/10.18608/jla.2015.22.1 | Url: https://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4733

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

In recent years, one of the main issues for companies in hiring new employees consists in the bridge between skills provided by HE and the ones required by a more and more fast and agile market. This is why it is particular important for students to identify as soon as possible what can be their career fields, taking into consideration natural attitudes and personal wishes.

In this scientific paper, authors describe the development of a tool, or a Portal, to improve career readiness of students during Higher Education. IT tools like this are particularly interesting, because gives to companies the potential capability to have visibility on the areas of interests of the best graduates, and contacting them soon as they finish their studies, as well as compare these attitudes with workplace results and skill improvements.

This Portal consists of three major processes:

1) Career Readiness

2) Career Prediction

3) Career Development.

Career Readiness is formed  by different software modules which allows to measure the general professional dispositions required for a successful career in the 21st Century (21st Century Skills). These professional dispositions have been divided in 6 macro-areas, which in general describe  "the natural tendencies, mind state and preparations of each individual towards a professional practice": Openness to challenge, Critical Thinking, Resilience, Learning Relationships, Responsibility for Learning, and Creativity.

Career Prediction is calculated on some indicators, which are transformed in "Raw scores", that allows to cluster individuals and compare them with the features of different Community of Practice (people who share knowledge, experiences and passion on the same set of topics).

Career Development, which allows people to join different Communities of Practice and build their career domain awareness and skills.

The  tool integrates Learning Analytics techniques for career readiness "by focusing on meta-learning dimensions that accompany formal education while positioning learners and instructors at the center of the analytics process". Learning Analytics allows to provide solutions to help learners to reflect and act upon feedback about their learning performance.  Furthermore, Learning Analytics reveals hidden patterns of common traits among learners viewed as future candidates of the job market.

Citation: AbuKhousa, E.; Atif, Y., "Big learning data analytics support for engineering career readiness," in Interactive Collaborative Learning (ICL), 2014 International Conference on , vol., no., pp.663-668, 3-6 Dec. 2014 doi: 10.1109/ICL.2014.7017849 | Url: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=7017849&punumber%3D7002490%26sortType%3Dasc_p_Sequence%26filter%3DAND(p_IS_Number%3A7017737)%26pageNumber%3D4