Archives

Type: Evidence | Proposition: B: Teaching | Polarity: | Sector: | Country:

In recent years, Massive Open Online Courses (MOOCs) have become a phenomenon offering the possibility to teach thousands of participants simultaneously. In the same time the platforms used to deliver these courses are still in their fledgling stages. While course content and didactics of those massive courses are the primary key factors for the success of courses, still an smart platform may increase or decrease the learners experience and his learning outcome. This paper at hand proposes the usage of an A/B testing framework that is able to be used within an microservice architecture to validate hypotheses about how learners use the platform and to enable data-driven decisions about new features and settings. To evaluate this framework three new features (Onboarding Tour, Reminder Mails and a Pinboard Digest) have been identified based on a user survey. They have been implemented and introduced on two large MOOC platforms and their influence on the learners behavior have been measured. Finally this paper proposes a data driven decision workflow for the introduction of new features and settings on e-learning platforms.

Citation: Jan Renz, Daniel Hoffmann, Thomas Staubitz and Christoph Meinel (2016). "Using A/B Testing in MOOC Environments". In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK '16). ACM, New York.

Type: Evidence | Proposition: A: Learning | Polarity: | Sector:

In this paper, the authors describe very well the issues and the opportunities of management of informal learning at workplace based on technology. Contents of this paper are derived by the experience of TRAILER project.

Authors identify informal learning as the most prominent way of learning at workplace (more than 70% of the total learning), and this is why scientific community and policy makers are trying to explore new approaches and technologies to formalize it.

However, "tracking" informal learning with technology raises some issues:

  • Systems can be rejected by users.
  • Difficulty in promoting the adoption of the approach in day-to-day professional activities.
  • The use of a catalogue of competences for informal learning classification can be seen as a major barrier.

These issues can be partially mitigated by using a more pragmatic approach, in particular by integrating systems with the applications which already formed part of the users working environment.

The authors discuss about the definitions of formal, non-formal and informal learning, suggesting that the distinction among them is more related to the presence or absence of a management process.

The introduction of this management process allows to reach two conditions:

  • The validation of informal learning ensures that individuals are not only assessed on their formal qualifications, but also given credit for the whole range of learning which they have achieved in their lives.
  • Whatever the intentions of their designers may be, competence based systems and systems for the validation of informal learning both inevitably extend the reach of educational management into areas to which it did not previously have access.

Hwever, according to authors and TRAILER project experience, "a high priority in supporting informal learning should be the avoidance of additional management processes".

In this scenario, Data and Learning Analytics could represent a solution, considering that "if it were possible to simply monitor people’s activities and deduce their capabilities from this, then the need to validate items of informal learning might disappear. There are increasing signs that this may be possible".

In the second part, authors describe the structure and functioning of a prototype of a Learning Analytics-based system for informal learning tracking.

Citation: F.J. García-Peñalvo, D. Griffiths, M. Johnson, P. Sharples, D. Sherlock. "Problems and opportunities in the use of technology to manage informal learning". 2014, Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM '14), pp. 573-580. DOI=10.1145/2669711.2669958 http://doi.acm.org/10.1145/2669711.2669958 | Url: http://dl.acm.org/citation.cfm?id=2669958

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

This paper talks about the introduction of a tool, named Network Awareness Tool, for the investigation of informal learning at workplace. The importance of tools like this is high, considering that "Informal learning is an important driver for professional development and workplace learning. However [...], there is a problem when it comes to making it a real asset within organizations: Informal learning activities are mostly invisible to others, sometimes the learners themselves might not even be aware of the learning that occurs. As a consequence informal learning in organizations goes undetected, remains off the radar of HR departments and is therefore hard to asses, manage and value".

In order to test the tool, a taget group has been selected, composed by teaching professionals working in school organizations. The tool is based on different theories:

  • Networked Learning Theory: "Networked Learning Theory is an emerging perspective that tries to understand learning by asking the question how people develop and maintain a ‘web’ of social relations used for their own and reciprocal learning and professional development".
  • Social Network Theory: "Social Network Theory asserts that the constitution of a network may influence the accessibility of information and resources and that the social structure may offer potential for the exchange of resources". The structure and the dimensions of a social network can be analysed by Social Network Analysis.
  • Social Network Analysis: "According Social Network Analysis a network consists of nodes and ties. Nodes are the individual actors within a network and ties are the relationships between the actors. The impact of the structure of social networks can be studied on three levels: first the positions people have in a network (individual dimension), the relational level (ties dimension) and finally the overall network structure (network
    dimension)".
  • Social Capital Theory: this theory concerns "the relational resources embedded in social ties and how actors interact to gain access to these resources".
  • Communities of Practice: "the collective advancement of knowledge and the development of shared identities comes together in the community aspect of social learning, which we base on the well known concept of communities of practice".
  • Individual demographics: this is an important aspect to be taken into accounts, considering that "age and years of experience can also have an impact on teachers’ professional development. Senior employees tend to take less initiatives in their professional development".

Thanks to Learning Analytics functionalities, Network Awareness Tool can depict the "actors" involved in the social network and to track the quality and the nature of ties between actors. Also the contents of ties can be tracked using meta-tags. Using Social Network Analysis, Network Awareness Tool can design the social network structure and density, indicating to HR department what topic are particularly relevant for informal learning at workplace.

Citation: B. Schreurs, M. De Laat. 2012. "Network awareness tool - learning analytics in the workplace: detecting and analyzing informal workplace learning". In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK '12). ACM, New York, NY, USA, 59-64. DOI=http://dx.doi.org/10.1145/2330601.2330620 | Url: http://dl.acm.org/citation.cfm?id=2330620

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

In this paper, the authors describe a conceptual model for the analysis of informal learning in online social networks for workers, and in particular for health professionals. This work sector has been selected due to its particular characteristics, considering that Staying up-to-date and delivering best evidence-based care is crucial for these professionals, and that they need to be lifelong learners as medical knowledge expands and changes rapidly.

In this environment, "Online social networking (OSN) provides a new way for health professionals to communicate, collaborate and share ideas with each other for informal learning on a massive scale. It has important implications for ongoing efforts to support Continuing Professional Development (CPD) in the health professions. However, the challenge of analysing the data generated in OSNs makes it difficult to understand whether and how they are useful for CPD".

The paper explores three approaches for the analysis of OSN: Content Analysis (CA), Social Network Analysis (SNA) and the most innovative one, the Social Learning Analytics (SLA), as a sub-field of Learning Analytics.

The described conceptual model tries to merge the CA and SNA approaches, considering also a survey to evaluate the learning outcome, instead of real clinical data. This model is divided in three sections:

  • Learning interactions: is focused on studying the structure of interactions and the level of and influential factors associated with engagement.
  • Learning process: involves the examination of cognitive presence, social presence, facilitation presence and learning presence.
  • Learning outcome: measures the  social value for online community members, in terms of valued activities, gained knowledge, changed practice, improved performance and redefined success.
Citation: Li, X; Gray, K; Chang, S; Elliott, K; Barnett, S, A conceptual model for analysing informal learning in online social networks for health professionals., Stud Health Technol Inform, 2014, 204 pp. 80 - 85 | Url: https://minerva-access.unimelb.edu.au/handle/11343/43110

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

This paper describes the analytical section of Lear-B (Learning Biosis), a first research prototype to address some of the challenges inherent to workplace learning, such as its informal aspect, the need to support workers in their self-regulatory learning (SRL) processes, the importance of collective and social sharing of the workplace knowledge and its "on-demand" and contextual characteristics.

The authors stress in particular the role of the workplace community in the development of knowledge and the necessity of each individual to develop a correct and useful Self-Regulatory Learning (SRL) approach. These aspects highlights some of the characteristics that a support IT tool should provide, such as:

  • Collection of learning–related contributions and their re-aggregation, analysis to create further new knowledge, as well as sharing results to users.
  • Identification of learning needs for individuals and self-setting of personal learning goals.
  • Monitoring and comparison of users learning progress.
  • Share of users learning experiences.
  • Necessity to integrate data from different tools and services that are used by workers in their everyday working.

In this prototype, Learning Analytics plays an important role: according to the authors, Learning Analytics "allows for the organization to better align its learning objectives with those of its employees by knowing about their learning practic-es; it supports users’ SRL processes by providing them with the necessary input from the social context of the workplace; and it enhances the motivation of individuals to take part in learning and knowledge building activities and sharing their experiences by providing them with feedback from the collective."

The paper describes then the internal environment of Lear-B (which integrates also wiki, social networking and bookmarking functionalities), indicating how the analysis of the platform is albe to enhance knowledge tracking, workers engagement and individual and collective learning improvements.

Citation: M. Siadaty, D. Gašević, J. Jovanović, N. Milikić, Z. Jeremić, L. Ali, A. Giljanović, M. Hatala. Learn-B: A Social Analytics-enabled Tool for Self-regulated Workplace Learning | Url: http://dl.acm.org/citation.cfm?id=2330632&dl=ACM&coll=DL&CFID=768069737&CFTOKEN=86753107

Type: Evidence | Proposition: A: Learning | Polarity: | Sector:

This paper is a preliminary study on how learning analytics can support tracking and measurement of informal learing, which represents a large part learning at the workplace.

The paper is in tight relationship with a previous evidence identified by this Hub (please , see http://evidence.laceproject.eu/?evidence=new-framework-informal-learning-workplace) about the development of Social Semantic Server (SSS). In this paper it is highlighted how SSS can generate semantically-enriched Actor-Artifact Networks (AANs) to describe the relationships among actors and artifacts in different learning contexts.

To simply  determine how an AAN can be generated in SSS, a preliminary study was developed using SSS and two of its applications, Bookmarker (a Chrome extension that allows users to submit bookmarks and tags to the SSS while browsing) and Attacher (a WordPress plugin that integrates a blog editor to the SSS and displays a tagcloud that includes tags registered in the SSS).  In this study, ten students of a training course for future teachers were asked to bookmark web resources they considered relevant (using Bookmarker) and to write "a blog post about their reflections of the subject using WordPress and Attacher to browse the bookmarks published in the social semantic infrastructure".

Results show that three AANs can be easily identified, helping the teacher to analyze the learning of the students and to understand their behavior. This study describes a first example on how the SSS can integrate the data from different applications and coherently combine it to support learning analytics at the workplace.

Citation: A. Ruiz-Calleja , S. Dennerlein, V. Tomberg, K. Pata, T. Ley, D. Theiler, E. Lex. Supporting Learning Analytics for Informal Workplace Learning with a Social Semantic Infrastructure. (2015)10th European Conference on Technology Enhanced Learning, EC-TEL 2015, Toledo, Spain, September 15-18, 2015, Proceedings, pp 634-637 | Url: http://link.springer.com/chapter/10.1007%2F978-3-319-24258-3_76

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

This literature paper briefly anticipates and outlines a project about integrating labour market information in a learning analytics goal-setting application, with the aim to help student to develop skills that are currently requested by the labour market.

Authors highlight the absurd contemporary trend that characterise labour market, with high levels of youth unemployment together with difficulties of companies to find candidates with the right job skills.

In the developed IT solution, abour market data will be analyzed to extract information that may impact student planning. This will lead to a "goal-setting program in which students can specify goals (e.g., learning goals as subgoals of more distant career goals)" and have access to "relevant labour market information, view progress and success indicators, and receive recommendations from course advisors".

According to authors, "this research expands the impact of learning analytics beyond the educational setting by helping students to navigate the education-to-employment pathway using a goal-setting application" characterized by analytical methods joint with an examination of learning constructs and educational theories.

Citation: V. Kobayashi, S.T. Mol, G. Kismihók (2014). Labour Market Driven Learning Analytics. Journal of Learning Analytics, 1(3), 207-210 | Url: https://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4194

Type: Evidence | Proposition: A: Learning | Polarity: | Sector:

This paper focuses on the necessity of theories and empirical data for the development of Learning Analytics in all the educational fields.

In a dedicated section, authors describe the issues that Learning Analytics are facing when it is applied to workplace learning, stating that "the current focus has been on formal education settings".

One of the main issues highlighted by authors is that "workplace learning is contextually embedded into the workplace tasks and most commonly occurs through informal opportunities", which are hardly catched and monitored by Learning Analytics.

Authors stated also two important differencec between workplace learning and formal learning:

  1. Upskilling and lifelong learning in the workforce are not necessarily associated to formal qualifications.
  2. Just-in-time learning that occurs during daily work is considerabily different from the formal learning sessions that characterized formal learning.
Citation: S. Dawson, N. Mirriahi, D. Gašević. (2015). Importance of theory in learning analytics in formal and workplace settings. Journal of Learning Analytics, 2(2), 1–4. http://dx.doi.org/10.18608/jla.2015.22.1 | Url: https://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4733

Type: Evidence | Proposition: A: Learning | Polarity: | Sector: | Country:

In recent years, one of the main issues for companies in hiring new employees consists in the bridge between skills provided by HE and the ones required by a more and more fast and agile market. This is why it is particular important for students to identify as soon as possible what can be their career fields, taking into consideration natural attitudes and personal wishes.

In this scientific paper, authors describe the development of a tool, or a Portal, to improve career readiness of students during Higher Education. IT tools like this are particularly interesting, because gives to companies the potential capability to have visibility on the areas of interests of the best graduates, and contacting them soon as they finish their studies, as well as compare these attitudes with workplace results and skill improvements.

This Portal consists of three major processes:

1) Career Readiness

2) Career Prediction

3) Career Development.

Career Readiness is formed  by different software modules which allows to measure the general professional dispositions required for a successful career in the 21st Century (21st Century Skills). These professional dispositions have been divided in 6 macro-areas, which in general describe  "the natural tendencies, mind state and preparations of each individual towards a professional practice": Openness to challenge, Critical Thinking, Resilience, Learning Relationships, Responsibility for Learning, and Creativity.

Career Prediction is calculated on some indicators, which are transformed in "Raw scores", that allows to cluster individuals and compare them with the features of different Community of Practice (people who share knowledge, experiences and passion on the same set of topics).

Career Development, which allows people to join different Communities of Practice and build their career domain awareness and skills.

The  tool integrates Learning Analytics techniques for career readiness "by focusing on meta-learning dimensions that accompany formal education while positioning learners and instructors at the center of the analytics process". Learning Analytics allows to provide solutions to help learners to reflect and act upon feedback about their learning performance.  Furthermore, Learning Analytics reveals hidden patterns of common traits among learners viewed as future candidates of the job market.

Citation: AbuKhousa, E.; Atif, Y., "Big learning data analytics support for engineering career readiness," in Interactive Collaborative Learning (ICL), 2014 International Conference on , vol., no., pp.663-668, 3-6 Dec. 2014 doi: 10.1109/ICL.2014.7017849 | Url: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=7017849&punumber%3D7002490%26sortType%3Dasc_p_Sequence%26filter%3DAND(p_IS_Number%3A7017737)%26pageNumber%3D4

Type: Evidence | Proposition: B: Teaching | Polarity: | Sector: | Country:

The paper describes a procedure evaluation/e-training tool (named PeT) for the oil and gas industry, related to the tracking of knowledge and confidence of trainees in emergency operating procedures.

The issue that this kind of tools try to solve is the fact that one of the main responsible for incidents in the oil & gas industry is the lacking of knowledge, both during standard and emergency situations. This lack of training can have serious consequences over the safety of workforce, such as losses of life and serious injuries. But incidents in oil & gas platforms can also lead to severe issues on the productivity of the plant, such as operations downtime, heavy costs and loss of reputation (e.g. the oil spill in the Gulf of Mexico on 2010).

PeT is a training and testing environment for both standard (SOP) and emergency operating procedures (EOP), implementing multiple-choice knowledge tests for two emergency procedures. The main objectives of PeT are:

  • Verify the knowledge of the standard procedures, with the aim to avoid incidents.
  • Ensure that workforce takes appropriate decisions in an emergency setting.
  • Create a competence portfolio for each operator while taking into account his/her past experience and expertise.
  • Provide remedial instructions and feedback to address gaps in the knowledge and competences of operators in the execution of both critical and non-critical tasks.

 

Concerning Learning Analytics, PeT tracks three kinds of data:

  • Session data, related to the time of completion of the overall knowledge verification session, ID of the employee and code of the operating procedure undertaken.
  • Question data, related to the time an user has spent in each single step of the procedure.
  • Choice data, related to the correctness of answers.

 

Moreover, the study has introduced an overall confidence metric (for the measurement of the ability of an operator to execute a task, both standard and emergency operating ones, correctly in the proper timeline with an optimal mindset) and a concept of criticality of each step.

The PeT tool has been tested and improved in two separate sessions in an oil & gas company in Canada in 2014. The experiment was conducted with several operators with different backgrounds and different levels of expertise.

Results show that, on the analysed group of workers, PeT is capable to efficiently assess the knowledge and behaviour of workforce in the oil and natural gas industry, in both standard and emergency procedures, ensuring traceable training for operators and leading to greater safety for workforce together with less risk of productivity losses for the plant.