Research trends in measurement and intervention tools for self-regulated learning for e-learning environments—systematic review (2008–2018)

Abstract

For the last one decade, research in self-regulated learning (SRL) and educational psychology has proliferated. Researchers and educators have focused on how to support leaners grow their SRL skills on both face-to-face and e-learning environments. In addition, recent studies and meta-analysis have greatly contributed to the domain knowledge on the use of SRL strategies and how they contribute and boost academic performance for learners. However, there is little systematic review on the literature on the techniques and tools used to measure SRL on e-learning platforms. This review sought to outline recent advances and the trends in this area to make it more efficient for researchers to establish the empirical studies and research patterns among different studies in the field of SRL. The findings from this study are concurrent with existing empirical evidence that traditional methods designed for classroom supports are being used for measuring SRL on e-learning environments. Few studies have used learner analytics and educational data mining (EDM) techniques to measure and promote SRL strategies for learners. The paper finally points out the existing gaps with the tools presently used to measure and support SRL on learning management systems and recommends further studies on the areas of EDM which can support SRL.

Introduction

Educational environments fall along a continuum from physical classroom where face-to-face interactions are common to fully online learning environments where asynchronous interactions are the default. This continuum of educational environments has provided opportunities for blended and web facilitated courses where learning materials and student-instructor interactions are delivered online with little or no face-to-face meetings (Allen & Seaman, 2013).

Presently, there is a recognized shift towards technology supported learning commonly known as e-learning with most of institutions of higher learning adopting e-learning for fully online courses or complementary to the face-to-face sessions in blended learning approach in order to curb the challenge of large backlog of students to be admitted (Hadullo, Oboko, & Omwenga, 2018; Luna, Castro, & Romero, 2017). As a result, there is increased number of students undertaking online learning courses (Bogarín, Cerezo, & Romero, 2018; Broadbent & Poon, 2015). The term “online learning” or as commonly known as e-learning refers to the web-based systems such as massive open online courses (MOOCs) and learning management systems (LMS) that are used by instructors to deliver learning materials and allow students to access the content and interact and obtain support during a learning episode (Delen & Liew, 2016). MOOCs are defined as open education systems for open and distance learning where students register for courses with limited admission restrictions such as pre-requisite courses and selection criteria.

Despite the benefits to online learning, existing literature indicates challenges that need to be addressed. First is offering adequate support and guidance to learners undertaking online learning (Kizilcec, Pérez-Sanagustín, & Maldonado, 2017; Terras & Ramsay, 2015). Offering individualized support and guidance may not easily be achieved because of large number of students enrolling on e-learning. The increased number of students taking online courses is likely to face the challenge of having enough human capacity to offer adequate support. To provide effective support and guidance to online students, we need to tap into the potential opportunities offered by educational data mining (EDM) and learner analytics (LA) tools. EDM is described as the approach of applying data mining algorithms on datasets generated from educational environments in order to understand learners and learning environments. The datasets which is in form of logs generated when learners engage to various online learning activities can be analyzed to produce inferences that can be used as indicators to provide interventions that reduce dropout rates and increase retention rates, profile learners, develop learner models, and recommender systems (Arnold & Pistilli, 2012; Romero, López, Luna, & Ventura, 2013; Romero & Ventura, 2007). LA on the other hand involves integration and analysis of data collected from educational environments for insights and patterns on how students engage on various learning activities during online learning. The main objective is to support students by providing interventions to improve on undesirable learning behaviors and reinforce positive learning (Lodge, Panadero, Broadbent, & De Barba, 2019). Lodge and Corrin (2017) opine that LA provide opportunity for monitoring students’ learning in order to understand their behavioral patterns and provide real-time interventions especially in online learning envrionments. According to Naif, Ayman, & Saeed-ul (2019), the outcome from LA helps in understanding the behaviour of learners with a view of providing early intervention mechanisms that enhances learning engagement which has been found to be positively correlated to academic performcance. This is likely to lead to reduced student dropout and increase the retention rates especially in higher education. In comparison, while EDM is concerned about techniques that can be used to explore data from educational environments and using the techniques to understand learners and learning environments, focusing on automated discovery of information, LA is more about analyzing and reporting of insights hidden in the data about learners and learning environments, focusing on insights to “inform and empower instructors and students” (Siemens & Baker, 2012).

Compared to physical classroom teaching where learners are confined together at certain periods, online learners are not restricted in managing their own schedules and learning process—what time to study and how long to engage in learning. The success of e-learning depends on the learner’s ability to take control of their own learning process (Nikolaki, Koutsouba, Lykesas, Venetsanou, & Savidou, 2017). The theory through which learners take control of the learning process is referred to as “self-regulated learning (SRL)”. Self-regulated learners are those who have the ability to take charge in managing their own learning while assuming an active role in achieving their academic goals (Zimmerman, 1990).

SRL is grounded on different theoretical models that provide frameworks on which research studies on SRL are carried out. According to Carlos Núñez, Romera, Magno, and Panadero (2017), the popular and commonly referred models include Zimmerman’s, Boekaerts’, Winne and Hadwin’s, Pintrich’s, Efklides’, and Hadwin, Järvelä, and Miller’s models. Each of these models describes phases, processes, and components that can be summed up into SRL strategies that are measured in a learning process. The strategies include time management, metacognition, effort regulation, critical thinking, rehearsal, elaboration, organization, peer-to-peer learning, and help seeking. Leaners who employ some or all of the identified strategies perform better than those with low level SRL skills and hence the need for supporting SRL on e-learning environments especially LMS which are majorly used by higher institutions of learning (Broadbent & Poon, 2015; Kizilcec et al., 2017; Littlejohn, Hood, Milligan, & Mustain, 2016). These strategies can be measured before, during, or after a learning process using instruments and methods specially designed for each of the SRL model.

According to Panadero, Klug, and Järvelä (2016), measurement of SRL is believed to have undergone through what is described as “three waves.” These waves are identified as first wave where self-regulation was conceptualized in terms of traits or characteristics that are inherent in learners and therefore measured using self-report tools such as interviews and questionnaires. In the second wave, SRL is viewed as events or processes that take place within a learner while being influenced by external environment through which learning takes place. The online SRL methods, which are used to measure SRL in the second wave, allow measurement without the learner being aware. This is achieved through the use of log data collected when learners are interacting with learning environments, instructors, and fellow students. The third wave is perceived as the “current wave” where SRL measurement approaches also serve as tools to promote or reinforce the self-regulatory skills in learners. Since the interest of this study was to outline the SRL measurement advances and trends, it is worthwhile to review the current studies in relation to the “three waves.” More importantly, this will be helpful to SRL researchers and educators to know the direction of these forms of SRL measurements.

Previous review studies

Recent studies and meta-analysis have greatly contributed to the domain knowledge that the use of SRL strategies boosts academic performance for learners. In this subsection, we highlight some of the recent systematic reviews that have been carried out and are related to the current study.

In their study, Wong et al. (2018) carried out a systematic review on the studies that have been carried out to support SRL in MOOCs and other online learning environment and investigated the role human factors play in SRL. The paper review focused on various levels of education from primary, secondary, and higher education and working adults.

In their review Lee, Watson, and Watson (2019) highlights the SRL strategies and interventions that have been employed from various studies with main focus on MOOCs. The SRL strategies reviewed in the study include self-efficacy and face value, goal setting, help seeking, time management, and effort regulation. Of the studies reviewed, only four employed some kind of interventions for promoting SRL, and these include software programs integrated in MOOCs, automated feedbacks, and prompts.

Garcia and his colleagues presents another review paper on e-learning tools and platforms that are used to support SRL strategies especially for computer science students using a taxonomy developed by Barry Zimmerman and Manuel Martinez Pons (1986). The study also sought to determine if other strategies have emerged that were not originally captured by the taxonomy. The categories being supported by various e-learning environments include self-evaluation, organizing and transferring, goal setting and planning, seeking information, keeping records and monitoring, environmental structuring, self-consequences, rehearsal and monitoring, seeking social assistance, and reviewing records. Of these categories, environmental structuring and seeking social assistance are not captured in the original taxonomy but have been identified and investigated by modern technologies employed by other e-learning tools (Garcia, Falkner, & Vivian, 2018).

Roth, Ogrin, and Schmitz (2016) highlight the self-report instruments used to measure SRL in higher education while capturing psychometric properties and characteristics. The self-report tools identified in the review include structured questionnaires such as Motivated Strategies for Learning Questionnaire (MSLQ), Learning and Study Strategies Inventory (LASSI), and Situational Judgment Tests (SJT); and interviews such as Self-Regulated Learning Interview Schedule (SRLIS), think-aloud protocols, and learning diaries.

Schraw (2010) reviewed four articles and highlighted the methods for measuring self-regulated learning in online learning environments. Schraw (2010) presents taxonomy of the tools and methods for measuring SRL—offline and online measures. Offline tools and methods are those that measure SRL before or after a learning period. Such methods include self-reports, current abilities, and expected performance. Online tools are those that are used during the learning process for example the use of analysis from data generated from an educational environment. Online techniques are unobtrusive as the measurements are taken while learners are unaware and therefore do not affect students’ engagement behaviors and performance.

These studies have greatly contributed to our understanding that use of SRL skills positively enhances academic performance for learners (Adam, Alzahri, Cik Soh, Abu Bakar, & Mohamad Kamal, 2017; Garcia et al., 2018; Lee et al., 2019; Roth et al., 2016; Wong et al., 2018). Despite these developments in SRL, our understanding on the measurement tools and instruments to measure SRL for online learning environments is limited as evidenced on the use of traditional tools that were developed for face-to-face still being used for measuring SRL online. Most of these studies focus on measuring SRL in MOOCs and only analyze SRL strategies with limited attention to the interventions to improve self-regulated learning. The last review on SRL scaffolds on computer-based learning environments was published in 2012 (Devolder, van Braak, & Tondeur, 2012).

In a recent study by Panadero et al. (2016), various measurement and intervention tools have been developed over the last decade. In what they term as “wave” of evolution of different SRL measurement tools, they argue that SRL measurement has undergone through “three waves” which can also be viewed as stages. This study sought to establish the status of these “three waves” in reference to the tools developed and belonging to each wave of evolution.

In this study, various methods and techniques that have been used to measure and promote SRL since its emergence in 1990s are reviewed. We take into account the historical context: the tools that have been used in the past and how they have fashioned the design of the tools used in measuring and promoting SRL presently and in future. The following research questions guided this study:

• RQ1: What methods and instruments are being used to measure SRL on e-learning environments in higher education?

• RQ2: What methods are being used to measure and promote SRL at the same time in on e-learning environments in higher education?

• RQ3: What is the trend in terms of measurements and interventions that are being used to promote SRL skills for learners on e-learning environments in higher education?

• RQ4: What EDM tools are being used to measure and promote SRL on e-learning environments in higher education?

Methodology

The review process followed the five-step methodology by Khan, Kunz, Kleijnen, and Antes (2003) for conducting systematic review which involves (a) framing the questions for the review, (b) literature identification, (c) assessing the quality of articles, (d) summary of the studies reviewed, and (e) result interpretation. Based on this methodology, we first discuss how the literature identification was done, secondly the criteria used to do quality assessment of the articles, thirdly summary of the studies reviewed, and finally the discussion of the results.

Identification of literature

Meta-search on papers with focus on SRL measurement tools and interventions on e-learning environments in higher education such as LMS and MOOCs was carried out. The keywords used include Self-Regulated Learning AND e-learning AND Learning Management Systems AND Assessing AND Measuring AND Supporting AND Promoting AND SRL Interventions. The databases incorporated in the search for articles include ERIC, PsycINFO, PsycARTICLES, Google Scholar, ACM, Research Gate, and IEEE Xplore digital library. A total of 158 articles published between 2008 and 2018 were identified.

Quality assessment and selection criteria

After reviewing the abstracts of the 158 papers, we identified 42 papers that were relevant to measuring and promoting SRL in higher education. Twelve more papers were removed from the analysis based on the inclusion criteria. The final set of 30 papers were reviewed, analyzed, and discussed in the “Results” section. After analysis of the content for each paper, we also identified eight studies that described SRL measurement approaches that also served to promote SRL in online learning environments, and the summary is presented in Table 2.

Inclusion criteria

The following inclusion criteria were used to obtain the relevant papers for this study:

• Articles that addressed the measurement or promotion of SRL

• Articles that described original work with actual SRL measurement or promotion tool developed

• Articles whereby SRL measurement or promotion tool was validated through experiments in institutions of higher learning

Results

This section describes the various tools and instruments used to measure and promote SRL on the various online learning environments in higher education that were identified from the review. Out of the 30 studies that were reviewed, 10 studies were carried out on LMS, 8 studies on MOOC environments, and 12 studies on other learning environments such personal learning environments (PLEs) while in four studies, the e-learning environment was not specified as presented in Table 1. In terms of the type and source of data used for SRL measurement or intervention, 16 out of the 30 studies used self-report tools such as structured questionnaires and interviews to measure the level of SRL skills in learners while 9 studies analyzed log data extracted from educational learning environments such as MOOCs or LMS to establish the levels of SRL for each learner. There were only 5 studies where both self-report tools and log data were used to measure SRL as presented in Table 1.

Table 1 Summary of the reviewed studies

RQ1: What methods and instruments are being used to measure SRL on e-learning environments in higher education?

The following SRL measurement tools and instruments were identified from the literature reviewed in this study:

• a. Online Self-Regulated Learning Questionnaire (OSLQ)

The Online Self-Regulated Learning Questionnaire (OSLQ) is a tool developed by Barnard et al. (2009) to assess students’ use SRL strategies in online or blended learning environments. The tool contain 24 items under five categories that are used to measure six SRL strategies which include goal seeking, help seeking, time management, task strategies, environment structuring, and self-evaluation. Onah and Sinclair (2017) also used a modified OSLQ tool to develop another tool used to measure SRL on massive open online courses systems known as MOSLQ. Similar to OSLQ is the Online Self-Regulation Questionnaire constructed by Cho and Cho (2017) to examine how learners interact with content, teachers, and other learners in an online course. The tool is a 19-item scaled tool that also assesses the six SRL strategies identified by Barnard et al. (2009). The tool was also used by Yen et al. (2016) during their study that investigated the impacts of SRL in online learning.

• b. Motivated Strategies for Learning Questionnaire (MSLQ)

The Motivated Strategies for Learning Questionnaire (MSLQ) has been in existence since the 1980s. Developed by Pintrich, the MSLQ is an 81-itemized self-report instrument which has been used for a longtime to measure SRL strategies mostly for physical classroom educational settings. In the studies surveyed in this study, the MSLQ instrument was administered online. The studies carried out by Hashemyolia et al. (2015), Müller and Seufert (2018), and Yamada et al. (2017) used the MSLQ tool to measure SRL strategies in online learning which included both the MOOCs and LMS environments.

• c. Learner analytics

Learner analytics (LA) is a recent and fast growing field that focuses on the use of learner data generated from various learning environments. Once log data is collected, it is analyzed for making inferences that can be used to inform and understand learners’ engagement behavior during online courses. Learning analytics is “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Long & Siemens, 2011). The log data collected during an online learning episode can also be analyzed using data mining algorithms to provide inferences on how students self-regulate and generate visualized SRL reports to teachers and students. According to Roll and Winne (2015), LA provides a new dimension to better SRL research for learners and learner environments. Unlike the use of self-report tools to collect information how students regulate based on their opinions about themselves, LA rely on the “traces” that students leave behind when studying an online course. Current online learning environments record and store all the logs capturing how and what activities students engaged while studying online. The data is analyzed to provide evidence for self-regulation for each learner. The advantage of using LA or EDM over other approaches of measuring and promoting SRL is that they are unobtrusive in nature as opposed to the obtrusive methods such as self-report methods. With obtrusive methods, students are aware that SRL is being measured about them and are likely to change behaviors for being aware that measures about themselves are taking place (Schraw, 2010). EDM or LA methods are also advantageous over self-report tools as allow both SRL measurement and interventions to co-occur.

In this review, some studies employed LA in SRL measurements through the use of student logs and traces from LMSs, MOOCs, and PLE. Alharbi et al. (2014) used LA to design a learning object system based on SRL and developed a software agent to measure and scaffold SRL strategies. Arnold and Pistilli (2012) employed LA on both online and offline data to provide visual reports to teachers about students’ SRL levels. Lecturers will then submit interventions feedback and reports to students manually. The use of LA in this project helped the lecturers to easily monitor SRL for students and avail reports to students. In other studies such as Cicchinelli et al. (2018), Davis, Chen, Jivet, et al. (2016), Lee and Recker (2017), and Nussbaumer et al. (2015), LA is used to identify SRL strategies for learners through the use of LMS and MOOC data and provide learner feedback through dashboards. In their study, Rodriguez Groba et al. (2014) used LA to develop a software agent known as SoftLearn tool used by teachers to assess SRL skills in students. The tool provides visualized learner activities which are made available to teachers for measuring SRL levels on LMS students.

• d. Other online survey tools

In this study, it was discovered that there are several other new online survey instruments that have been developed and implemented. The survey tools are self-report instruments which were administered online in the various projects. The studies in which online survey tools were used include Chaves-Barboza et al. (2017), Gaupp et al. (2018), Jansen et al. (2017), Kizilcec et al. (2016), and Kizilcec et al. (2017).

Other self-report tools developed also include the Self-Regulation Measure for Computer based Learning (SRMC) which was developed by Song et al. (2011) by modifying the Zimmerman’s Self-Regulated Learning Interview Schedule (SRLIS). The SRMC was used to assess medical students’ self-regulation as aptitude in computer-based learning environment.

RQ2: What methods are being used to measure and promote SRL at the same time in on e-learning environments in higher education?

Self-regulated learning promotion or intervention is described as an activity or event that can “trigger SRL development” within an online student during learning episode. While SRL strategy measures seeks to establish SRL levels for learners, SRL interventions aim at strengthening or stimulating the growth of the inherent SRL skills in leaners Triquet, Peeters, and Lombaerts (2017). After content analysis of the 30 studies reviewed in this study, eight studies that described SRL measurement approaches that also served to promote SRL were identified and the summary is provided in Table 2.

• a. Learner analytics and dashboard visualizations: The findings from the reviewed literature indicate that LA is used as a tool to measure SRL strategy and also used to promote SRL at the same time by enhancing learner activities through provision of insights using dashboards visualization. The data used for the LA is generated from the various learning environments such as LMSs, MOOCs, and PLEs. These studies used learner analytics by offering dashboard feedback to students (Arnold and Pistilli, 2012, Davis, Chen, Jivet, et al., 2016, Davis et al., 2016, and Nussbaumer et al., 2015).

• b. Software agents: The use of artificial intelligent software agents was also identified in the study. Software agents are intelligently powered to offer assistive activity guidelines that stimulate the growth of SRL skills for the students. When the learners engage on the suggested SRL tasks by agents, the SRL skill growth is triggered. The software agents identified in this review include ProSOLO software by Dawson et al. (2015), eLDa tool by Onah and Sinclair (2017), SoftLearn tool by Rodriguez Groba et al. (2014), and Learn-B software by Siadaty (2016).

• c. Learner feedback: As presented in Table 2, we established that a number of studies employed web-enabled prompts to provide feedback to learners. The feedback provided to learners was in most studies facilitated through the use of LA reports submitted to teacher who then used the feedback to assess learners’ use of SRL strategies (Cho & Shen, 2013; Dawson et al., 2015; Onah & Sinclair, 2017; Winne & Hadwin, 2013)

• d. Other systems and applications

• i. The Student Relationship Engagement System (SRES)

Table 2 SRL measurement tools that also acted as intervention tools

The SRES, a standalone learner analytics tool that enhances teacher and student interactions, developed by Liu and his colleagues, allows data from different sources to be fed into the system for analysis. The teachers use the information to contextualize and understand each learner. The learners are notified through automated feedback and therefore are able to receive support from teachers through data-driven personalization. The analyzed data is used by teachers to align the needs of the students of each student through the use of educational data from the various sources. The only drawback to the system is that it is a standalone and not integrated to LMS or e-learning system (Liu, Bartimote-Aufflick, Pardo, & Bridgeman, 2017).

• ii. SoftLearn

The SoftLearn was developed by Groba, Barreiros, Lama, Gewerc, and Mucientes (2015) and is based on learning analytics. The tool is used by instructors to evaluate students’ SRL skills. The preferred learning environments where the tool can be deployed are PLEs, e-portfolios, and social networks. The tool presents to the instructor the learning path being followed by learners for assessment through a graphical interface. The limitation to this tool is that the instructor is placed at the center of the evaluation process, and when the number of students is large, the challenge will be if the instructor will be able to provide the required assessment for each learner.

• iii. OnTask

OnTask is an open source platform implemented by (Pardo et al., 2018). Instructors use data collected from learning environments and other sources to provide personalized learning student actions. The instructors are able to use the tool to manage data to understand the learner and then specify the actions for learners through automated feedback that enable the student reflect on various learning activities and apply them in the learning process. Like the SoftLearn tool, the OnTask model presents analyzed information to the instructors inform of visualized feedback. Instructors will then provide personalized support to students based on the information presented to them.

RQ3: What is the trend in terms of measurements and interventions that are being used to promote SRL skills for learners on e-learning environments in higher education?

This study sought to establish the status of the “three waves” in reference to the tools developed and belonging to each wave of evolution (Panadero et al., 2016). In taking account of the historical context, the tools that have been used in the past and how they have fashioned the design of the tools used in measuring and promoting SRL presently and in future are presented in this section. The SRL measurements tools and methods therefore were analyzed in relation to the “three waves” that are explained below:

• a. The first wave: SRL measured through self-report tools

This is believed to be the first era of SRL measurement where SRL was viewed as trait based. Self-regulated learning was conceptualized as an individual inclination that based on traits of the learner without contextual considerations of the learning environment. This led to the development of tools and measurement methods that were trait-based. These methods and tools include the self-report instruments such as structured questionnaires and interviews that were used to assess SRL before or after a learning process. When self-reports measures are used, there is tendency for learners overestimating the use of their SRL skills (Winne & Perry, 2000). Self-reports are user-oriented and therefore depends on how learner perceives themselves as far as SRL level skills are concerned. The measures are also deployed outside the learning environment (Roth et al., 2016). According to Lee (2008), SRL can only be measured in the context of an actual learning environment where it occurs. Since the self-report tools analyze SRL before or after a learning period and are designed to only measure SRL, they do not provide interventions for SRL. Such tools include structured questionnaires, interviews, current abilities, and expected performance (Schraw, 2010). The trait-based measures could later be replaced by domain-based measures where SRL was measured during learning process (Boekaerts & Cascallar, 2006).

• b. The second wave: online measures of SRL

During this wave, SRL was conceptualized as a process-oriented and measured SRL as a series of events or processes. As a result, tools that assess SRL during the learning process started emerging. SRL started to be perceived as event-based where approaches such as the use log data (traces) and observation of learner behaviors are used to measure SRL. The objective was to be able to measure SRL without students being aware while considering the contextual learning environment. This approach is unobtrusive since SRL is measured during learning process without triggering change of behaviors of learning (Schraw, 2010).

• c. The third wave: measurement approaches that provide SRL interventions

In this wave, the SRL measurements and interventions are designed so that they can co-occur. According to Panadero et al. (2016), we are now in the phase of the third wave where tools used to measure SRL also provide interventions for enhancing learners’ self-regulatory skills. They argue that we have started to witness development and deployment of SRL tools that not only measure SRL but also provide interventions for supporting SRL skills. According to Triquet et al. (2017), SRL measurement is carried out to establish the levels SRL skills for learners while SRL interventions stimulate the growth of SRL skills within learners hence becoming beneficial to learners when interventions are rendered during the learning episode. While the “three waves” are independent to each other, the usage is expected to overlap.

From the findings, it can be noted that the traditional SRL measurement methods and tools that were designed for physical classroom setting are still used for measuring SRL on e-learning environments. Although the self-report tools and instruments that include questionnaires and interviews have been found to be reliable and effective in measuring SRL strategies in learners (Roth et al., 2016), there are challenges that have been raised by researchers that need to be considered. The self-report tools have been found to be are bias and only record SRL based on students’ perceptions about themselves. Additionally, researchers argue that the tools were effectively designed for use in face-to-face classroom settings and may not therefore apply in online learning environments. The challenges are presented in Table 3.

Table 3 Challenges of measuring SRL on e-learning platforms

The findings from this review also indicate that while the self-report tools have continued to dominate in the SRL measurements, there is a recognizable shift towards the use of approaches that are able to provide SRL measurements and interventions as well. All the studies that provided SRL interventions relied on log data that was analyzed to provide levels of SRL and at the same time provided mechanisms for SRL improvement (see Table 3). These studies used learner analytics and/or dashboard feedback to students (Arnold & Pistilli, 2012; Davis, Chen, Jivet, et al., 2016; Davis, Chen, van der Zee, Hauff, & Houben, 2016; Nussbaumer et al., 2015). The reviewed studies indicate that there are two types of SRL interventions that are provided in online learning environments: (a) interactive feedback through visualized dashboards and (b) metacognitive and behavioral prompts and hints that aim at engaging students to enhance their SRL capability. The use of visualization dashboards allow learners to see their learning behaviors without stimulating development of SRL skills. The effect of visualized dashboard is therefore passive in nature as they only provide one-way and non-interactive feedback. From the studies reviewed, it was established that in some studies, visualized reports regarding learners’ behaviors with respect to SRL are delivered first to instructors for their synthesis and interpretation so as to know how to provide individualized support to learners. For example, the OnTask model by Pardo and his colleagues is designed to provide students’ support actions based on analyzed data from different educational sources. With this model, the instructors are presented with analyzed information from various sources inform of visualized feedback on learners. This information then guides the instructors on how to offer personalized support (Pardo et al., 2018). The other model which is similar to OnTask is the SRES (Liu et al., 2017). SRES is standalone learner analytics tool that is intended to enhance instructor-student interactions. It allows data from different sources to be fed into the system for analysis. Instructors rely on the information to contextualize and understand each learner’s learning behaviors. In essence, the analyzed information is used by instructors to align each of the students’ needs through the use of educational data from the various sources.

RQ4: What EDM tools are being used to measure and promote SRL on e-learning environments in higher education?

In this section, we present the findings on how EDM and LA are being used to measure and promote SRL on e-learning environments in higher education settings. From the review, it is evident that data mining techniques are being used in measuring and promoting SRL. Educational data mining methods and learner analytics are now being applied on data collected from educational environments such LMS, PLEs, and MOOCS. The datasets from the educational learning environments is collected for analysis in order to understand learners, tutors, and learning environments. EDM also helps the researcher and educators to understand how students engage during learning process with respect to self-regulation and to what extent to which the students employ self-regulating skills on e-learning (Kizilcec et al., 2017).

In their study, Cavalcanti et al. (2018) developed a model to predict the performance of students based on self-regulated learning skills by use of EDM methods in identifying SRL indicators from datasets collected from an e-learning system. The prediction was based on SRL indictors and learners’ behaviors, motivation, and application of cognitive abilities. Other software tools that have employed the use of educational data mining include MetaTutor metacognitive tool and nStudy tool (Azevedo et al., 2009; Winne & Hadwin, 2013). The MetaTutor software is a learning environment that promotes self-regulatory skills to high school and college students through the provision of feedback inform of prompts that guides students on the activities to engage on during learning. The nStudy tool also provides learning hints to students. Prompts and hints provided by these tools are geared towards enhancing SRL learning in students and hence qualify as intervention approaches.

Notably, there is progress on the SRL measurement tools used in measuring SRL in online learning environments—from the self-report tools that measured SRL before or after learning to analytic tools that are employed to measure SRL during a learning episode.

Discussion

The purpose of the current study was to investigate the research trends in terms of SRL measurement and promotion for SRL online and establish research gaps between the period 2008 and 2018. From the review, it can be noted that measurement and promotion of SRL has greatly advanced from the use of traditional methods which relied on learner perceptions on their SRL skills and use of offline data to online measures such as the use of learner logs from e-learning environments. The findings from the review indicate that there is noticeable evidence of a recognizable shift from using tools that only measures SRL to tools that measure SRL while providing interventions that stimulates growth of SRL in learners during the learning process (Azevedo & Witherspoon, 2009; Barnard et al., 2009; Cho & Shen, 2013; Dawson et al., 2015; Delen et al., 2014; Onah & Sinclair, 2017; Siadaty, 2016; Winne & Hadwin, 2013).

The study provides an understanding on the research trends in terms of the tools and instruments used to evaluate and promote SRL online learning environments. In some studies, instructors were presented with processed information through data analytics dashboards. The instructors could utilize the analyzed information in offering support to students based of the processed information (Liu et al., 2017; Pardo et al., 2018). The visualized reports about learners’ behaviors are first delivered to instructors for their synthesis and interpretation so as to know how to provide individualized support to learners. While these tools support instructors to gain insights on how learning is taking place and allow them to customize SRL support to students, one issue need to be addressed: the large number of students enrolled on online courses and reliance on human judgments in the provision of interventions. This approach requires that instructors know how to interpret data and make human judgments before offering feedback to students. The increased number of students taking online courses could be an hindrance to human capacity of offer adequate support. Offering individualized feedback to e-learning students may no longer be tenable as the number of learners is becoming large for tutors to guide them individually (Nussbaumer et al., 2015). Researchers and educators there need to shift from instructor-centered support to data-centered applications based data collected from various e-learning sources such as LMSs, MOOCs, PLEs, social education networks, and e-portfolios.

From the review, the SRL measurement and promotion approaches can be categorized into two. First category is approaches that extend the decision making capability for teachers to be able to offer data-driven and personalized support to learners. These approaches take advantage of teachers’ knowledge and augment its information from the analyzed data. The second type of approaches are those that offer metacognitive feedback by making learners stop learning and reflect on the learning process and then proceed. Most of the existing studies focus on the first model. The increase in number of online learners however implies that it may not be easy for online instructors to interact with every student and provide individualized guidance and support. Researchers therefore argue that the effective way on how EDM can help promote SRL is the provision of individualized interventions through metacognitive feedback such as hints and prompts (Lodge et al., 2019).

It can also be noted that there is developing potential of using EDM tools to provide measurement and interventions concurrently. Interventions, when implemented within the measurement tools, could play a significant role in stimulating the growth of SRL skills. Although this study identified an observational trend that the use of learning analytics and EDM in measuring and promoting SRL has started to emerge and now advancing, literature indicates that there is continued use of self-report tools that were originally created for the traditional face-to-face classroom set-up. According to Winne and Baker (2013), EDM can been used to identify, model, and predict learners’ behavior. It can also be noted that that the SRL measurement + intervention tools have started to emerge. So far, we have had separate tools for measuring and promoting SRL. Some of the studies that used LA as an alternative approach to measuring SRL also used the dashboard results from LA to promote SRL skills for learners on e-learning environments. This review indicates that EDM and LA are now being applied to establishing learner behaviors in online learning. Their guided implementation could lead to the development of various tools that are used specifically to mine education data generated from various learning environments including web-based systems such as LMS. The EDM tools accomplish various aspects of data preparation, modeling, processing, analysis, and visualizations. The tools ensure availability of visualized and interactive feedback on learner styles to both students and instructors. Real-time access to visualized feedback will enhance continuous monitoring and support for the benefit of learners to self-regulate based on the personalized feedback received on LMS dashboard. External agents such as instructors will also be able to use the feedback to provide personalized support/scaffolding to learners. The impact will increase learner motivation, satisfaction, and better learning outcome. This will also enable researchers and instructors to detect, isolate, and engineer changes on e-learning environments that impact learners. Early interventions and support that can be offered to learners via EDM tools will lead to improved performance for learners and reduce drop-out rates and reduce the time learners take to graduate especially through online courses. The real-time visualized feedback from actual datasets can also be accessed by instructors to monitor how SRL skills for learners change over time. Additionally, the reviewed literature indicates some challenges experienced when measuring SRL on online learning environments as presented in Table 3.

Despite the challenges identified in this review, it has been established that self-report methods that were designed to measure SRL in face-to-face classroom setup are continuously being used to measure SRL in online learning environments. Although researchers argue that the popularity of self-report tools could be their reliability and validity that has been proven over the years (Roth et al., 2016), self-report tools are obtrusive in nature. When learners are prompted to provide their perceptions on their SRL skills, they may not only overestimate their responses but also fail to capture their actual study behaviors. Literature also indicates that self-report tools are usually modified or enhanced to fit the context of online learning environment while the items still remain same as those that were designed for face-to-face classroom settings. This denotes that continued use of self-report tools is likely to lead to situations where the measured SRL levels do not represent actual learner behaviors (Broadbent & Poon, 2015). According to Winne and Baker (2013), measuring metacognition and motivation faces the challenge of identifying the constructs that can be modeled especially when self-report tools are used. They argue that the instruments used to gather and process metacognition and motivation are unreliable and erroneous in noticing change of state of learners’ skills and that in most research studies, experimental data is usually manipulated in order to improve reliability of the instruments used.

It would also be important to observe that there is a lack of a model that can be used to implement the “third wave” of SRL measurement and promotion in higher education especially in online learning environments. The existing studies did not describe an EDM model for implementation except for one study that proposed a conceptual model (Araka et al., 2019). To address the challenges encountered when using self-report tools for measuring SRL in online learning environments, we propose the use of EDM techniques for measuring and promoting SRL as EDM relies on the use log or trace data collected from educational environments as indicators of SRL. Given the observed challenges, there is a need to develop a framework that helps to integrate the SRL measurement and promotion tools with EDM tools.

Conclusion

This study presents various tools and methods that have been used to measure and promote SRL for online learning environment for the last one decade. The potential of EDM in measuring and promoting SRL on e-learning environments has also been established. While there are challenges in measuring and promoting SRL strategies in online learning environments, some of them may be addressed through the implementation of EDM techniques. Effectively, the techniques need to be deployed on e-learning systems such as the popular learning management systems which are being used by most of institutions of higher learning to offer both blended and online course to students. Similarly, the EDM tools have the potential of capturing and analyzing real-time learner traces and present visualized feedback to learners and hence allowing continuous assessment of SRL. Learner scaffolds can also be provided through learner dashboard. This implementation allows students to be supported and guided while studying online without the limitation of numbers of students enrolled (Araka et al., 2019). However, we found out that there is lack of a framework on how the EDM measurement and intervention models can be conceptualized and deployed on LMSs.

Additionally, this systematic review indicates SRL interventions have been important in improving or stimulating growth of SRL. Nonetheless, one critical issue that researchers need to address is the lack of a model that captures all the SRL strategies from the existing SRL conceptual models and maps them to the LMS data indicators from which SRL strategies for each learner can be inferred. In order to design a model that can be generalized, there is a need for further study on existing frameworks that are used to provide SRL interventions, nature of interventions, and indicators that were used in each study. This will then act as guide to understanding the nature and effectiveness of SRL interventions provided for each learner. According to Lodge et al. (2019), it is indispensable that e-learning environments provide tools to evaluate students, establish the levels of their individual engagements, and identify those who need interventions and reinforce each learner with the right kind of interventions.

Researchers in the field of SRL have started to embrace the use EDM and LA in measuring and providing SRL interventions. Future research should now focus on implementing EDM approaches that measure and provide SRL interventions in real-time. More interesting will be the integration of EDM tools into existing LMS for higher institutions of learning and MOOCs. This will enable collection, analysis, and provision of feedback to learners in real-time ensuring individualized support to learners. The educational data mining tools must provide solutions that should make use the rich and actual datasets from online learning environments to provide inferences on students’ levels of SRL skills while at the same time providing interventions for promoting SRL skills among learners.

Currently, most institutions of higher learning have adopted e-learning for online courses to curb the increased demand of higher education (Hadullo et al., 2018). However, the numbers of instructors are not enough to provide the support in terms of self-regulated learning (Muuro, Wagacha, Oboko, & Kihoro, 2014). Consequently, there is a need to include other tools such as EDM which can be used to promote SRL with little human intervention. Even though such tools have been developed, from this review, there is a need to carry out more empirical evidence on the effectiveness of using EDM measurement and intervention models compared to human interventions in promoting learning in institutions of higher learning given the increased demand of higher education.