Learning analytics of humanities course: reader profiles in critical reading activity

Abstract

This study investigates learner’s reading behaviors in a critical reading task in humanities course using learning analytics techniques. A Critical Analysis of Literature and Cinema course was selected as a context. The course activities evolved over 10 years, and for this instance, some face-to-face classroom critical reading activities were migrated to online mode by using BookRoll, a learning analytics enhanced eBook platform. Students (n=22 out of the 50 registered) accessed Hayavadana, an Indian play uploaded on BookRoll, and attempted to identify performative elements and cultural references in the text and highlight them. In this study, we analyze learner’s reading logs gathered in the learning record store linked to BookRoll during that activity. We extend our previous work where we identify four online reading profiles: effortful, strategic, wanderers, and check-out, based on learner’s clickstream interactions and time spent with the content. We validate the profiles with qualitative interview data collected from the learners and illustrate the quantified learning behaviors of each of those profiles based on an engagement metric. Our work aims to initiate further discussion related to the application of learning analytics in humanities courses both to probe into the learning behaviors of the students and thereby enhance the experiences with the use of interactive learning environments and data-driven services.

Background and motivation

Learning analytics as a domain has evolved over the last decade to apply various computational techniques to collect, analyze, and understand data related to teaching-learning experiences and thereby enhance them. There are many studies which look at the different applications of learning analytics (LA) in STEM domains (Sergis et al., 2019). At a university level, tools like AcaWriter (Knight et al., 2020) have implemented LA techniques to support both students to provide feedback and teachers to assess critical writing assignments across various disciplines such as law, pharmacy, and accounting. In this pilot attempt, we focus on a humanities course with critical reading as the activity. Using BookRoll, a learning analytics enhanced e-book-based learning platform, we investigate student’s behaviors at a critical analysis in a literature course during their specific activity to analyze an act of a play. Learning traces are automatically collected as interaction logs based on the reader’s actions (navigation by clicking on buttons, annotating by using highlighting functions, etc.) which could then be analyzed. Understanding the process of development of critical thinking skills is an important research aspect (Douglas, 2000). It can potentially help to inform teachers to design learning activities to support their students and also system developers to create technology assistance to orchestrate those activities. Hence, in such an authentic natural learning setting, we investigate the following two research questions:

• 1. What are the reading behaviors of the learners given the critical reading task in terms of interactions within the content and the time spent on that task?

• 2. What are the different profiles of learners during the critical reading task, and what are the characteristics of those profile groups?

The article is organized in the following sections. The “Foundation of the study” section provides the foundation of the study and looks at the related work. The “Study context and methods” section illustrates the context, learning task, and the research methods. The “Results and interpretation” section presents the results of the analysis. The “Discussions and conclusions” section discusses the implications of the study. This article extends an initial conference publication (Majumdar, Bakilapadavu, Majumder, Chen, Flanagan & Ogata 2020) by conducting further data analysis to quantify engagement metrics of the different reading profiles and triangulating them with focused interview data. Further, an approach to apply learning analytics in humanities education context is discussed as a reflection on the current study.

Foundation of the study

Critical reading activities

Critical reading is an active, in-depth reading of a text that calls forth a deeper engagement with the text. Such an activity requires cognitive tasks such as comprehending, analyzing, evaluating, interpreting, and synthesizing. A critical reading activity in the context of the present study requires one to highlight important ideas in the text, relate it to one’s personal experiences, pose questions and think about answers for such questions, look into the patterns within the text, and make connections with other texts. In certain contexts, it would involve identifying sociocultural contexts and reading through them. Critical reading enables the reader to read not only the explicit meanings but the layered and the implicit meanings as well. Over and above, critical reading enhances one’s ability for task-focused thinking. Critical thinking is reasonable, reflective thinking, focusing on a task, people, or belief (Ennis, 1993). Also referred to as “good thinking,” “thinking well,” and “smart thinking,” it enables one to identify questions worth pursuing through self-directed search and interrogation of knowledge (Pithers & Soden, 2000). Studies have also highlighted how inquiry and critical thinking are related as a process (Spector & Ma, 2019).

From the humanities education standpoint, developing critical reading skills is crucial. One of the essential values of humanities domain is identified as critical thinking (Holm, Jarrick, & Scott, 2015), especially so in the case of courses that deal with cultural texts including from the field of narrative arts. While understanding subjective experiences embedded in the texts and relating with them are important, deciphering the layered meanings is also equally significant. Developing critical reading skills enables one to do all the above tasks for a much-enriched meaning-making process.

In the context of reading a cultural text, it is imperative for the reader to be able to identify various pointers or references that bring the cultural context to the fore. It is well established that the reader’s prior exposure to the cultural context enhances how readers relate to the text. The instructor, through her classroom experiences and reflections, had identified the role of annotation in making the reader dive deeper into the text. In this particular module being discussed, the text chosen is loaded in its cultural references and certain traditional theatrical conventions as performative elements. Hence, the instructor identified the learning task to annotate cultural references and performative elements. Learning logs were gathered for analytics during that task.

Learning analytics and critical reading activity

Critical reading from the perspective of critical thinking by the learners was studied using a technology environment at the school level in Singapore (Jonathan, Tan, Koh, Caleon, & Tay, 2017; Tan, Yang, Koh, & Jonathan, 2016). The study used a collaborative environment and a LA dashboard for supporting critical reading activities at the school level. User acceptance of the innovation and its associated usability issues were its main focus. Our earlier studies with university-level students proposed a specific in-class pedagogical model and studied their reading behaviors while they were comprehending English as a foreign language (Chen, Ogata, Hwang, Lin, & Akçapınar, 2019). Other works looked at embedding strategy prompts in digital text and found it had a positive effect on learners’ cognitive load, achievement, attitudes, metacomprehension, and calibration accuracy (Reid, Morrison, & Bol, 2017). However, another study from the perspective of the influence of the media (digital text vs physical text) on comprehension of text (Ben-Yehudah & Eshet-Alkalai, 2018) found learners improved only in printed condition while answering questions that required inferential processing.

Our study focused on gathering learning traces during an authentic learning task related to critical reading. Such a process-driven narrative regarding learner’s behavior in a critical reading task, specifically in a humanities course, is relatively rare, and we want to fill in that knowledge gap.

Learning Evidence Analytics Framework (LEAF)

Learning Evidence Analytics Framework (LEAF) is an overarching technology framework to collect evidence of learning and teaching from the logs generated in a technology-enhanced learning environment (Ogata et al., 2018). In this instantiation of the framework, the instructor coordinated the course on Moodle, an LMS. BookRoll, an e-book reader, was used to upload reading contents like lecture slides, reference articles, and reading assignments in PDF format for students to access. Tools like BookRoll can be considered as a learning behavior sensor as it can log student’s reading and annotation interactions in a Learning Record Store (LRS) as standard Experience API (xAPI) statements. Figure 1 presents the technical architecture based on LEAF that is used in our study and the user’s reading interface in BookRoll which supports annotation functions such as highlighting with different colors and adding memos and bookmarks in the content. As long as there is an internet connection, students can read their books anytime from a web browser on their personal computer or smartphones. Student’s reading activity log from the LRS is then provided to the dashboard database and visualized for both the instructors and students appropriately.

Fig. 1

Learning Evidence Analytics Framework and BookRoll reading interface

Study context and methods

As an initiative of a collaborative research project, access to the LEAF platform was given to the instructor and her students at one of the private universities in India. An overall phenomenographic research approach was chosen for the study (Larsson & Holmström, 2007; Marton, 1981). It guided the research questions to focus on a single activity undertaken by the students enrolled in the course. The team of researchers including the course instructor then interpreted the different approaches that emerged from the learning logs.

Context

Course and its objectives

This particular study was conducted in an undergraduate elective course, Critical Analysis of Literature and Cinema (CALC) offered by the Humanities and Social Sciences Department. The objectives of the CALC course are three-fold: (1) to inculcate in students a critical insight required to interpret a work of literature and cinema, (2) to enable the students to perceive the subtle nuances of such works and to develop critical judgment, and (3) to introduce different forms, terminologies, and trends prevalent in such artistic ventures to enable them to place a work of art in the proper context. The class was scheduled for 3 h each week, split across three sessions. Students met for a total of 15 weeks. In addition to these classroom interactions, students were given take-home readings and film viewings. The semester in question had to undergo a sudden change of plan due to the pandemic and the early lockdown. It was the 9th week into the semester that the instructor had to shift the regular classes to online mode.

Participants

Students enrolled in the course (n=50, 17 males, 23 females) were pursuing their undergraduate program in engineering and sciences in the university. Their ages ranged from 19 to 23 years, and the class included students in their second, third, or fourth year of study in the university. At the time of the research, they had been introduced to approximately 1 to 3 humanities courses as electives. All the students were registered on Moodle for the following activity.

Critical reading activity: instructions and example

One of the modules in the course related to critical reading and analysis of a play was orchestrated on the LEAF platform. The instructor chose an Indian play titled Hayavadana (Karnad, 1972), originally written in Kannada and then translated into English by the playwright himself. The content was uploaded on BookRoll, and the students were given the task of going through the first act of the play to identify and highlight the cultural references (red highlight) and performative elements (yellow highlight) in the text designated. The activity was designed around these two factors as the play is deeply steeped in the cultural milieu of traditional Indian theater. Also, these two tasks were significant for a critical understanding of the play; however, it was not a graded activity in the course.

Instructions for the task given to the students were posted on the LMS followed by announcing them to do it during the online synchronous class. There was no intervention from the instructor’s side during the reading and annotation activity. An example of the cultural reference and the performative element is shown below. We have selected 2 of the pages which the students have spent most of the time (Fig. 2) (see “Overview of interactions and time spent on learning task” section for further details).

Fig. 2

Cultural references and performative elements in page 8 and page 12 of content

Data collection and analysis

The data extracted for this study included the reading logs from 13 April 2020 to 1 May 2020 in BookRoll. Of the total 50 students enrolled in this course, 22 accessed BookRoll for the reading activity. 2587 logs were captured when the students were reading the play. Out of that, 1575 actions corresponded to opening the play, navigating through its pages, and annotating its content. The highlighting annotation is considered towards fulfilling the given critical reading task.

To answer RQ1, what are the reading behaviors of the learners given the critical reading task in terms of interactions within the content and the time spent on that task? LAViEW, the learning analytics dashboard associated with BookRoll, provided processed data. Students’ access count, their annotations, and the time spent on the specific page were reported. Using the data export option in the dashboard, the interaction data for each student was retrieved for further analysis. We considered the following interaction logs: OPEN, ADD MARKER, ADD MEMO, NEXT, and PREV. Table 1 provides the definition of the different interaction logs considered and its possible interpretation. For each of the interaction details logged in the xAPI statement, include the instance of the user’s machine readable user id, page number of the content where action was enacted, timestamp, and context information such as the course id in the LMS and the role of the specific user. Also, some details are specific to some actions such as in ADD MEMO the input is recorded, for ADD MARKER the text highlighted is captured, and for PAGE JUMP the final page visited is noted.

Table 1 Description and interpretations of the reading action logs

For this study, the total interactions for each student on the reading content are first aggregated. In the LAViEW dashboard, it is presented as events corresponding to each user. The time between each action is computed by the system; it can be aggregated for each individual user or average value on each page. Reading time in the dashboard is the count of the time in minutes. Long events are counted as the percentage of actions whose time spent is greater than 3 s (Akçapınar, et al., 2019). Total counts of the red and yellow markers are aggregated and also represented for each page in the dashboard. The count of markers on the pages relevant for the portion of the text is considered as an attempt on the task of critical reading. The total number of pages and the sequence of the pages where action is logged are computed to represent the navigation pattern across the content.

To answer RQ2, what are the different profiles of learners during the critical reading task, and what are the characteristics of those profile groups? we calculated three learner attributes from the learning logs for the specific learning task. First was the task attempt indicated by ADD MARKER action by the reader, the second was the reading pages calculated as the set of pages visited in the content, and third was the reading time as the sum of the time across pages. Figure 3 presents the simple algorithm to differentiate the four profiles of the readers based on the three attributes extracted above. They are effortful, strategic, wanderers, and check-outs.

Fig. 3

Flowchart to separate learner profiles based on the learning tasks and reading behaviors

For the members of each group, the engagement score (Akçapınar et al., 2019) is computed and then aggregated for each group. The engagement score has 9 different features including behavioral engagement indicators such as total number of events, number of times the content was opened, and cognitive engagement indicators such as red/yellow marker usage counts and memo counts.

Lastly a semi-structured email-based qualitative response was requested from the selected representative participants from each profile group. Their responses were used to preliminarily triangulate their reading profiles. The question asked to them was to write a short memo on the experience of the critical reading activity by reflecting upon it. Further, they were asked to mention if and in what way the annotation task impacted their critical reading of the text. One participant from the effortful and one from the strategic group responded. Their email responses were interpreted by two of the researchers and reported.

Results and interpretation

Overview of interactions and time spent on learning task

In LAViEW, the learning analytics dashboard, the overview of the content engagement is aggregated (see Fig. 4). It provides the instructor the aggregate counts of learners who accessed the material and their average interactions. A single student’s attributes can also be selected in the dashboard. Here, student #2548 is selected as an example case. It shows that the student kept the content open for 108.2 min when the class average recorded was 35.5 min and completion of the material as measured by pages browsed in 83% compared to the class average of 41.4%. The student had a total of 326 events with 47.2% of them more than 3 s duration (considered as a long event). The count of the reading activities such as maker and memo annotations are also given in the overview widget for both the teacher and the students. As the instructions for the reading task did not need any commenting in memos, and no introduction of the memo feature was done explicitly for the students, we found only one student used the memo function for noting reflection while reading the text.

Fig. 4

Overview information from LAViEW dashboard

The aggregated average page-wise viewing time is presented in Fig. 5. It is calculated by considering content open as a proxy. The difference in time from landing on a page till the next page or close interaction is considered as time spent on a particular page. Average page-wise viewing time was 2 min considering all the users who viewed the book. Based on the distribution, we analyzed the annotation activities only in the pages with the top two average times spent, page 8 (4.3 min) and page 12 (4 min). The distribution of the annotation behavior across the pages is presented in Fig. 6. It also indicates that page 8 had the greatest number of yellow markers, annotating the performative elements, and page 11 had most of the red markers annotating the cultural references.

Fig. 5

Average page-viewing time across all viewers who read the content

Fig. 6

Distribution of markers across the pages of the content

Distribution of profiles

The defined four basic profiles of the readers were based on their interaction log counts and the interaction duration in the BookRoll system. Data is then processed for one member of each profile to illustrate the characteristics of that profile in terms of time spent, navigation pattern, and highlighting activity in pages 8 and 12 of the uploaded content. The profiles and their corresponding characteristic value of interactions and time spent aggregated across all their reading sessions are presented in Table 2.

Table 2 Distribution of profiles of readers during the learning activity

Next, we highlight example profiles of time spent and navigation patterns of individual members in the identified groups along with the overall group’s engagement score.

Effortful reader

The aggregated engagement profile of the effortful readers was 648, and the distribution of the individual indicator weights is shown in Fig. 7. From the visualized score, it is seen that the group had higher interactions as well as reading time. The annotating by markers was also high. The only memos that were created during this learning task were from a member of the effortful group.

Fig. 7

Engagement score of effortful reader

Learner 2548 is selected to illustrate an effortful reader. The average page viewing time was 3 min (Fig. 8a). The navigation pattern is presented in Fig. 8b. The blue lines indicate the learner did NEXT on a specific page, and the red indicates PREV. The y-axis has the accumulated count of those transitions. Highlighted portions on pages 8 and 12 by the learner are shown in Fig. 8c and d, respectively.

Fig. 8

a Page-wise viewing duration. b Navigation pattern. c Highlight on pp.8. d Highlight on pp.12

From the email excerpt of the response of one of the engaged readers, it is clear that a specific task, even at an “identify” level, is an important trigger for the students to engage with the content. “This activity was an analytical task for me. It required me to read the drama critically in order to identify the elements of drama.” The student recognized how the activity called forth the critical faculty to distinguish it from the usual reading s/he would have done otherwise and that it helped to move beyond the plot. Specifically, an example of highlighting cultural references was mentioned that one was familiar with, would have gone unnoticed otherwise, became clearer due to the task given: “For instance, as an Indian, the cultural references in the drama don’t really stand out to me, because these references are a part of my culture, and therefore, don’t instantly catch my attention. In this activity, though, since I was specifically looking out for cultural references, I realized how deeply rooted in Hindu mythology Hayavadana is.”

Further, as seen in Fig. 8b, there is a back and forth navigation of this particular student. A reason was provided during the elaboration of the experience “As I flipped the pages of the drama, I sometimes came across some elements of drama which I realized I may have missed out in the previous pages. This made me read some sections again, looking out for different elements, and made me realize that every page had several instances of performative and cultural elements that I could identify. The activity surely helped me gain a critical perspective while reading literature and cinema.”

Strategic reader

The aggregated engagement profile of the strategic readers was 380, and distribution of the individual indicator weights is shown in Fig. 9. The group had a higher completion rate and long event and also attempted the highlighting task.

Fig. 9

Engagement score of strategic reader

Learner 2546 is selected to illustrate a strategic learner. The strategic learner focused on the area of the task (Fig. 10b) and spent more time (average 4 min) across the pages (Fig. 10a). The specific highlighted portions on pages 8 and 12 are shown in Fig. 10c and d, respectively.

Fig. 10

a Page-wise viewing duration. b Navigation pattern. c Highlight on pp.8. d Highlight on pp.12

In the email response of the specific learner, a strategic reader in this case acknowledges the task and its effectiveness. “This activity was a great learning experience for me and contributed a great deal to understanding the text thoroughly. With a task in hand, while reading the text, it helped me focus better on its various elements and increased my attention span towards the same.” The perceived advantage of the task was to focus the attention to the specific elements that were mentioned in the task to identify. The participant further mentioned that “Doing a thorough analysis of the text helped me derive a more profound meaning beyond the story, and it helped in connecting with the message it was trying to convey.” This supports the behavioral indicator where the participant spent an average of 4 min on a page across the sessions and focused on the specific part concerning the reading assignment. There was a perceived advantage to using the platform towards a better outcome of the task “The interactive platform to perform the task kept me engaged throughout and led to a much better outcome.” However, this would require a different instrument to evaluate the outcome performance and validate such perception. Lastly, the activity was regarded as “a great break from the usual way of analyzing the text by reading and merely writing about it and gaining similar insights from the text as from the former approach”. Introducing a different medium of interaction, in this case highlighting to identify, seems to be more motivating and perhaps engaging in the reading itself than the traditional writing-based exercises where the direct interaction with the text that has been read is disconnected.

Wanderer and check-out

The aggregated engagement profile of the wanderer readers was 132, and distribution of the individual indicator weights is shown in the Fig. 11. The group had relatively lower values for each of the indicators too.

Fig. 11

Engagement score of wanderer reader

Learner 2557 is designated as a wanderer reader who navigated through the content (Fig. 12b) and spent time (average 4 min) but did not attend to the task. The navigation pattern is presented in Fig. 12b.

Fig. 12

a Page-wise viewing duration. b Navigation pattern

Learner 2568 is designated as a check-out, who just opened the content once and closed the content without even browsing through any page. Learners in this category had an average weight of ranks below 0.1 for most of the 9 engagement indicators and 0 in the annotation activity showing that that the task was not attempted by any of them also.

Discussions and conclusions

This study investigated the reading behaviors of learners during critical reading tasks executed on an online e-book reader. The learning logs of interactions in the e-book system and the processed data from the learning analytics dashboard were used to define and describe four different reader profiles: effortful, strategic, wanderer, and check-outs. While effortful and strategic readers attempted to complete the critical reading-based highlighting task, the wanderer and checkouts did not attempt it at all. Visualization of the navigation patterns of learners and their engagement score in the task context is presented in the “Results and interpretation” section. It is used to illustrate the characteristics of different profiles. This is also the first attempt to discuss the quantitative account of reading the specific play, Hayavadana, and to our knowledge initiates a systematic inclusion of a learning analytics system in a humanities course. Based on this study findings, we discuss from the more specific aspect related to the particular critical reading task to a more generic aspect of approach of applying learning analytics research in a specific disciplinary domain.

Critical reading of a play: reflections from data

The play Hayavadana is firmly rooted in the Indian tradition and Hindu mythology and yet uses certain subversive tactics to question some of these deep-seated traditions and beliefs. At the performative level, it draws from Yakshagana, a traditional theatrical form practiced in the coastal belt of Karnataka, a state in southern India. It also uses a narrator, a chorus, and a self-reflexive plot within plot strategy thereby bringing in much talked about alienation effect. Before any critical analysis of the play can commence, students are nudged to engage in a critical reading task—which typically is done before the classroom discussions. In this study, the activity was done fully using an online environment. The log data provides reading and annotation behavior of a group of learners interacting with the content of the play for the first time. Even though the readers were uninitiated about the conventions of Yakshagana, from the highlights drawn during their reading, it is evident that they did identify simple directions for performance. Tagging such simple directorial notes as performative elements can be attributed to their unfamiliarity with the more nuanced elements taken from the Yakshagana style. Further, the navigation graph data is in congruence with the above reasoning. It shows that effortful and strategic readers are particularly engaged in these parts of the text. The data also suggests these readers were more clear and accurate in the tagging of cultural references as compared to performative elements. Such data points give a clear indication to the instructor as to where the focus of discussions should be when she goes to the class—whether face to face or online.

Limitations

This was a pilot attempt to understand and share some of the observed reading patterns and discuss possible ways the learner interacted with the task at hand. With the given data collection affordances in BookRoll (see Table 1), we can interpret the interactions as the only learner behaviors. Hence, the design of the learning task becomes very crucial to ground the interpretation of the action. In our case of critical reading, the explicit learner action is to navigate the portion of the text and highlight the identified critical elements with markers. Thus, based on the interaction log analysis, the count and the navigation patterns in a given reading episode has construct validity to interpret the critical reading activity (Winne, 2020). However, some of the collected data remains difficult to interpret, for instance, the wanderers, who spent time within the content without attempting the task (indicated by annotation action), it is not possible to distinguish whether they are coping up with comprehending the text before engaging in the critical analysis or just being off task in the system. Such disengaged or distracted behavior is still difficult to detect in the system.

While consolidating the action logs of 22 learners (44% of the registered participants) generated our dataset, it is still from a smaller sample space to fully comment on critical reading behaviors. This might be primarily due to the fact that the activity was ungraded. Further, in this analysis, we did not consider any explicit learner output apart from the highlights as the artefacts. Given our analysis, the model considers only clickstream interactions; we claim that the profiles generated are only of the readers based on their behavioral trace-based interactions in the system and the time in between the action, and cannot distinguish learners for their critical reading skills yet.

As a future work, there remains further analysis of the data from the pilot study itself. We aim to investigate the quality of the highlighted text by the learner with respect to the instructor’s annotation and further compute inferential statistics for the difference of the profiles identified. These would lead to developing learner models specific to critical reading activities.

Reflection from practitioners’ point of view

Critical reading activity and critical analysis have been two crucial components of this course since its inception. During the years in which the course was being offered in offline mode with the students being in the physical classroom, the activity took different shapes. While the activities around the critical reading task—ranging from reading, synthesizing, and responding to questions through written essays or discussion groups—enabled the instructor to observe the learning patterns, using the BookRoll during the said semester in which classes were suddenly shifted to online mode enabled us to ask various research questions about reading profiles of the students during a critical reading activity.

The main consideration while designing the activity was to make students critically read with an annotation task at hand. The task of making students read the text of the play Hayavadana to identify cultural references and performative elements was well-thought-out. It called forth the students’ critical faculty and their knowledge of the cultural context within which the play grounded itself. The instructor, from her prior experience, had noted that a challenging task at hand makes students alert to the richness of any text, and hence, the annotation task was zeroed in on.

Post activity classroom discussion (in the online mode of classroom) enabled a greater level of interaction as compared to earlier classroom scenarios. One of the main differences the instructor observed was that the percentage of students accessing the text in BookRoll before the classroom discussion was marginally higher than that of earlier scenarios where the instructor would have given them the reading as a homework activity. This was evident as a greater level of participation in the classroom was observed after the BookRoll activity. This was observed in spite of the fact that there was still a large chunk of the class that had not accessed the text through the BookRoll.

Developing profiles of reflective reading and implications for technology design

In earlier works, Binder and Lee (2012) proposed four types of adult readers: unskilled readers, resilient readers, good decoders/poor comprehenders, and skilled reader. Later, Putro and Lee (2018) conducted a latent profile analysis of readers across different modes (printed, online, and social media) and for different purposes (academic and recreational) of reading. They classified low-interest readers, traditional readers, moderate readers, and high-interest readers. Still, specifically for critical reading, previous literature lacks any reader’s profile. We attempted to approach and fill that gap using learning logs and computing broader navigation patterns of different readers.

Reading strategies and comprehension strategies are considered as cognitive action and remedial action respectively and both assist the learners in achieving reading success (Yang, 2006). A technology framework like LEAF is capable of supporting these aspects by collecting learning logs from the e-reader and using learning dashboards to visualize the traces. Recent work (Gibson et al., 2017) focused on data-driven technology-supported feedback for reflective writing. However, for reflective reading activities, such data-informed digital services are still lacking. This study conceptualized using the interaction count and time as indicators of different profiles of readers. Such indicators are often included in LA dashboards (Tan et al., 2016) and can assist the teachers to directly check the visualized data and decide the status of reflective reading behavior of the learner.

At another level, technical support can also be developed to automatically evaluate the highlighting actions of learners and to give them feedback. During the data analysis process, the instructor highlighted the portions of the text for reference. Presenting the instructor’s highlighted part to the learners in the learning dashboard can also assist the learners.

Contributions and future work

This pilot study is part of the overarching research project that aims at developing a data-driven narrative of learner behaviors during reflective tasks in humanities and design courses and then support it with technology (Majumdar et al., 2020). Here, we focus on the context of humanities investigating a well-designed activity plan with technology affordances to collectit with technology traces of learning behaviors and then applying learning analytics techniques to highlight indicators of that specific activity. The collaborative work brings in expertise from the domain of humanities and learning analytics and learning tool design. The future work aims to integrate the reflective activity context data and the learner’s interaction data to build models of the learning process and thereby design possible learning feedback and teaching support.