Using reflections and questioning to engage and challenge online graduate learners in education

Abstract

This paper reports an action research that aimed to examine how the author’s weekly use of reflection and questioning instructional methods affected learners’ learning in an online graduate class at a midwestern public university in the USA. The author employed the asynchronous online discussion on the discussion board in Blackboard for learners to complete course assignments. Specifically, the online instructional approach started with learners’ initial written reflection posts based on the assigned readings and personal experience, then followed by the instructor’s challenging questions based on the Socratic method and learners’ written responses to those questions. This method included discipline-specific questioning, creating a community where learners replied to the instructor’s and learners’ questions through modelling and facilitation, and promoting the instructor’s thinking-encouraging approach. The results of both quantitative and qualitative data indicated that the use of the reflection and questioning methods was effective in engaging and challenging online graduate learners. International implications across the discipline will result from the study.

Introduction

Higher education plays a significant role in producing qualified workers for the workforce. It is well known that teaching is an important process of solidifying the connection between theory and practice in both traditional and online instruction. In recent years, online education has increased so quickly in higher education. According to Wals and Jickling (2002), one of the most important responsibilities in high education is to continuously challenge and critique value and knowledge claims. Further, Wals and Jickling stated “…Higher education has first and foremost something to do with creating possibilities, not defining or prescribing the future for our learners. These possibilities arise when universities promote the exploration, evaluation, and critique of emerging ideas and the creative contribution to their development….” (p. 130).

In online courses, the online discussion has been widely used to encourage learners’ interactive communication. Such an emphasis will not only help learners exchange information, but also enable and promote learners’ reflection of their own understanding and connections between theory and practice or experience, as well as promote peer learning from each other (Gasparič and Pečar 2016). However, according to Stott (2016), many online courses in higher education involving “poor levels of student engagement pose challenges to institutions, instructors and learners” (p. 51). There are various ways to address the poor levels of student engagement in online courses in higher education. However, this study focuses on the examination of the use of reflection and questioning techniques on online graduate learners’ learning.

Reflection

The seminal work of Schon (1987) on reflection-in-action vs reflection-on-action is a very important framework for this paper. He offered a new approach to professional education in several subjects. Specifically, Schon argued that professional education should focus on improving the practitioners’ ability for reflection-in-action. In order to legitimize such new scholarship, Schon (1995) advocated that colleges and universities should be prepared to foster such new forms of reflective action research. According to Yang (2010), “Reflection is considered as a mental process of an individual’s internal problem-solving activity and rarely observed in face-to-face instruction” (p. 1202). However, as some researchers such as Chen et al. (2011) have pointed out, such a reflective process requires the instructor’s help, facilitation, and guidance at the time of learners’ learning. So in the asynchronous learning, the importance of learners’ reflection assignment seems even more difficult to be carried out since online learners have the possibility to learn at any time of the day every week.

The use of reflection in online teaching has been consistently reported to be important and helpful in learning processes for promoting learning performance (Chen et al. 2009; Gasparič and Pečar 2016; Salmon 2002; Smith and Greene 2013). Kusumi et al. (2012) found that reflection about one’s specific activities in a discussion was one of the five instructional methods to effectively promote freshmen’s critical thinking over a semester. Gummesson and Nordmark (2012) indicate that online written self-reflections are a crucial approach of enhancing learning and learning strategies via sharing thoughts, experiences, opinions, and feelings.

Bye et al. (2009) compared the online reflection assignments on the discussion forum and the hard copy reflection assignments. They found that the online learners prefer the online reflection assignment since it allowed them to learn from their classmates and then helped them build trust and willingness to share experiences each other. Other researchers such as Lee (2010) and Muncy (2014) reported that reflective assignment such as online blogging is very helpful for reflective learning online for learners. Online reflective entries/posts have some advantages. They are date stamped and interactive and can be reviewed by the instructor, student, and peers any time in the semester. Larkin and Beatson (2014) studied a teaching innovation including the use of online reflective journals in an undergraduate marketing class. They found that “learners definitely see benefits in the online journals…as reflective tools highlighting the benefit of mixed tools for reflection” (p. 12).

In addition, Makoul et al. (2010) investigated the use of an online forum to encourage reflection about difficult issues in medicine. They found that a structured format for reflection and related feedback are crucial elements of the online learning process about difficult topics. Additionally, Lord et al. (2017) studied effect of the type of online reflection strategy on university juniors’ reflective judgment performance and perceived learning. Lord et al. found that all of the three online reflection techniques they used could equally enhance participants’ reflective judgment practice. Therefore, recent literature has consistently reported that reflection can be used to engage learners and promote online learning across the disciplines.

Questioning

According to Maxwell (2013), the Socratic method can be divided into two types: the classic Socratic method and the modern Socratic method. The classic Socratic method focuses on preparing learners to think and improve themselves through increased understanding of knowledge. The modern Socratic method emphasizes teachers asking well-formed questions and encouraging learners to continue the investigation instead of finding absolute answers.

There are a few approaches in the literature about how to use questioning to engage and challenge learners. Garrison et al. (2000) emphasized the importance of a true community of inquiry (COI) and pointed out that “the tone of the messages is questioning but engaging, expressive but responsive, skeptical but respectful, and challenging but supportive” (p. 96). In higher education, one of the most frequently cited one is the Socratic method of questioning (Delić and Bećirović 2016; Healey 2012). Based on Delić and Bećirović, the Socratic method is “a pedagogy that uses guided questions, dialog, and refutation to help learners critically reflect on their understanding of a particular issue. As learners reflect, the instructor’s questions stimulate them to reject misconceptions and gain an understanding of what they know and also what they do not know.” (p. 516–517). According to Kingsley (2011), asynchronous online discussion is one of the most valuable formats of the Socratic dialog. This format has multiple benefits including, but are not limited to, allowing learners to reflect and ongoing research before responding and allowing multiple learners to reply simultaneously. Additionally, according to Oyler and Romanelli (2014), the Socratic method has been reported as an effective tool for teaching critical thinking in many fields. However, Oyler and Romanelli (2014) pointed out that “the use of Socratic questioning is a lack of evidence-based research related to the use of this method within and across formal educational programs” (p. 6) in various disciplines in higher education.

Based on the Socratic method and Garrison et al. (2000) COI, Golding (2011) further proposed a community of inquiry model. This model includes starting with discipline-specific questioning, creating a community where learners reply to questions through modelling and facilitation, and promoting the instructor’s thinking-encouraging approach. The author employed Golding’s model in his online teaching in the study. Specifically, the instructional approach started with learners’ initial reflection essay posts based on the assigned content and personal experience, as well as an extensive participation in written discussions every week. This process included discipline-specific questioning, creating a community where learners reply to the instructor and learners’ questions through modelling and facilitation, and promoting the instructor’s thinking-encouraging approach.

Research questions

The purpose of this study was to investigate the impact of the weekly use of reflection and questioning instructional methods on learning in an online graduate course in education. The study analyzed related quantitative and qualitative data to examine whether the author’s weekly use of reflection and questioning instructional methods had helped engage and challenge online graduate learners in learning. Two specific research questions guiding the study were:

• 1. RQ1: How did online learners perceive the impact of the instructional strategy of reflection and questioning on their learning?

• 2. RQ2: How did the strategy of reflection and questioning impact online students’ learning outcomes?

Method

Participants

The participants came from one of the author’s online graduate courses entitled performance technology at a midwest public university in the USA in the spring semester of 2015. The course lasted 17 weeks. The class had seven graduate learners and all learners had part-time or full-time jobs. Three were males and four were females. There were six Caucasians and one Black. Their ages varied in the 20s and 30s. They were all graduate learners in the instructional technology program. Some learners worked in the educational setting while others worked in the non-educational settings. The author of this article was the instructor who taught the course and who had taught the same course previously.

Research design and instructional intervention

The research design in the study was individual’s practical action research (Gay et al. 2012). Specifically, the author identified an innovative teaching method—online reflection and questioning to teach an online course, then collected and analyzed various related data, and developed the action plan to implement in the future online teaching.

In the course presented previously, the learning management system used was Blackboard. The major textbook used in this course was Handbook of human performance technology (Pershing 2006). Specifically, learners were required to complete the following three major components involving the reflection and questioning methods in addition to the two individual projects: One was due at midterm and the other one due in the final week. Both individual projects required learners to make connections between theories and practice as well as to apply related theories into hypothetical or real situations.

• 1. Weekly chapter/topic reflection essays (12 total, 12% of course grade). Learners were required to post a brief weekly paper in about 200 words about (a) what major concepts he/she has learned from the readings and (b) what major concepts interest her/him the most and why, online by every Wednesday evening. Learners were also required to respond to the instructor’s questions related to the essays every week.

• 2. Weekly journal article review essays (12 total, 24% of course grade). These essays were designed to broaden and extend learners’ knowledge in human performance technology. Learners were required to locate a peer-reviewed journal article related to each week’s topic published in recent 10 years; summarize the article in 3 parts, key points of the article, personal reflections, and APA citation of the source; and then post the essays online by every Wednesday evening. Learners were also required to respond to the instructor’s questions related to the essays every week.

• 3. Weekly interactions and critiques (12% of course grade). Learners were required to respond to the instructor’s questions and feedback about their above weekly posts and make at least 2 substantive comments in each response to the peers’ posts by Sunday.

The two major instructional methods the author used were reflection and questioning. As stated previously, reflection was required in both type 1 and type 2 assignments above. The questioning method was primarily based on Golding’s (2011) community of inquiry model described previously. Specifically, first, each student was required to read an assigned topic and a refereed journal article each week and write/post a short reflection essay for both type 1 and type 2 assignments respectively. The two essay posts were due on the Blackboard by each Wednesday evening. After that, the author as the instructor read each essay and commented on and graded each essay on the Blackboard on Thursday. In the instructor’s comments to each student, he proposed a series of critical questions related to the initial reflection posts each week. Meantime, learners were required to write and post at least two peer critiques including asking critical questions online by Sunday. The questions proposed by the instructor and the peers stimulated learners’ critical thinking and forced them to answer questions critically and deeply every week.

Data collection

There were three different data points of collections. First, the author voluntarily invited a peer consultant from a faculty volunteer group sponsored by the Provost office at the author’s university to conduct a Group Instructional Feedback Technique (GIFT) survey before the mid-point of the semester. The consultant conducted an online focus group interview with the learners to collect qualitative data, then presented a summary report to the author as the instructor of the course. Second, the author conducted a mid-point course evaluation survey including three open-ended questions to collect qualitative data. The three questions were (1) “What have you learned so far?,” (2) “What have you liked so far?,” and (3) “Any suggestions for instructional improvement in the rest of the semester?” He then presented a summary of the midpoint survey for sharing in Blackboard. Third, the author’s department conducted an online end-of-the-semester course evaluation survey to collect both quantitative and qualitative data. The survey included 11 objective items on a 5-point Likert scale, with 5 being the highest or the best. In addition, the survey also contained three open-ended questions. The above related results are discussed in the next section.

Results

The results in all three types of data mentioned previously were consistent. That is, overall, learners were surprisingly very positive about the online innovative reflection and questioning instructional methods used. The innovative instructional intervention used in the study was effective.

First, the qualitative results from the GIFT survey were overall positive, and no major suggestions for improvement were offered. Only one suggestion was offered, that is, adding a Collaborate session with the instructor. Based on that suggestion, a Collaborate session in Blackboard was offered by the instructor after the mid-point in the semester, but no online learners showed up.

Second, the qualitative data from the midpoint survey involved the thematic analysis, the most common form of analysis in qualitative research (Gay et al. 2012). Its results showed that all feedback was positive and no major suggestion was offered. So the author did not change any instructional strategies in the second half of the semester.

Third, the results from the departmental course evaluation are used for major discussions here. Five of the seven learners completed the online departmental evaluation survey. The quantitative data was analyzed using Statistical Package for the Social Science (SPSS) 20.0 and indicated that the mean for each of the 11 Likert scale items was 5. The qualitative comments in those three open-ended questions involved the thematic analysis (Gay et al. 2012). The qualitative results indicated that most comments were very positive. About questions #1—“What factors in this class enhanced your learning”, most learners liked the weekly readings, reflection and connection essays, questions by the instructor, and course projects. About question #2—“What factors in this class inhibited your learning”, a majority of learners did not think that there were any factors inhibiting their learning. One student did think that teamwork could help with his/her learning. About question #3—“Please write any comments that you would like to make about the course and/or the Instructor”, all five comments were positive.

Further, a careful analysis of the learners’ qualitative data in the departmental course evaluation indicated three major themes below.

First, learners worked hard to write and post topic reflection essays online based on the topic and question prompts by the deadline. A majority of learners’ topic reflection posts were of high quality and received a grade of “A” in all 12 posts based on the assessment rubric. Learners’ comments in the course evaluation included “I really enjoyed the chapter reading and reflections each week….”

Second, learners worked hard to write and post the journal article reflection essays online based on the topic and question prompts by the deadline. It is clear that learners enjoyed this assignment and were motivated to find articles related to their own interests and to make connections between the article and their own experience or observations accordingly. A majority of learners’ article review essays were of high quality and received a grade of “A” in all 12 posts based on the assessment rubric. Learners’ comments in the course evaluation included “What I thought was great about this course was when I make so many connections between the reading, article reflections, and discussion boards. I got extremely interested in the subject, because I started to visualize myself as a HPT.” “I think the weekly article assignment really helped me learn the concepts of the course in an effective context. Mostly, I chose articles that are based in the corporate field. It was Interesting to see how those concepts work, and interact with my peers to discuss other ideas.”

Third, learners enjoyed being challenged to work hard to address and replied to the content-specific challenging questions from the instructor and the peers accordingly every week. Generally, they exhibited higher order thinking in their peer reviewing and critiquing every week using reflection, questioning, organization, applications, connections, and synthesis. In addition, they engaged in critical reflections of their own work and work of their peers by giving and receiving positive and negative feedback that leads to furthering their thinking in the field. A majority of learners’ responses to the instructor’s and peers’ questions every week were of high quality and detailed and received a grade of “A” in all 12 weeks based on the assessment rubric. Learners’ comments in the course evaluation included “The questions posed by Dr. L in response to our weekly reflections were challenging and enhanced my learning.” “I find myself being challenged in Dr. L’s courses, but not by the material, but myself, I am seeing growth and courage ooze out, and I am starting to qet excited about my future as a Instructional Designer and Performance.”

However, one learner did offer one suggestion—“Maybe in the future incorporate some type of group projects….” This suggestion has advantages and disadvantages. Some learners liked group project while others disliked it. In fact, at the beginning of the course, the instructor asked learners individually whether they would like to complete the two required projects in groups. Only one learner expressed to work in a group. That was why learners were not assigned to work on the two projects in groups. Anyway, learners really enjoyed the two individual projects as well. Learners’ comments in the course evaluation included “I especially enjoyed the project 2 assignment. Not until I started my literature review that I noticed how much HPT knowledge I had retained. Having to compile multiple lit reviews over the semester, combining my thoughts for project to become much easier.” “I also thought both projects [I]ntroduced me to two different ideas and concepts that I have not heard of until this course.”

Discussion

As indicated in the “Results” section, both research questions were clearly answered. That is, first, online learners favorably perceived the impact of the reflection and questioning instructional strategy on their learning; second, the strategy of reflection and questioning positively and actively impacted online students’ learning outcomes. The following will focus on the impacts of reflection and questioning on learners’ online learning.

Impact of reflection

As stated previously, this study found that reflection assignments were effective to engage online learners and to promote their learning. This finding was consistent with some previous results. Burhan-Horasanlı and Ortaçtepe (2016) conducted a qualitative study of in-service teachers. They found that the online reflective practice-oriented online discussion is an effective device to help teachers collaboratively reflect on their own and peers’ beliefs and practices. Additionally, Guldberg and Pilkington (2007) studied the impact of teaching and learning in a blended online course which emphasized learners’ reflection and transformation of their own practice. They found that (1) the questions better at enabling dialog and reciprocity included those asking learners to reflect on and connect to personal experiences and express their own opinions and (2) the questions providing learners opportunities to comment on their own experiences led to more monolog exchanges.

Impact of questioning

This study also found that the type of questions the instructor asked the learners every week has crucial effects on subsequent discussions and directly leads to certain types of student contributions and responses. That is, the instructor’s questions focused on how to stimulate learners’ critical thinking and the connections between theory and practice in the real world. This is also consistent with some previous findings. Wang (2005) studied the impact of using different questioning skills on pre-service teacher education learners in an online educational technology course. He found that good use of divergent thinking questions promotes healthy online synchronous discussion. In addition, open-ended questions have advantages. These include eliciting various perspectives by enhancing student participation, helpful to assess student thinking ability, as well as to develop thinking and responding abilities. However, Cho et al. (2011) found that the deep reasoning questions and knowledge integration responses had no significant impact for learning outcomes in undergraduate educational learners since participants were homogenous with similar age and educational backgrounds.

Limitations and implications for future research

Similar to other studies, this study has its limitations too. The study only focused on one instructor in a small online graduate course among a group of homogenous group in education. It primarily focused on the learners’ perception of the impacts of online student reflection and the instructor’s questioning that benefited individual and group online learning. Therefore, further research is needed to better understand the long-term impact of online reflection and questioning instructional techniques across the disciplines.in the world.

In addition, the definitions and various issues related to reflection and questioning instructional methods in online courses could be discussed further. Critical reflection refers to a much deeper and wider view of learning with its roots in critical theory (Reynolds 1998). Thus, further critical reflective studies should also be done in the future. Meantime, issues related to this study in the future should include, but are not limited to, operationalizations of reflection and questioning instructional methods, across the instructors, across the disciplines, undergraduate and graduate courses, large classes, and a diversified learner group. In addition, how the author’s subsequent research in the future could be built upon the related seminal work of other researchers should be further explored to better situate reflection in the online learning process. Those research frameworks could include, but are not limited to, Rothstein et al.’s (2015) work on the Question Formulation Technique and the STARL-P Framework (Situation, Task, Action, Result/ Reflection, and Learning/Planning) (Queensland University of Technology 2011).

Conclusion

Having used the reflection and questioning instructional methods and witnessed their positive effects described previously for a few years, the author likes this approach more and more due to several reasons. First, it engages learners actively and deeply in the discipline. The questions force learners to go beyond traditional thinking and to think deeply and critically for alternative possibilities. Second, it is a rewarding experience as an instructor and a researcher to see how learners have grown professionally by being intellectually challenged in the class all the time.

The results of student evaluation indicated that the use of reflections and questioning instructional methods was very effective in engaging and challenging online graduate learners. The results are consistent with other recent research and further indicated this innovative instructional technique was effective for online courses. International implications for online college teaching across the discipline will result from the study. First, the practical impacts of reflection and questioning instruction on online students’ learning have been supported by the literature in some subjects. Second, all subjects in higher education in the world could benefit from reflection and questioning since this type of online instruction targets students’ deep thinking and higher order cognitive functioning. However, care should be taken when generalizing the results to other situations since this study was limited to one small graduate online course in education.