Enhancing achievement and interest in mathematics learning through Math-Island

Abstract

Conventional teacher-led instruction remains dominant in most elementary mathematics classrooms in Taiwan. Under such instruction, the teacher can rarely take care of all students. Many students may then continue to fall behind the standard of mathematics achievement and lose their interest in mathematics; they eventually give up on learning mathematics. In fact, students in Taiwan generally have lower interest in learning mathematics compared to many other regions/countries. Thus, how to enhance students’ mathematics achievement and interest are two major problems, especially for those low-achieving students. This paper describes how we designed a game-based learning environment, called Math-Island, by incorporating the mechanisms of a construction management game into the knowledge map of the elementary mathematics curriculum. We also report an experiment conducted with 215 elementary students for 2 years, from grade 2 to grade 3. In this experiment, in addition to teacher-led instruction in the classroom, students were directed to learn with Math-Island by using their own tablets at school and at home. As a result of this experiment, we found that there is an increase in students’ mathematics achievement, especially in the calculation and word problems. Moreover, the achievements of low-achieving students in the experimental school outperformed the low-achieving students in the control school (a control group in another school) in word problems. Moreover, both the low-achieving students and the high-achieving students in the experimental school maintained a rather high level of interest in mathematics and in the system.

Introduction

Mathematics has been regarded as a fundamental subject because arithmetic and logical reasoning are the basis of science and technology. For this reason, educational authorities emphasize students’ proficiency in computational skills and problem-solving. Recently, the results of the Program for International Student Assessment (PISA) and the Trends in Mathematics and Science Study (TIMSS) in 2015 (OECD 2016; Mullis et al. 2016) revealed a challenge for Taiwan. Although Taiwanese students had higher average performance in mathematics literacy compared to students in other countries, there was still a significant percentage of low-achieving students in Taiwan. Additionally, most Taiwanese students show low levels of interest and confidence in learning mathematics (Lee 2012).

The existence of a significant percentage of low-achieving students is probably due to teacher-led instruction, which still dominates mathematics classrooms in most Asian countries. It should be noted that students in every classroom possess different abilities and hence demonstrate different achievements. Unfortunately, in teacher-led instruction, all the students are required to learn from the teacher in the same way at the same pace (Hwang et al. 2012). Low-achieving students, without sufficient time, are forced to receive knowledge passively. Barr and Tagg (1995) pointed out that it is urgent for low-achieving students to have more opportunities to learn mathematics at their own pace. Researchers suggested one-to-one technology (Chan et al. 2006) through which every student is equipped with a device to learn in school or at home seamlessly. Furthermore, they can receive immediate feedback from Math-Island, which supports their individualized learning actively and productively. Thus, this may provide more opportunities for helping low-achieving students improve their achievement.

The low-interest problem for almost all students in Taiwan is usually accompanied by low motivation (Krapp 1999). Furthermore, students with continuously low performance in mathematics may eventually lose their interest and refuse to learn further (Schraw et al. 2001). This is a severe problem. To motivate students to learn, researchers design educational games to provide enjoyable and engaging learning experiences (Kiili and Ketamo 2007). Some of these researchers found that game-based learning may facilitate students’ learning in terms of motivation and learning effects (Liu and Chu 2010), spatial abilities and attention (Barlett et al. 2009), situated learning, and problem-solving (Li and Tsai 2013). Given these positive results, we hope that our educational game can enhance and sustain the student’s interest in learning mathematics.

In fact, many researchers who endeavored to develop educational games for learning mathematics have shown that their games could facilitate mathematics performance, enjoyment, and self-efficacy (Ku et al. 2014; McLaren et al. 2017). Although some of the studies were conducted for as many as 4 months (e.g., Hanus and Fox 2015), one may still criticize them for the possibility that the students’ interest could be a novelty effect—meaning their interest will decrease as the feeling of novelty diminishes over time (Koivisto and Hamari 2014). Due to the limitations of either experimental time or sample sizes, most studies could not effectively exclude the novelty effect of games, unless they were conducted in a natural setting for a long time.

In this study, we collaborated with an experimental elementary school for more than 2 years. The mathematics teachers in the school adopted our online educational game, Math-Island. The students used their own tablet PCs to learn mathematics from the game in class or at home at their own pace. In particular, low-achieving students might have a chance to catch up with the other students and start to feel interested in learning mathematics. Most importantly, because the online educational game was a part of the mathematics curriculum, the students could treat the game as their ordinary learning materials like textbooks. In this paper, we reported a 2-year study, in which 215 second graders in the school adopted the Math-Island game in their daily routine. More specifically, the purpose of this paper was to investigate the effect of the game on students’ mathematics achievement. Additionally, we were also concerned about how well the low-achieving students learned, whether they were interested in mathematics and the game, and how their interest in mathematics compared with that of high-achieving students. In such a long-term study with a large sample size, it was expected that the novelty effect would be considerably reduced, allowing us to evaluate the effect of the educational game on students’ achievement and interest.

The paper is organized as follows. In the “Related works” section, we review related studies on computer-supported mathematics learning and educational games. In the “Design” section, the game mechanism and the system design are presented. In the “Method” section, we describe the research method and the procedures of this study. In the “Results” section, the research results about students’ achievement and interest are presented. In the “Discussion on some features of this study” section, we discuss the long-term study, knowledge map design, and the two game mechanisms. Finally, the summary of the current situation and potential future work is described in the “Conclusion and future work” section.

Related works

Computer-supported mathematics learning

The mathematics curriculum in elementary schools basically includes conceptual understanding, procedural fluency, and strategic competence in terms of mathematical proficiency (see Kilpatrick et al. 2001). First, conceptual understanding refers to students’ comprehension of mathematical concepts and the relationships between concepts. Researchers have designed various computer-based scaffolds and feedback to build students’ concepts and clarify potential misconceptions. For example, for guiding students’ discovery of the patterns of concepts, Yang et al. (2012) adopted an inductive discovery learning approach to design online learning materials in which students were provided with similar examples with a critical attribute of the concept varied. McLaren et al. (2017) provided students with prompts to correct their common misconceptions about decimals. They conducted a study with the game adopted as a replacement for seven lessons of regular mathematics classes. Their results showed that the educational game could facilitate better learning performance and enjoyment than a conventional instructional approach.

Second, procedural fluency refers to the skill in carrying out calculations correctly and efficiently. For improving procedural fluency, students need to have knowledge of calculation rules (e.g., place values) and practice the procedure without mistakes. Researchers developed various digital games to overcome the boredom of practice. For example, Chen et al. (2012a, 2012b) designed a Cross Number Puzzle game for practicing arithmetic expressions. In the game, students could individually or collaboratively solve a puzzle, which involved extensive calculation. Their study showed that the low-ability students in the collaborative condition made the most improvement in calculation skills. Ku et al. (2014) developed mini-games to train students’ mental calculation ability. They showed that the mini-games could not only improve students’ calculation performance but also increase their confidence in mathematics.

Third, strategic competence refers to mathematical problem-solving ability, in particular, word problem-solving in elementary education. Some researchers developed multilevel computer-based scaffolds to help students translate word problems to equations step by step (e.g., González-Calero et al. 2014), while other researchers noticed the problem of over-scaffolding. Specifically, students could be too scaffolded and have little space to develop their abilities. To avoid this situation, many researchers proposed allowing students to seek help during word problem-solving (Chase and Abrahamson 2015; Roll et al. 2014). For example, Cheng et al. (2015) designed a Scaffolding Seeking system to encourage elementary students to solve word problems by themselves by expressing their thinking first, instead of receiving and potentially abusing scaffolds.

Digital educational games for mathematics learning

Because mathematics is an abstract subject, elementary students easily lose interest in it, especially low-achieving students. Some researchers tailored educational games for learning a specific set of mathematical knowledge (e.g., the Decimal Points game; McLaren et al. 2017), so that students could be motivated to learn mathematics. However, if our purpose was to support a complete mathematics curriculum for elementary schools, it seemed impractical to design various educational games for all kinds of knowledge. A feasible approach is to adopt a gamified content structure to reorganize all learning materials. For example, inspired by the design of most role-playing games, Chen et al. (2012a, 2012b) proposed a three-tiered framework of game-based learning—a game world, quests, and learning materials—for supporting elementary students’ enjoyment and goal setting in mathematics learning. Furthermore, while a game world may facilitate students’ exploration and participation, quests are the containers of learning materials with specific goals and rewards. In the game world, students receive quests from nonplayer virtual characters, who may enhance social commitments. To complete the quests, students have to make efforts to undertake learning materials. Today, quests have been widely adopted in the design of educational games (e.g., Azevedo et al. 2012; Hwang et al. 2015).

However, in educational games with quests, students still play the role of receivers rather than active learners. To facilitate elementary students’ initiative, Lao et al. (2017) designed digital learning contracts, which required students to set weekly learning goals at the beginning of a week and checked whether they achieved the goals at the end of the week. More specifically, when setting weekly goals, students had to decide on the quantity of learning materials that they wanted to undertake in the coming week. Furthermore, they also had to decide the average correctness of the tests that followed the learning materials. To help them set reasonable and feasible goals, the system provided statistics from the past 4 weeks. As a result, the students may reflect on how well they learned and then make appropriate decisions. After setting goals, students are provided with a series of learning materials for attempting to accomplish those goals. At the end of the week, they may reflect on whether they achieved their learning goals in the contracts. In a sense, learning contracts may not only strengthen the sense of commitment but also empower students to take more control of their learning.

In textbooks or classrooms, learning is usually predefined as a specific sequence, which students must follow to learn. Nevertheless, the structure of knowledge is not linear, but a network. If we could reorganize these learning materials according to the structure of knowledge, students could explore knowledge and discover the relationships among different pieces of knowledge when learning (Davenport and Prusak 2000). Knowledge mapping has the advantage of providing students concrete content through explicit knowledge graphics (Ebener et al. 2006). Previous studies have shown that the incorporation of knowledge structures into educational games could effectively enhance students’ achievement without affecting their motivation and self-efficacy (Chu et al. 2015). For this reason, this study attempted to visualize the structure of knowledge in an educational game. In other words, a knowledge map was visualized and gamified so that students could make decisions to construct their own knowledge map in games.

Design

To enhance students’ mathematics achievement and interests, we designed the Math-Island online game by incorporating a gamified knowledge map of the elementary mathematics curriculum. More specifically, we adopt the mechanisms of a construction management game, in which every student owns a virtual island (a city) and plays the role of the mayor. The goal of the game is to build their cities on the islands by learning mathematics.

System architecture

The Math-Island game is a Web application, supporting cross-device interactions among students, teachers, and the mathematics content structure. The system architecture of the Math-Island is shown in Fig. 1. The pedagogical knowledge and learning materials are stored in the module of digital learning content, organized by a mathematical knowledge map. The students’ portfolios about interactions and works are stored in the portfolio database and the status database. When a student chooses a goal concept in the knowledge map, the corresponding digital learning content is arranged and delivered to his/her browser. Besides, when the student is learning in the Math-Island, the feedback module provides immediate feedback (e.g., hints or scaffolded solutions) for guidance and grants rewards for encouragement. The learning results can also be shared with other classmates by the interaction module. In addition to students, their teachers can also access the databases for the students’ learning information. Furthermore, the information consists of the students’ status (e.g., learning performance or virtual achievement in the game) and processes (e.g., their personal learning logs). In the Math-Island, it is expected that students can manage their learning and monitor the learning results by the construction management mechanism. In the meantime, teachers can also trace students’ learning logs, diagnose their weaknesses from portfolio analysis, and assign students with specific tasks to improve their mathematics learning.

Fig. 1

The system architecture of Math-Island

Knowledge map

To increase students’ mathematics achievement, the Math-Island game targets the complete mathematics curriculum of elementary schools in Taiwan, which mainly contains the four domains: numerical operation, quantity and measure, geometry, and statistics and probability (Ministry of Education of R.O.C. 2003). Furthermore, every domain consists of several subdomains with corresponding concepts. For instance, the domain of numerical operation contains four subdomains: numbers, addition, and subtraction for the first and second graders. In the subdomain of subtraction, there are a series of concepts, including the meaning of subtraction, one-digit subtraction, and two-digit subtraction. These concepts should be learned consecutively. In the Math-Island system, the curriculum is restructured as a knowledge map, so that they may preview the whole structure of knowledge, recall what they have learned, and realize what they will learn.

More specifically, the Math-Island system uses the representational metaphor of an “island,” where a virtual city is located and represents the knowledge map. Furthermore, the island comprises areas, roads, and buildings, which are the embodiments of domains, subdomains, and concepts in the curriculum, respectively. As shown in Fig. 2, for example, in an area of numeral operation in Math-Island, there are many roads, such as an addition road and a subtraction road. On the addition road, the first building should be the meaning of addition, followed by the buildings of one-digit addition and then two-digit addition. Students can choose these buildings to learn mathematical concepts. In each building, the system provides a series of learning tasks for learning the specific concept. Currently, Math-Island provides elementary students with more than 1300 learning tasks from the first grade to the sixth grade, with more than 25,000 questions in the tasks.

Fig. 2

The knowledge map

In Math-Island, a learning task is an interactive page turner, including video clips and interactive exercises for conceptual understanding, calculation, and word problem-solving. In each task, the learning procedure mainly consists of three steps: watching demonstrations, practicing examples, and getting rewards. First, students learn a mathematical concept by watching videos, in which a human tutor demonstrates examples, explains the rationale, and provides instructions. Second, students follow the instructions to answer a series of questions related to the examples in the videos. When answering questions, students are provided with immediate feedback. Furthermore, if students input wrong answers, the system provides multilevel hints so that they could figure out solutions by themselves. Finally, after completing learning tasks, students receive virtual money according to their accuracy rates in the tasks. The virtual money is used to purchase unique buildings to develop their islands in the game.

Game mechanisms

In the Math-Island game, there are two game mechanisms: construction and sightseeing (as shown in Fig. 3). The former is designed to help students manage their learning process, whereas the latter is designed to facilitate social interaction, which may further motivate students to better develop their cities. By doing so, the Math-Island can be regarded as one’s learning portfolio, which is a complete record that purposely collects information about one’s learning processes and outcomes (Arter and Spandel 2005). Furthermore, learning portfolios are a valuable research tool for gaining an understanding about personal accomplishments (Birgin and Baki 2007), because learning portfolios can display one’s learning process, attitude, and growth after learning (Lin and Tsai 2001). The appearance of the island reflects what students have learned and have not learned from the knowledge map. When students observe their learning status in an interesting way, they may be concerned about their learning status with the enhanced awareness of their learning portfolios. By keeping all activity processes, students can reflect on their efforts, growth, and achievements. In a sense, with the game mechanisms, the knowledge map can be regarded as a manipulatable open learner model, which not only represents students’ learning status but also invites students to improve it (Vélez et al. 2009).

Fig. 3

Two game mechanisms for Math-Island

First, the construction mechanism allows students to plan and manage their cities by constructing and upgrading buildings. To do so, they have to decide which buildings they want to construct or upgrade. Then, they are required to complete corresponding learning tasks in the building to determine which levels of buildings they can construct. As shown in Fig. 4, the levels of buildings depend on the completeness of a certain concept, compared with the thresholds. For example, when students complete one third of the learning tasks, the first level of a building is constructed. Later, when they complete two thirds of the tasks, the building is upgraded to the second level. After completing all the tasks in a building, they also complete the final level and are allowed to construct the next building on the road. Conversely, if students failed the lowest level of the threshold, they might need to watch the video and/or do the learning tasks again. By doing so, students can make their plans to construct the buildings at their own pace. When students manage their cities, they actually attempt to improve their learning status. In other words, the construction mechanism offers an alternative way to guide students to regulate their learning efforts.

Fig. 4

Screenshots of construction and sightseeing mechanisms in Math-Island

Second, the sightseeing mechanism provides students with a social stage to show other students how well their Math-Islands have been built. This mechanism is implemented as a public space, where other students play the role of tourists who visit Math-Island. In other words, this sightseeing mechanism harnesses social interaction to improve individual learning. As shown in Fig. 4, because students can construct different areas or roads, their islands may have different appearances. When students visit a well-developed Math-Island, they might have a positive impression, which may facilitate their self-reflection. Accordingly, they may be willing to expend more effort to improve their island. On the other hand, the student who owns the island may also be encouraged to develop their island better. Furthermore, when students see that they have a completely constructed building on a road, they may perceive that they are good at these concepts. Conversely, if their buildings are small, the students may realize their weaknesses or difficulties in these concepts. Accordingly, they may be willing to make more effort for improvement. On the other hand, the student who owns the island may also be encouraged to develop their island better. In a word, the visualization may play the role of stimulators, so that students may be motivated to improve their learning status.

Method

This paper reported a 2-year study in which the Math-Island system was adopted in an elementary school. The study addressed the following two research questions: (1) Did the Math-Island system facilitate students’ mathematics achievement in terms of conceptual understanding, calculating, and word problem-solving? In particular, how was the mathematics achievement of the low-achieving students? (2) What was students’ levels of interest in mathematics and the system, particularly that of low-achieving students?

Participants

The study, conducted from June 2013 to June 2015, included 215 second graders (98 females and 117 males), whose average age was 8 years old, in an elementary school located in a suburban region of a northern city in Taiwan. The school had collaborated with our research team for more than 2 years and was thus chosen as an experimental school for this study. In this school, approximately one third of the students came from families with a low or middle level of socioeconomic status. It was expected that the lessons learned from this study could be applicable to other schools with similar student populations in the future. The parents were supportive of this program and willing to provide personal tablets for their children (Liao et al. 2017). By doing so, the students in the experimental school were able to use their tablets to access the Math-Island system as a learning tool at both school and home. To compare the students’ mathematics achievement with a baseline, this study also included 125 second graders (63 females and 62 males) from another school with similar socioeconomic backgrounds in the same region of the city as a control school. The students in the control school received only conventional mathematics instruction without using the Math-Island system during the 2-year period.

Procedure

Before the first semester, a 3-week training workshop was conducted to familiarize the students with the basic operation of tablets and the Math-Island system. By doing so, it was ensured that all participants had similar prerequisite skills. The procedure of this study was illustrated in Table 1. At the beginning of the first semester, a mathematics achievement assessment was conducted as a pretest in both the experimental and the control school to examine the students’ initial mathematics ability as second graders. From June 2013 to June 2015, while the students in the control school learned mathematics in a conventional way, the students in the experimental school learned mathematics not only in mathematics classes but also through the Math-Island system. Although the teachers in the experimental school mainly adopted lectures in mathematics classes, they used the Math-Island system as learning materials at school and for homework. At the same time, they allowed the students to explore the knowledge map at their own pace. During the 2 years, every student completed 286.78 learning tasks on average, and each task took them 8.86 min. Given that there were 344 tasks for the second and third graders, the students could finish 83.37% of tasks according to the standard progress. The data also showed that the average correctness rate of the students was 85.75%. At the end of the second year, another mathematics achievement assessment was administered as a posttest in both schools to evaluate students’ mathematics ability as third graders. Additionally, an interest questionnaire was employed in the experimental school to collect the students’ perceptions of mathematics and the Math-Island system. To understand the teachers’ opinions of how they feel about the students using the system, interviews with the teachers in the experimental school were also conducted.

Table 1 The experimental procedure (from June 2013 to June 2015)

Data collection

Mathematics achievement assessment

To evaluate the students’ mathematics ability, this study adopted a standardized achievement assessment of mathematics ability (Lin et al. 2009), which was developed from a random sample of elementary students from different counties in Taiwan to serve as a norm with appropriate reliability (the internal consistency was 0.85, and the test-retest reliability was 0.86) and validity (the correlation by domain experts in content validity was 0.92, and the concurrent validity was 0.75). As a pretest, the assessment of the second graders consisted of 50 items, including conceptual understanding (23 items), calculating (18 items), and word problem-solving (9 items). As a posttest, the assessment of the third graders consisted of 60 items, including conceptual understanding (18 items), calculating (27 items), and word problem-solving (15 items). The scores of the test ranged from 0 to 50 points. Because some students were absent during the test, this study obtained 209 valid tests from the experimental school and 125 tests from the control school.

Interest questionnaire

The interest questionnaire comprised two parts: students’ interest in mathematics and the Math-Island system. Regarding the first part, this study adopted items from a mathematics questionnaire of PISA and TIMSS 2012 (OECD 2013; Mullis et al. 2012), the reliability of which was sound. This part included three dimensions: attitude (14 items, Cronbach’s alpha = .83), initiative (17 items, Cronbach’s alpha = .82), and confidence (14 items Cronbach’s alpha = .72). Furthermore, the dimension of attitude was used to assess the tendency of students’ view on mathematics. For example, a sample item of attitudes was “I am interested in learning mathematics.” The dimension of initiatives was used to assess how students were willing to learn mathematics actively. A sample item of initiatives was “I keep studying until I understand mathematics materials.” The dimension of confidences was used to assess students’ perceived mathematics abilities. A sample item was “I am confident about calculating whole numbers such as 3 + 5 × 4.” These items were translated to Chinese for this study. Regarding the second part, this study adopted self-made items to assess students’ motivations for using the Math-Island system. This part included two dimensions: attraction (8 items) and satisfaction (5 items). The dimension of attraction was used to assess how well the system could attract students’ attention. A sample item was “I feel Math-island is very appealing to me.” The dimension of satisfaction was used to assess how the students felt after using the system. A sample item was “I felt that upgrading the buildings in my Math-Island brought me much happiness.” These items were assessed according to a 4-point Likert scale, ranging from “strongly disagreed (1),” “disagreed (2),” “agreed (3),” and “strongly agreed (4)” in this questionnaire. Due to the absences of several students on the day the questionnaire was administered, there were only 207 valid questionnaires in this study.

Teacher interview

This study also included teachers’ perspectives on how the students used the Math-Island system to learn mathematics in the experimental school. This part of the study adopted semistructured interviews of eight teachers, which comprised the following three main questions: (a) Do you have any notable stories about students using the Math-Island system? (b) Regarding Math-Island, what are your teaching experiences that can be shared with other teachers? (c) Do you have any suggestions for the Math-Island system? The interview was recorded and transcribed verbatim. The transcripts were coded and categorized according to the five dimensions of the questionnaire (i.e., the attitude, initiative, and confidence about mathematics, as well as the attraction and satisfaction with the system) as additional evidence of the students’ interest in the experimental school.

Data analysis

For the first research question, this study conducted a multivariate analysis of variance (MANOVA) with the schools as a between-subject variable and the students’ scores (conceptual understanding, calculating, and word problem-solving) in the pre/posttests as dependent variables. Moreover, this study also conducted a MANOVA to compare the low-achieving students from both schools. In addition, the tests were also carried out to compare achievements with the norm (Lin et al. 2009). For the second research question, several z tests were used to examine how the interests of the low-achieving students were distributed compared with the whole sample. Teachers’ interviews were also adopted to support the results of the questionnaire.

Results

Mathematics achievement

To examine the homogeneity of the students in both schools in the first year, the MANOVA of the pretest was conducted. The results, as shown in Table 2, indicated that there were no significant differences in their initial mathematics achievements in terms of conceptual understanding, calculating, and word problem-solving (Wilks’ λ = 0.982, F(3330) = 2.034, p > 0.05). In other words, the students of both schools had similar mathematics abilities at the time of the first mathematics achievement assessment and could be fairly compared.

Table 2 The results of the pretest (the second graders)

At the end of the fourth grade, the students of both schools received the posttest, the results of which were examined by a MANOVA. As shown in Table 3, the effect of the posttest on students’ mathematics achievement was significant (Wilks’ λ = 0.946, p < 0.05). The results suggested that the students who used Math-Island for 2 years had better mathematics abilities than those who did not. The analysis further revealed that the univariate effects on calculating and word problem-solving were significant, but the effect on conceptual understanding was insignificant. The results indicated that the students in the experimental school outperformed their counterparts in terms of the procedure and application of arithmetic. The reason may be that the system provided students with more opportunities to do calculation exercises and word problems, and the students were more willing to do these exercises in a game-based environment. Furthermore, they were engaged in solving various exercises with the support of immediate feedback until they passed the requirements of every building in their Math-Island. However, the students learned mathematical concepts mainly by watching videos in the system, which provided only demonstrations like lectures in conventional classrooms. For this reason, the effect of the system on conceptual understanding was similar to that of teachers’ conventional instruction.

Table 3 The results of the posttest (the fourth graders)

Furthermore, to examine the differences between the low-achieving students in both schools, another MANOVA was also conducted on the pretest and the posttest. The pretest results indicated that there were no significant differences in their initial mathematics achievement in terms of conceptual understanding, calculating, and word problem-solving (Wilks’ λ = 0.943, F(3110) = 2.210, p > 0.05).

The MANOVA analysis of the posttest is shown in Table 4. The results showed that the effect of the system on the mathematics achievement of low-achieving students was significant (Wilks’ λ = 0.934, p < 0.05). The analysis further revealed that only the univariate effect on word problem-solving was significant. The results suggested that the low-achieving students who used Math-Island for 2 years had better word problem-solving ability than those students in the control school, but the effect on conceptual understanding and procedural fluency was insignificant. The results indicated that the Math-Island system could effectively enhance low-achieving students’ ability to solve word problems.

Table 4 The posttest results of the low-achieving students (the second graders)

Because the mathematics achievement assessment was a standardized achievement assessment (Lin et al. 2009), the research team did a further analysis of the assessments by comparing the results with the norm. In the pretest, the average score of the control school was the percentile rank of a score (PR) 55, but their average score surprisingly decreased to PR 34 in the posttest. The results confirmed the fact that conventional mathematics teaching in Taiwan might result in an M-shape distribution, suggesting that low-achieving students required additional learning resources. Conversely, the average score of the experimental school was PR 48 in the pretest, and their score slightly decreased to PR 44 in the posttest. Overall, both PR values were decreasing, because the mathematics curriculum became more and more difficult from the second grade to the fourth grade. However, it should be noted that the experimental school has been less affected, resulting in a significant difference compared with the control school (see Table 5). Notably, the average score of word problem-solving in the posttest of the experimental school was PR 64, which was significantly higher than the nationwide norm (z = 20.8, p < .05). The results were consistent with the univariate effect of the MANOVA on word problem-solving, suggesting that the Math-Island system could help students learn to complete word problems better. This may be because the learning tasks in Math-Island provided students with adequate explanations for various types of word problems and provided feedback for exercises.

Table 5 The result of the interest questionnaire

Interest

To examine whether the low-achieving students had low levels of interest in mathematics and the Math-Island system, the study adopted z tests on the data of the interest questionnaire. Table 5 shows the descriptive statistics and the results of the z tests. Regarding the interest in mathematics, the analysis showed that the interest of the low-achieving students was similar to that of the whole sample in terms of attitude, initiative, and confidence. The results were different from previous studies asserting that low-achieving students tended to have lower levels of interest in mathematics (Al-Zoubi and Younes 2015). The reason was perhaps that the low-achieving students were comparably motivated to learn mathematics in the Math-Island system. As a result, a teacher (#T-301) said, “some students would like to go to Math-Island after school, and a handful of students could even complete up to forty tasks (in a day),” implying that the students had a positive attitude and initiative related to learning mathematics.

Another teacher (T-312) also indicated “some students who were frustrated with math could regain confidence when receiving the feedback for correct answers in the basic tasks. Thanks to this, they would not feel high-pressure when moving on to current lessons.” In a sense, the immediate feedback provided the low-achieving students with sufficient support and may encourage them to persistently learn mathematics. Furthermore, by learning individually after class, they could effectively prepare themselves for future learning. The results suggested that the system could serve as a scaffolding on conventional instruction for low-achieving students. The students could benefit from such a blended learning environment and, thus, build confidence in mathematics by learning at their own paces.

The low-achieving students as a whole were also attracted to the system and felt satisfaction from it. Teacher (#T-307) said that, “There was a hyperactive and mischievous student in my class. However, when he was alone, he would go on to Math-Island, concentrating on the tasks quietly. He gradually came to enjoy learning mathematics. It seemed that Math-Island was more attractive to them than a lecture by a teacher. I believed that students could be encouraged, thus improve their ability and learn happily.” Another teacher (#T-304) further pointed out that, “For students, they did not only feel like they were learning mathematics because of the game-based user interface. Conversely, they enjoyed the contentment when completing a task, as if they were going aboard to join a competition.” In teachers’ opinions, such a game-based learning environment did not disturb their instruction. Instead, the system could help the teachers attract students’ attention and motivate them to learn mathematics actively because of its appealing game and joyful learning tasks. Furthermore, continuously overcoming the tasks might bring students a sense of achievement and satisfaction.

Discussion on some features of this study

In addition to the enhancement of achievement and interest, we noticed that there are some features in this study and our design worth some discussion.

The advantages of building a long-term study

Owing to the limitations of deployment time and sample sizes, it is hard for most researchers to conduct a longitudinal study. Fortunately, we had a chance to maintain a long-term collaboration with an experimental school for more than 2 years. From this experiment, we notice that there are two advantages to conducting a long-term study.

Obtaining substantial evidence from the game-based learning environment

The research environment was a natural setting, which could not be entirely controlled and manipulated like most experiments in laboratories. However, this study could provide long-term evidence to investigate how students learned in a game-based learning environment with their tablets. It should be noted that we did not aim to replace teachers in classrooms with the Math-Island game. Instead, we attempted to establish an ordinary learning scenario, in which the teachers and students regarded the game as one of the learning resources. For example, teachers may help low-achieving students to improve their understanding of a specific concept in the Math-Island system. When students are learning mathematics in the Math-Island game, teachers may take the game as a formative assessment and locate students’ difficulties in mathematics.

Supporting teachers’ instructions and facilitating students’ learning

The long-term study not only proved the effectiveness of Math-Island but also offered researchers an opportunity to determine teachers’ roles in such a computer-supported learning environment. For example, teachers may encounter difficulties in dealing with the progress of both high- and low-achieving students. How do they take care of all students with different abilities at the same time? Future teachers may require more teaching strategies in such a self-directed learning environment. Digital technology has an advantage in helping teachers manage students’ learning portfolios. For example, the system can keep track of all the learning activities. Furthermore, the system should provide teachers with monitoring functions so that they know the average status of their class’s and individuals’ learning progress. Even so, it is still a challenge for researchers to develop a well-designed visualization tool to support teachers’ understanding of students’ learning conditions and their choice of appropriate teaching strategies.

Incorporating a gamified knowledge map of the elementary mathematics curriculum

Providing choices of learning paths

Math-Island uses a representational metaphor of an “island,” where a virtual city is located and represents the knowledge map. Furthermore, the island comprises areas, roads, and buildings, which are the embodiments of domains, subdomains, and concepts in the curriculum, respectively. Because the gamified knowledge map provides students with multiple virtual roads to learn in the system, every student may take different routes. For instance, some students may be more interested in geometry, while others may be confident in exploring the rules of arithmetic. In this study, we noticed that the low-achieving students needed more time to work on basic tasks, while high-achieving students easily passed those tasks and moved on to the next ones. As a result, some of the high-achieving students had already started to learn the materials for the next grade level. This was possibly because high-achieving students were able to respond well to challenging assignments (Singh 2011). Therefore, we should provide high-achieving students with more complex tasks to maintain their interest. For example, Math-Island should provide some authentic mathematical problems as advanced exercises.

Visualizing the learning portfolio

In this study, we demonstrated a long-term example of incorporating a gamified knowledge map in an elementary mathematical curriculum. In the Math-Island game, the curriculum is visualized as a knowledge map instead of a linear sequence, as in textbooks. By doing so, students are enabled to explore relationships in the mathematics curriculum represented by the knowledge map; that is, the structure of the different roads on Math-Island. Furthermore, before learning, students may preview what will be learned, and after learning, students may also reflect on how well they learned. Unlike traditional lectures or textbooks, in which students could only follow a predefined order to learn knowledge without thinking why they have to learn it, the knowledge map allows students to understand the structure of knowledge and plan how to achieve advanced knowledge. Although the order of knowledge still remains the same, students take primary control of their learning. In a sense, the knowledge map may liberate elementary students from passive learning.

Adopting the mechanisms of a construction management game

This 2-year study showed that the adaptation of two game mechanisms, construction and sightseeing, into the elementary mathematical curriculum could effectively improve students’ learning achievement. The reason may be that students likely developed interests in using Math-Island to learn mathematics actively, regardless of whether they are high- and low-achieving students.

Gaining a sense of achievement and ownership through the construction mechanism

Regardless of the construction mechanism, Math-Island allows students to plan and manage their cities by constructing and upgrading buildings. Math-Island took the advantages of construction management games to facilitate elementary students’ active participation in their mathematical learning. Furthermore, students may manage their knowledge by planning and constructing of buildings on their virtual islands. Like most construction management games, students set goals and make decisions so that they may accumulate their assets. These assets are not only external rewards but also visible achievements, which may bring a sense of ownership and confidence. In other words, the system gamified the process of self-directed learning.

Demonstrating learning result to peers through the sightseeing mechanism

As for the sightseeing mechanism, in conventional instruction, elementary students usually lack the self-control to learn knowledge actively (Duckworth et al. 2014) or require a social stage to show other students, resulting in low achievement and motivation. On the other hand, although previous researchers have already proposed various self-regulated learning strategies (such as Taub et al. 2014), it is still hard for children to keep adopting specific learning strategies for a long time. For these reasons, this study uses the sightseeing mechanism to engage elementary students in a social stage to show other students how well their Math-Islands have been built. For example, in Math-Island, although the students think that they construct buildings in their islands, they plan the development of their knowledge maps. After learning, they may also reflect on their progress by observing the appearance of the buildings.

In brief, owing to the construction mechanism, the students are allowed to choose a place and build their unique islands by learning concepts. During the process, students have to do the learning task, get feedback, and get rewards, which are the three major functions of the construction functions. In the sightseeing mechanism, students’ unique islands (learning result) can be shared and visited by other classmates. The student’s Math-Island thus serves as a stage for showing off their learning results. The two mechanisms offer an incentive model connected to the game mechanism’s forming a positive cycle: the more the students learn, the more unique islands they can build, with more visitors.

Conclusion and future work

This study reported the results of a 2-year experiment with the Math-Island system, in which a knowledge map with extensive mathematics content was provided to support the complete elementary mathematics curriculum. Each road in Math-Island represents a mathematical topic, such as addition. There are many buildings on each road, with each building representing a unit of the mathematics curriculum. Students may learn about the concept and practice it in each building while being provided with feedback by the system. In addition, the construction management online game mechanism is designed to enhance and sustain students’ interest in learning mathematics. The aim of this study was not only to examine whether the Math-Island system could improve students’ achievements but also to investigate how much the low-achieving students would be interested in learning mathematics after using the system for 2 years.

As for enhancing achievement, the result indicated that the Math-Island system could effectively improve the students’ ability to calculate expressions and solve word problems. In particular, the low-achieving students outperformed those of the norm in terms of word problem-solving. For enhancing interest, we found that both the low-achieving and the high-achieving students in the experimental school, when using the Math-Island system, maintained a rather high level of interest in learning mathematics and using the system. The results of this study indicated some possibility that elementary students could be able to learn mathematics in a self-directed learning fashion (Nilson 2014; Chen et al. 2012a, b) under the Math-Island environment. This possibility is worthy of future exploration. For example, by analyzing student data, we can investigate how to support students in conducting self-directed learning. Additionally, because we have already collected a considerable amount of student data, we are currently employing machine learning techniques to improve feedback to the students. Finally, to provide students appropriate challenges, the diversity, quantity, and difficulty of content may need to be increased in the Math-Island system.