 Research
 Open Access
 Published:
Enhancing achievement and interest in mathematics learning through MathIsland
Research and Practice in Technology Enhanced Learningvolume 14, Article number: 5 (2019)
Abstract
Conventional teacherled instruction remains dominant in most elementary mathematics classrooms in Taiwan. Under such instruction, the teacher can rarely take care of all students. Many students may then continue to fall behind the standard of mathematics achievement and lose their interest in mathematics; they eventually give up on learning mathematics. In fact, students in Taiwan generally have lower interest in learning mathematics compared to many other regions/countries. Thus, how to enhance students’ mathematics achievement and interest are two major problems, especially for those lowachieving students. This paper describes how we designed a gamebased learning environment, called MathIsland, by incorporating the mechanisms of a construction management game into the knowledge map of the elementary mathematics curriculum. We also report an experiment conducted with 215 elementary students for 2 years, from grade 2 to grade 3. In this experiment, in addition to teacherled instruction in the classroom, students were directed to learn with MathIsland by using their own tablets at school and at home. As a result of this experiment, we found that there is an increase in students’ mathematics achievement, especially in the calculation and word problems. Moreover, the achievements of lowachieving students in the experimental school outperformed the lowachieving students in the control school (a control group in another school) in word problems. Moreover, both the lowachieving students and the highachieving students in the experimental school maintained a rather high level of interest in mathematics and in the system.
Introduction
Mathematics has been regarded as a fundamental subject because arithmetic and logical reasoning are the basis of science and technology. For this reason, educational authorities emphasize students’ proficiency in computational skills and problemsolving. Recently, the results of the Program for International Student Assessment (PISA) and the Trends in Mathematics and Science Study (TIMSS) in 2015 (OECD 2016; Mullis et al. 2016) revealed a challenge for Taiwan. Although Taiwanese students had higher average performance in mathematics literacy compared to students in other countries, there was still a significant percentage of lowachieving students in Taiwan. Additionally, most Taiwanese students show low levels of interest and confidence in learning mathematics (Lee 2012).
The existence of a significant percentage of lowachieving students is probably due to teacherled instruction, which still dominates mathematics classrooms in most Asian countries. It should be noted that students in every classroom possess different abilities and hence demonstrate different achievements. Unfortunately, in teacherled instruction, all the students are required to learn from the teacher in the same way at the same pace (Hwang et al. 2012). Lowachieving students, without sufficient time, are forced to receive knowledge passively. Barr and Tagg (1995) pointed out that it is urgent for lowachieving students to have more opportunities to learn mathematics at their own pace. Researchers suggested onetoone technology (Chan et al. 2006) through which every student is equipped with a device to learn in school or at home seamlessly. Furthermore, they can receive immediate feedback from MathIsland, which supports their individualized learning actively and productively. Thus, this may provide more opportunities for helping lowachieving students improve their achievement.
The lowinterest problem for almost all students in Taiwan is usually accompanied by low motivation (Krapp 1999). Furthermore, students with continuously low performance in mathematics may eventually lose their interest and refuse to learn further (Schraw et al. 2001). This is a severe problem. To motivate students to learn, researchers design educational games to provide enjoyable and engaging learning experiences (Kiili and Ketamo 2007). Some of these researchers found that gamebased learning may facilitate students’ learning in terms of motivation and learning effects (Liu and Chu 2010), spatial abilities and attention (Barlett et al. 2009), situated learning, and problemsolving (Li and Tsai 2013). Given these positive results, we hope that our educational game can enhance and sustain the student’s interest in learning mathematics.
In fact, many researchers who endeavored to develop educational games for learning mathematics have shown that their games could facilitate mathematics performance, enjoyment, and selfefficacy (Ku et al. 2014; McLaren et al. 2017). Although some of the studies were conducted for as many as 4 months (e.g., Hanus and Fox 2015), one may still criticize them for the possibility that the students’ interest could be a novelty effect—meaning their interest will decrease as the feeling of novelty diminishes over time (Koivisto and Hamari 2014). Due to the limitations of either experimental time or sample sizes, most studies could not effectively exclude the novelty effect of games, unless they were conducted in a natural setting for a long time.
In this study, we collaborated with an experimental elementary school for more than 2 years. The mathematics teachers in the school adopted our online educational game, MathIsland. The students used their own tablet PCs to learn mathematics from the game in class or at home at their own pace. In particular, lowachieving students might have a chance to catch up with the other students and start to feel interested in learning mathematics. Most importantly, because the online educational game was a part of the mathematics curriculum, the students could treat the game as their ordinary learning materials like textbooks. In this paper, we reported a 2year study, in which 215 second graders in the school adopted the MathIsland game in their daily routine. More specifically, the purpose of this paper was to investigate the effect of the game on students’ mathematics achievement. Additionally, we were also concerned about how well the lowachieving students learned, whether they were interested in mathematics and the game, and how their interest in mathematics compared with that of highachieving students. In such a longterm study with a large sample size, it was expected that the novelty effect would be considerably reduced, allowing us to evaluate the effect of the educational game on students’ achievement and interest.
The paper is organized as follows. In the “Related works” section, we review related studies on computersupported mathematics learning and educational games. In the “Design” section, the game mechanism and the system design are presented. In the “Method” section, we describe the research method and the procedures of this study. In the “Results” section, the research results about students’ achievement and interest are presented. In the “Discussion on some features of this study” section, we discuss the longterm study, knowledge map design, and the two game mechanisms. Finally, the summary of the current situation and potential future work is described in the “Conclusion and future work” section.
Related works
Computersupported mathematics learning
The mathematics curriculum in elementary schools basically includes conceptual understanding, procedural fluency, and strategic competence in terms of mathematical proficiency (see Kilpatrick et al. 2001). First, conceptual understanding refers to students’ comprehension of mathematical concepts and the relationships between concepts. Researchers have designed various computerbased scaffolds and feedback to build students’ concepts and clarify potential misconceptions. For example, for guiding students’ discovery of the patterns of concepts, Yang et al. (2012) adopted an inductive discovery learning approach to design online learning materials in which students were provided with similar examples with a critical attribute of the concept varied. McLaren et al. (2017) provided students with prompts to correct their common misconceptions about decimals. They conducted a study with the game adopted as a replacement for seven lessons of regular mathematics classes. Their results showed that the educational game could facilitate better learning performance and enjoyment than a conventional instructional approach.
Second, procedural fluency refers to the skill in carrying out calculations correctly and efficiently. For improving procedural fluency, students need to have knowledge of calculation rules (e.g., place values) and practice the procedure without mistakes. Researchers developed various digital games to overcome the boredom of practice. For example, Chen et al. (2012a, 2012b) designed a Cross Number Puzzle game for practicing arithmetic expressions. In the game, students could individually or collaboratively solve a puzzle, which involved extensive calculation. Their study showed that the lowability students in the collaborative condition made the most improvement in calculation skills. Ku et al. (2014) developed minigames to train students’ mental calculation ability. They showed that the minigames could not only improve students’ calculation performance but also increase their confidence in mathematics.
Third, strategic competence refers to mathematical problemsolving ability, in particular, word problemsolving in elementary education. Some researchers developed multilevel computerbased scaffolds to help students translate word problems to equations step by step (e.g., GonzálezCalero et al. 2014), while other researchers noticed the problem of overscaffolding. Specifically, students could be too scaffolded and have little space to develop their abilities. To avoid this situation, many researchers proposed allowing students to seek help during word problemsolving (Chase and Abrahamson 2015; Roll et al. 2014). For example, Cheng et al. (2015) designed a Scaffolding Seeking system to encourage elementary students to solve word problems by themselves by expressing their thinking first, instead of receiving and potentially abusing scaffolds.
Digital educational games for mathematics learning
Because mathematics is an abstract subject, elementary students easily lose interest in it, especially lowachieving students. Some researchers tailored educational games for learning a specific set of mathematical knowledge (e.g., the Decimal Points game; McLaren et al. 2017), so that students could be motivated to learn mathematics. However, if our purpose was to support a complete mathematics curriculum for elementary schools, it seemed impractical to design various educational games for all kinds of knowledge. A feasible approach is to adopt a gamified content structure to reorganize all learning materials. For example, inspired by the design of most roleplaying games, Chen et al. (2012a, 2012b) proposed a threetiered framework of gamebased learning—a game world, quests, and learning materials—for supporting elementary students’ enjoyment and goal setting in mathematics learning. Furthermore, while a game world may facilitate students’ exploration and participation, quests are the containers of learning materials with specific goals and rewards. In the game world, students receive quests from nonplayer virtual characters, who may enhance social commitments. To complete the quests, students have to make efforts to undertake learning materials. Today, quests have been widely adopted in the design of educational games (e.g., Azevedo et al. 2012; Hwang et al. 2015).
However, in educational games with quests, students still play the role of receivers rather than active learners. To facilitate elementary students’ initiative, Lao et al. (2017) designed digital learning contracts, which required students to set weekly learning goals at the beginning of a week and checked whether they achieved the goals at the end of the week. More specifically, when setting weekly goals, students had to decide on the quantity of learning materials that they wanted to undertake in the coming week. Furthermore, they also had to decide the average correctness of the tests that followed the learning materials. To help them set reasonable and feasible goals, the system provided statistics from the past 4 weeks. As a result, the students may reflect on how well they learned and then make appropriate decisions. After setting goals, students are provided with a series of learning materials for attempting to accomplish those goals. At the end of the week, they may reflect on whether they achieved their learning goals in the contracts. In a sense, learning contracts may not only strengthen the sense of commitment but also empower students to take more control of their learning.
In textbooks or classrooms, learning is usually predefined as a specific sequence, which students must follow to learn. Nevertheless, the structure of knowledge is not linear, but a network. If we could reorganize these learning materials according to the structure of knowledge, students could explore knowledge and discover the relationships among different pieces of knowledge when learning (Davenport and Prusak 2000). Knowledge mapping has the advantage of providing students concrete content through explicit knowledge graphics (Ebener et al. 2006). Previous studies have shown that the incorporation of knowledge structures into educational games could effectively enhance students’ achievement without affecting their motivation and selfefficacy (Chu et al. 2015). For this reason, this study attempted to visualize the structure of knowledge in an educational game. In other words, a knowledge map was visualized and gamified so that students could make decisions to construct their own knowledge map in games.
Design
To enhance students’ mathematics achievement and interests, we designed the MathIsland online game by incorporating a gamified knowledge map of the elementary mathematics curriculum. More specifically, we adopt the mechanisms of a construction management game, in which every student owns a virtual island (a city) and plays the role of the mayor. The goal of the game is to build their cities on the islands by learning mathematics.
System architecture
The MathIsland game is a Web application, supporting crossdevice interactions among students, teachers, and the mathematics content structure. The system architecture of the MathIsland is shown in Fig. 1. The pedagogical knowledge and learning materials are stored in the module of digital learning content, organized by a mathematical knowledge map. The students’ portfolios about interactions and works are stored in the portfolio database and the status database. When a student chooses a goal concept in the knowledge map, the corresponding digital learning content is arranged and delivered to his/her browser. Besides, when the student is learning in the MathIsland, the feedback module provides immediate feedback (e.g., hints or scaffolded solutions) for guidance and grants rewards for encouragement. The learning results can also be shared with other classmates by the interaction module. In addition to students, their teachers can also access the databases for the students’ learning information. Furthermore, the information consists of the students’ status (e.g., learning performance or virtual achievement in the game) and processes (e.g., their personal learning logs). In the MathIsland, it is expected that students can manage their learning and monitor the learning results by the construction management mechanism. In the meantime, teachers can also trace students’ learning logs, diagnose their weaknesses from portfolio analysis, and assign students with specific tasks to improve their mathematics learning.
Knowledge map
To increase students’ mathematics achievement, the MathIsland game targets the complete mathematics curriculum of elementary schools in Taiwan, which mainly contains the four domains: numerical operation, quantity and measure, geometry, and statistics and probability (Ministry of Education of R.O.C. 2003). Furthermore, every domain consists of several subdomains with corresponding concepts. For instance, the domain of numerical operation contains four subdomains: numbers, addition, and subtraction for the first and second graders. In the subdomain of subtraction, there are a series of concepts, including the meaning of subtraction, onedigit subtraction, and twodigit subtraction. These concepts should be learned consecutively. In the MathIsland system, the curriculum is restructured as a knowledge map, so that they may preview the whole structure of knowledge, recall what they have learned, and realize what they will learn.
More specifically, the MathIsland system uses the representational metaphor of an “island,” where a virtual city is located and represents the knowledge map. Furthermore, the island comprises areas, roads, and buildings, which are the embodiments of domains, subdomains, and concepts in the curriculum, respectively. As shown in Fig. 2, for example, in an area of numeral operation in MathIsland, there are many roads, such as an addition road and a subtraction road. On the addition road, the first building should be the meaning of addition, followed by the buildings of onedigit addition and then twodigit addition. Students can choose these buildings to learn mathematical concepts. In each building, the system provides a series of learning tasks for learning the specific concept. Currently, MathIsland provides elementary students with more than 1300 learning tasks from the first grade to the sixth grade, with more than 25,000 questions in the tasks.
In MathIsland, a learning task is an interactive page turner, including video clips and interactive exercises for conceptual understanding, calculation, and word problemsolving. In each task, the learning procedure mainly consists of three steps: watching demonstrations, practicing examples, and getting rewards. First, students learn a mathematical concept by watching videos, in which a human tutor demonstrates examples, explains the rationale, and provides instructions. Second, students follow the instructions to answer a series of questions related to the examples in the videos. When answering questions, students are provided with immediate feedback. Furthermore, if students input wrong answers, the system provides multilevel hints so that they could figure out solutions by themselves. Finally, after completing learning tasks, students receive virtual money according to their accuracy rates in the tasks. The virtual money is used to purchase unique buildings to develop their islands in the game.
Game mechanisms
In the MathIsland game, there are two game mechanisms: construction and sightseeing (as shown in Fig. 3). The former is designed to help students manage their learning process, whereas the latter is designed to facilitate social interaction, which may further motivate students to better develop their cities. By doing so, the MathIsland can be regarded as one’s learning portfolio, which is a complete record that purposely collects information about one’s learning processes and outcomes (Arter and Spandel 2005). Furthermore, learning portfolios are a valuable research tool for gaining an understanding about personal accomplishments (Birgin and Baki 2007), because learning portfolios can display one’s learning process, attitude, and growth after learning (Lin and Tsai 2001). The appearance of the island reflects what students have learned and have not learned from the knowledge map. When students observe their learning status in an interesting way, they may be concerned about their learning status with the enhanced awareness of their learning portfolios. By keeping all activity processes, students can reflect on their efforts, growth, and achievements. In a sense, with the game mechanisms, the knowledge map can be regarded as a manipulatable open learner model, which not only represents students’ learning status but also invites students to improve it (Vélez et al. 2009).
First, the construction mechanism allows students to plan and manage their cities by constructing and upgrading buildings. To do so, they have to decide which buildings they want to construct or upgrade. Then, they are required to complete corresponding learning tasks in the building to determine which levels of buildings they can construct. As shown in Fig. 4, the levels of buildings depend on the completeness of a certain concept, compared with the thresholds. For example, when students complete one third of the learning tasks, the first level of a building is constructed. Later, when they complete two thirds of the tasks, the building is upgraded to the second level. After completing all the tasks in a building, they also complete the final level and are allowed to construct the next building on the road. Conversely, if students failed the lowest level of the threshold, they might need to watch the video and/or do the learning tasks again. By doing so, students can make their plans to construct the buildings at their own pace. When students manage their cities, they actually attempt to improve their learning status. In other words, the construction mechanism offers an alternative way to guide students to regulate their learning efforts.
Second, the sightseeing mechanism provides students with a social stage to show other students how well their MathIslands have been built. This mechanism is implemented as a public space, where other students play the role of tourists who visit MathIsland. In other words, this sightseeing mechanism harnesses social interaction to improve individual learning. As shown in Fig. 4, because students can construct different areas or roads, their islands may have different appearances. When students visit a welldeveloped MathIsland, they might have a positive impression, which may facilitate their selfreflection. Accordingly, they may be willing to expend more effort to improve their island. On the other hand, the student who owns the island may also be encouraged to develop their island better. Furthermore, when students see that they have a completely constructed building on a road, they may perceive that they are good at these concepts. Conversely, if their buildings are small, the students may realize their weaknesses or difficulties in these concepts. Accordingly, they may be willing to make more effort for improvement. On the other hand, the student who owns the island may also be encouraged to develop their island better. In a word, the visualization may play the role of stimulators, so that students may be motivated to improve their learning status.
Method
This paper reported a 2year study in which the MathIsland system was adopted in an elementary school. The study addressed the following two research questions: (1) Did the MathIsland system facilitate students’ mathematics achievement in terms of conceptual understanding, calculating, and word problemsolving? In particular, how was the mathematics achievement of the lowachieving students? (2) What was students’ levels of interest in mathematics and the system, particularly that of lowachieving students?
Participants
The study, conducted from June 2013 to June 2015, included 215 second graders (98 females and 117 males), whose average age was 8 years old, in an elementary school located in a suburban region of a northern city in Taiwan. The school had collaborated with our research team for more than 2 years and was thus chosen as an experimental school for this study. In this school, approximately one third of the students came from families with a low or middle level of socioeconomic status. It was expected that the lessons learned from this study could be applicable to other schools with similar student populations in the future. The parents were supportive of this program and willing to provide personal tablets for their children (Liao et al. 2017). By doing so, the students in the experimental school were able to use their tablets to access the MathIsland system as a learning tool at both school and home. To compare the students’ mathematics achievement with a baseline, this study also included 125 second graders (63 females and 62 males) from another school with similar socioeconomic backgrounds in the same region of the city as a control school. The students in the control school received only conventional mathematics instruction without using the MathIsland system during the 2year period.
Procedure
Before the first semester, a 3week training workshop was conducted to familiarize the students with the basic operation of tablets and the MathIsland system. By doing so, it was ensured that all participants had similar prerequisite skills. The procedure of this study was illustrated in Table 1. At the beginning of the first semester, a mathematics achievement assessment was conducted as a pretest in both the experimental and the control school to examine the students’ initial mathematics ability as second graders. From June 2013 to June 2015, while the students in the control school learned mathematics in a conventional way, the students in the experimental school learned mathematics not only in mathematics classes but also through the MathIsland system. Although the teachers in the experimental school mainly adopted lectures in mathematics classes, they used the MathIsland system as learning materials at school and for homework. At the same time, they allowed the students to explore the knowledge map at their own pace. During the 2 years, every student completed 286.78 learning tasks on average, and each task took them 8.86 min. Given that there were 344 tasks for the second and third graders, the students could finish 83.37% of tasks according to the standard progress. The data also showed that the average correctness rate of the students was 85.75%. At the end of the second year, another mathematics achievement assessment was administered as a posttest in both schools to evaluate students’ mathematics ability as third graders. Additionally, an interest questionnaire was employed in the experimental school to collect the students’ perceptions of mathematics and the MathIsland system. To understand the teachers’ opinions of how they feel about the students using the system, interviews with the teachers in the experimental school were also conducted.
Data collection
Mathematics achievement assessment
To evaluate the students’ mathematics ability, this study adopted a standardized achievement assessment of mathematics ability (Lin et al. 2009), which was developed from a random sample of elementary students from different counties in Taiwan to serve as a norm with appropriate reliability (the internal consistency was 0.85, and the testretest reliability was 0.86) and validity (the correlation by domain experts in content validity was 0.92, and the concurrent validity was 0.75). As a pretest, the assessment of the second graders consisted of 50 items, including conceptual understanding (23 items), calculating (18 items), and word problemsolving (9 items). As a posttest, the assessment of the third graders consisted of 60 items, including conceptual understanding (18 items), calculating (27 items), and word problemsolving (15 items). The scores of the test ranged from 0 to 50 points. Because some students were absent during the test, this study obtained 209 valid tests from the experimental school and 125 tests from the control school.
Interest questionnaire
The interest questionnaire comprised two parts: students’ interest in mathematics and the MathIsland system. Regarding the first part, this study adopted items from a mathematics questionnaire of PISA and TIMSS 2012 (OECD 2013; Mullis et al. 2012), the reliability of which was sound. This part included three dimensions: attitude (14 items, Cronbach’s alpha = .83), initiative (17 items, Cronbach’s alpha = .82), and confidence (14 items Cronbach’s alpha = .72). Furthermore, the dimension of attitude was used to assess the tendency of students’ view on mathematics. For example, a sample item of attitudes was “I am interested in learning mathematics.” The dimension of initiatives was used to assess how students were willing to learn mathematics actively. A sample item of initiatives was “I keep studying until I understand mathematics materials.” The dimension of confidences was used to assess students’ perceived mathematics abilities. A sample item was “I am confident about calculating whole numbers such as 3 + 5 × 4.” These items were translated to Chinese for this study. Regarding the second part, this study adopted selfmade items to assess students’ motivations for using the MathIsland system. This part included two dimensions: attraction (8 items) and satisfaction (5 items). The dimension of attraction was used to assess how well the system could attract students’ attention. A sample item was “I feel Mathisland is very appealing to me.” The dimension of satisfaction was used to assess how the students felt after using the system. A sample item was “I felt that upgrading the buildings in my MathIsland brought me much happiness.” These items were assessed according to a 4point Likert scale, ranging from “strongly disagreed (1),” “disagreed (2),” “agreed (3),” and “strongly agreed (4)” in this questionnaire. Due to the absences of several students on the day the questionnaire was administered, there were only 207 valid questionnaires in this study.
Teacher interview
This study also included teachers’ perspectives on how the students used the MathIsland system to learn mathematics in the experimental school. This part of the study adopted semistructured interviews of eight teachers, which comprised the following three main questions: (a) Do you have any notable stories about students using the MathIsland system? (b) Regarding MathIsland, what are your teaching experiences that can be shared with other teachers? (c) Do you have any suggestions for the MathIsland system? The interview was recorded and transcribed verbatim. The transcripts were coded and categorized according to the five dimensions of the questionnaire (i.e., the attitude, initiative, and confidence about mathematics, as well as the attraction and satisfaction with the system) as additional evidence of the students’ interest in the experimental school.
Data analysis
For the first research question, this study conducted a multivariate analysis of variance (MANOVA) with the schools as a betweensubject variable and the students’ scores (conceptual understanding, calculating, and word problemsolving) in the pre/posttests as dependent variables. Moreover, this study also conducted a MANOVA to compare the lowachieving students from both schools. In addition, the tests were also carried out to compare achievements with the norm (Lin et al. 2009). For the second research question, several z tests were used to examine how the interests of the lowachieving students were distributed compared with the whole sample. Teachers’ interviews were also adopted to support the results of the questionnaire.
Results
Mathematics achievement
To examine the homogeneity of the students in both schools in the first year, the MANOVA of the pretest was conducted. The results, as shown in Table 2, indicated that there were no significant differences in their initial mathematics achievements in terms of conceptual understanding, calculating, and word problemsolving (Wilks’ λ = 0.982, F(3330) = 2.034, p > 0.05). In other words, the students of both schools had similar mathematics abilities at the time of the first mathematics achievement assessment and could be fairly compared.
At the end of the fourth grade, the students of both schools received the posttest, the results of which were examined by a MANOVA. As shown in Table 3, the effect of the posttest on students’ mathematics achievement was significant (Wilks’ λ = 0.946, p < 0.05). The results suggested that the students who used MathIsland for 2 years had better mathematics abilities than those who did not. The analysis further revealed that the univariate effects on calculating and word problemsolving were significant, but the effect on conceptual understanding was insignificant. The results indicated that the students in the experimental school outperformed their counterparts in terms of the procedure and application of arithmetic. The reason may be that the system provided students with more opportunities to do calculation exercises and word problems, and the students were more willing to do these exercises in a gamebased environment. Furthermore, they were engaged in solving various exercises with the support of immediate feedback until they passed the requirements of every building in their MathIsland. However, the students learned mathematical concepts mainly by watching videos in the system, which provided only demonstrations like lectures in conventional classrooms. For this reason, the effect of the system on conceptual understanding was similar to that of teachers’ conventional instruction.
Furthermore, to examine the differences between the lowachieving students in both schools, another MANOVA was also conducted on the pretest and the posttest. The pretest results indicated that there were no significant differences in their initial mathematics achievement in terms of conceptual understanding, calculating, and word problemsolving (Wilks’ λ = 0.943, F(3110) = 2.210, p > 0.05).
The MANOVA analysis of the posttest is shown in Table 4. The results showed that the effect of the system on the mathematics achievement of lowachieving students was significant (Wilks’ λ = 0.934, p < 0.05). The analysis further revealed that only the univariate effect on word problemsolving was significant. The results suggested that the lowachieving students who used MathIsland for 2 years had better word problemsolving ability than those students in the control school, but the effect on conceptual understanding and procedural fluency was insignificant. The results indicated that the MathIsland system could effectively enhance lowachieving students’ ability to solve word problems.
Because the mathematics achievement assessment was a standardized achievement assessment (Lin et al. 2009), the research team did a further analysis of the assessments by comparing the results with the norm. In the pretest, the average score of the control school was the percentile rank of a score (PR) 55, but their average score surprisingly decreased to PR 34 in the posttest. The results confirmed the fact that conventional mathematics teaching in Taiwan might result in an Mshape distribution, suggesting that lowachieving students required additional learning resources. Conversely, the average score of the experimental school was PR 48 in the pretest, and their score slightly decreased to PR 44 in the posttest. Overall, both PR values were decreasing, because the mathematics curriculum became more and more difficult from the second grade to the fourth grade. However, it should be noted that the experimental school has been less affected, resulting in a significant difference compared with the control school (see Table 5). Notably, the average score of word problemsolving in the posttest of the experimental school was PR 64, which was significantly higher than the nationwide norm (z = 20.8, p < .05). The results were consistent with the univariate effect of the MANOVA on word problemsolving, suggesting that the MathIsland system could help students learn to complete word problems better. This may be because the learning tasks in MathIsland provided students with adequate explanations for various types of word problems and provided feedback for exercises.
Interest
To examine whether the lowachieving students had low levels of interest in mathematics and the MathIsland system, the study adopted z tests on the data of the interest questionnaire. Table 5 shows the descriptive statistics and the results of the z tests. Regarding the interest in mathematics, the analysis showed that the interest of the lowachieving students was similar to that of the whole sample in terms of attitude, initiative, and confidence. The results were different from previous studies asserting that lowachieving students tended to have lower levels of interest in mathematics (AlZoubi and Younes 2015). The reason was perhaps that the lowachieving students were comparably motivated to learn mathematics in the MathIsland system. As a result, a teacher (#T301) said, “some students would like to go to MathIsland after school, and a handful of students could even complete up to forty tasks (in a day),” implying that the students had a positive attitude and initiative related to learning mathematics.
Another teacher (T312) also indicated “some students who were frustrated with math could regain confidence when receiving the feedback for correct answers in the basic tasks. Thanks to this, they would not feel highpressure when moving on to current lessons.” In a sense, the immediate feedback provided the lowachieving students with sufficient support and may encourage them to persistently learn mathematics. Furthermore, by learning individually after class, they could effectively prepare themselves for future learning. The results suggested that the system could serve as a scaffolding on conventional instruction for lowachieving students. The students could benefit from such a blended learning environment and, thus, build confidence in mathematics by learning at their own paces.
The lowachieving students as a whole were also attracted to the system and felt satisfaction from it. Teacher (#T307) said that, “There was a hyperactive and mischievous student in my class. However, when he was alone, he would go on to MathIsland, concentrating on the tasks quietly. He gradually came to enjoy learning mathematics. It seemed that MathIsland was more attractive to them than a lecture by a teacher. I believed that students could be encouraged, thus improve their ability and learn happily.” Another teacher (#T304) further pointed out that, “For students, they did not only feel like they were learning mathematics because of the gamebased user interface. Conversely, they enjoyed the contentment when completing a task, as if they were going aboard to join a competition.” In teachers’ opinions, such a gamebased learning environment did not disturb their instruction. Instead, the system could help the teachers attract students’ attention and motivate them to learn mathematics actively because of its appealing game and joyful learning tasks. Furthermore, continuously overcoming the tasks might bring students a sense of achievement and satisfaction.
Discussion on some features of this study
In addition to the enhancement of achievement and interest, we noticed that there are some features in this study and our design worth some discussion.
The advantages of building a longterm study
Owing to the limitations of deployment time and sample sizes, it is hard for most researchers to conduct a longitudinal study. Fortunately, we had a chance to maintain a longterm collaboration with an experimental school for more than 2 years. From this experiment, we notice that there are two advantages to conducting a longterm study.
Obtaining substantial evidence from the gamebased learning environment
The research environment was a natural setting, which could not be entirely controlled and manipulated like most experiments in laboratories. However, this study could provide longterm evidence to investigate how students learned in a gamebased learning environment with their tablets. It should be noted that we did not aim to replace teachers in classrooms with the MathIsland game. Instead, we attempted to establish an ordinary learning scenario, in which the teachers and students regarded the game as one of the learning resources. For example, teachers may help lowachieving students to improve their understanding of a specific concept in the MathIsland system. When students are learning mathematics in the MathIsland game, teachers may take the game as a formative assessment and locate students’ difficulties in mathematics.
Supporting teachers’ instructions and facilitating students’ learning
The longterm study not only proved the effectiveness of MathIsland but also offered researchers an opportunity to determine teachers’ roles in such a computersupported learning environment. For example, teachers may encounter difficulties in dealing with the progress of both high and lowachieving students. How do they take care of all students with different abilities at the same time? Future teachers may require more teaching strategies in such a selfdirected learning environment. Digital technology has an advantage in helping teachers manage students’ learning portfolios. For example, the system can keep track of all the learning activities. Furthermore, the system should provide teachers with monitoring functions so that they know the average status of their class’s and individuals’ learning progress. Even so, it is still a challenge for researchers to develop a welldesigned visualization tool to support teachers’ understanding of students’ learning conditions and their choice of appropriate teaching strategies.
Incorporating a gamified knowledge map of the elementary mathematics curriculum
Providing choices of learning paths
MathIsland uses a representational metaphor of an “island,” where a virtual city is located and represents the knowledge map. Furthermore, the island comprises areas, roads, and buildings, which are the embodiments of domains, subdomains, and concepts in the curriculum, respectively. Because the gamified knowledge map provides students with multiple virtual roads to learn in the system, every student may take different routes. For instance, some students may be more interested in geometry, while others may be confident in exploring the rules of arithmetic. In this study, we noticed that the lowachieving students needed more time to work on basic tasks, while highachieving students easily passed those tasks and moved on to the next ones. As a result, some of the highachieving students had already started to learn the materials for the next grade level. This was possibly because highachieving students were able to respond well to challenging assignments (Singh 2011). Therefore, we should provide highachieving students with more complex tasks to maintain their interest. For example, MathIsland should provide some authentic mathematical problems as advanced exercises.
Visualizing the learning portfolio
In this study, we demonstrated a longterm example of incorporating a gamified knowledge map in an elementary mathematical curriculum. In the MathIsland game, the curriculum is visualized as a knowledge map instead of a linear sequence, as in textbooks. By doing so, students are enabled to explore relationships in the mathematics curriculum represented by the knowledge map; that is, the structure of the different roads on MathIsland. Furthermore, before learning, students may preview what will be learned, and after learning, students may also reflect on how well they learned. Unlike traditional lectures or textbooks, in which students could only follow a predefined order to learn knowledge without thinking why they have to learn it, the knowledge map allows students to understand the structure of knowledge and plan how to achieve advanced knowledge. Although the order of knowledge still remains the same, students take primary control of their learning. In a sense, the knowledge map may liberate elementary students from passive learning.
Adopting the mechanisms of a construction management game
This 2year study showed that the adaptation of two game mechanisms, construction and sightseeing, into the elementary mathematical curriculum could effectively improve students’ learning achievement. The reason may be that students likely developed interests in using MathIsland to learn mathematics actively, regardless of whether they are high and lowachieving students.
Gaining a sense of achievement and ownership through the construction mechanism
Regardless of the construction mechanism, MathIsland allows students to plan and manage their cities by constructing and upgrading buildings. MathIsland took the advantages of construction management games to facilitate elementary students’ active participation in their mathematical learning. Furthermore, students may manage their knowledge by planning and constructing of buildings on their virtual islands. Like most construction management games, students set goals and make decisions so that they may accumulate their assets. These assets are not only external rewards but also visible achievements, which may bring a sense of ownership and confidence. In other words, the system gamified the process of selfdirected learning.
Demonstrating learning result to peers through the sightseeing mechanism
As for the sightseeing mechanism, in conventional instruction, elementary students usually lack the selfcontrol to learn knowledge actively (Duckworth et al. 2014) or require a social stage to show other students, resulting in low achievement and motivation. On the other hand, although previous researchers have already proposed various selfregulated learning strategies (such as Taub et al. 2014), it is still hard for children to keep adopting specific learning strategies for a long time. For these reasons, this study uses the sightseeing mechanism to engage elementary students in a social stage to show other students how well their MathIslands have been built. For example, in MathIsland, although the students think that they construct buildings in their islands, they plan the development of their knowledge maps. After learning, they may also reflect on their progress by observing the appearance of the buildings.
In brief, owing to the construction mechanism, the students are allowed to choose a place and build their unique islands by learning concepts. During the process, students have to do the learning task, get feedback, and get rewards, which are the three major functions of the construction functions. In the sightseeing mechanism, students’ unique islands (learning result) can be shared and visited by other classmates. The student’s MathIsland thus serves as a stage for showing off their learning results. The two mechanisms offer an incentive model connected to the game mechanism’s forming a positive cycle: the more the students learn, the more unique islands they can build, with more visitors.
Conclusion and future work
This study reported the results of a 2year experiment with the MathIsland system, in which a knowledge map with extensive mathematics content was provided to support the complete elementary mathematics curriculum. Each road in MathIsland represents a mathematical topic, such as addition. There are many buildings on each road, with each building representing a unit of the mathematics curriculum. Students may learn about the concept and practice it in each building while being provided with feedback by the system. In addition, the construction management online game mechanism is designed to enhance and sustain students’ interest in learning mathematics. The aim of this study was not only to examine whether the MathIsland system could improve students’ achievements but also to investigate how much the lowachieving students would be interested in learning mathematics after using the system for 2 years.
As for enhancing achievement, the result indicated that the MathIsland system could effectively improve the students’ ability to calculate expressions and solve word problems. In particular, the lowachieving students outperformed those of the norm in terms of word problemsolving. For enhancing interest, we found that both the lowachieving and the highachieving students in the experimental school, when using the MathIsland system, maintained a rather high level of interest in learning mathematics and using the system. The results of this study indicated some possibility that elementary students could be able to learn mathematics in a selfdirected learning fashion (Nilson 2014; Chen et al. 2012a, b) under the MathIsland environment. This possibility is worthy of future exploration. For example, by analyzing student data, we can investigate how to support students in conducting selfdirected learning. Additionally, because we have already collected a considerable amount of student data, we are currently employing machine learning techniques to improve feedback to the students. Finally, to provide students appropriate challenges, the diversity, quantity, and difficulty of content may need to be increased in the MathIsland system.
Abbreviations
 PISA:

Program for International Student Assessment
 PR:

The percentile rank of a score
 TIMSS:

Trends in Mathematics and Science Study
References
AlZoubi, S. M., & Younes, M. A. B. (2015). Low academic achievement: causes and results. Theory and Practice in Language Studies, 5(11), 2262.
Arter, J. A., & Spandel, V. (2005). Using portfolios of student work in instruction and assessment. Educational Measurement Issues and Practice, 11(1), 36–44.
Azevedo, R., FeyziBehnagh, R., Duffy, M., Harley, J., & Trevors, G. (2012). Metacognition and selfregulated learning in studentcentered leaning environments. In D. Jonassen & S. Land (Eds.), Theoretical foundations of studentcentered learning environments (pp. 171–197). New York: Routledge.
Barlett, C. P., Anderson, C. A., & Swing, E. L. (2009). Video game effects confirmed, suspected and speculative: a review of the evidence. Simulation & Gaming, 40(3), 377–403.
Barr, R. B., & Tagg, J. (1995). From teaching to learning—a new paradigm for undergraduate education. Change The Magazine of Higher Learning, 27(6), 12–26.
Birgin, O., & Baki, A. (2007). The use of portfolio to assess student’s performance. Journal of Turkish Science Education, 4(2), 75–90.
Chan, T. W., Roschelle, J., Hsi, S., Kinshuk, Sharples, M., Brown, T., et al. (2006). Onetoone technologyenhanced learning: an opportunity for global research collaboration. Research and Practice in Technology Enhanced Learning, 1(01), 3–29.
Chase, K., & Abrahamson, D. (2015). Reversescaffolding algebra: empirical evaluation of design architecture. ZDM Mathematics Education, 47(7), 1195–1209.
Chen, Y. H., Looi, C. K., Lin, C. P., Shao, Y. J., & Chan, T. W. (2012a). Utilizing a collaborative cross number puzzle game to develop the computing ability of addition and subtraction. Educational Technology & Society, 15(1), 354–366.
Chen, Z. H., Liao, C. C., Cheng, H. N., Yeh, C. Y., & Chan, T. W. (2012b). Influence of game quests on pupils’ enjoyment and goalpursuing in math learning. Journal of Educational Technology & Society, 15(2), 317–327.
Cheng, H. N. H., Yang, E. F. Y., Liao, C. C. Y., Chang, B., Huang, Y. C. Y., & Chan, T. W. (2015). Scaffold seeking: a reverse design of scaffolding in computersupported word problem solving. Journal of Educational Computing Research, 53(3), 409–435.
Chu, H. C., Yang, K. H., & Chen, J. H. (2015). A time sequenceoriented concept map approach to developing educational computer games for history courses. Interactive Learning Environments, 23(2), 212–229.
Davenport, T. H. & Prusak, L. (2000). Working knowledge: How organizations manage what they know. Boston: Harvard Business School Press.
Duckworth, A. L., Gendler, T. S., & Gross, J. J. (2014). Selfcontrol in schoolage children. Educational Psychologist, 49(3), 199–217.
Ebener, S., Khan, A., Shademani, R., Compernolle, L., Beltran, M., Lansang, M. A., & Lippman, M. (2006). Knowledge mapping as a technique to support knowledge translation. Bulletin of the World Health Organization, 84, 636–642.
GonzálezCalero, J. A., Arnau, D., Puig, L., & ArevalilloHerráez, M. (2014). Intensive scaffolding in an intelligent tutoring system for the learning of algebraic word problem solving. British Journal of Educational Technology, 46(6), 1189–1200.
Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: a longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education, 80, 152–161.
Hwang, G. J., Chiu, L. Y., & Chen, C. H. (2015). A contextual gamebased learning approach to improving students’ inquirybased learning performance in social studies courses. Computers & Education, 81, 13–25.
Hwang, G. J., Su, J. M., & Chen, N. S. (2012). Elearning introduction and practice. Taiwan: Drmaste.
Kiili, K., & Ketamo, H. (2007). Exploring the learning mechanism in educational games. Journal of Computing and Information Technology, 15(4), 319–324.
Kilpatrick, J., Swafford, J., & Findell, B. (Eds.). (2001). Adding it up: helping children learn mathematics. Washington, DC: National Academies Press.
Koivisto, J., & Hamari, J. (2014). Demographic differences in perceived benefits from gamification. Computers in Human Behavior, 35, 179–188.
Krapp, A. (1999). Interest, motivation and learning: an educationalpsychological perspective. European Journal of Psychology of Education, 14(1), 23–40.
Ku, O., Chen, S. Y., Wu, D. H., Lao, A. C., & Chan, T. W. (2014). The effects of gamebased learning on mathematical confidence and performance: high ability vs. low ability. Journal of Educational Technology & Society, 17(3), 65–78.
Lao, A. C. C., Cheng, H. N., Huang, M. C., Ku, O., & Chan, T. W. (2017). Examining motivational orientation and learning strategies in computersupported selfdirected learning (CSSDL) for mathematics: the perspective of intrinsic and extrinsic goals. Journal of Educational Computing Research, 54(8), 1168–1188.
Lee, Y. M. (2012). Discriminating math lowachievement motivation patterns: comparing disadvantaged and other students in elementary and junior high school. Journal of Research in Education Sciences, 57(4), 39–71. https://doi.org/10.3966/2073753X2012125704002.
Li, M.C., & Tsai, C.C. (2013). Gamebased learning in science education: a review of relevant research. Journal of Science Education and Technology, 22(6), 877–898. https://doi.org/10.1007/s109560139436x.
Liao, C. C., Cheng, H. N., Chang, W. C., & Chan, T. W. (2017). Supporting parental engagement in a BYOD (bring your own device) school. Journal of Computers in Education, 4(2), 107–125.
Lin, B. G., Li, R. P., & Huang, Y. Z. (2009). Instructional manual of mathematical ability test for the schoolaged. Taipei: Ministry of Education.
Lin, P. J., & Tsai, W. H. (2001). Using researchbased cases to enhance prospective teachers’ understanding of teaching mathematics and their reflections. In F. L. Lin (Ed.), Common sense in mathematics education. Proceedings of 2001 the Netherlands and Taiwan Conference on Common Sense in Mathematics Education (pp. 231–272). Taipei: Taiwan.
Liu, T. Y., & Chu, Y. L. (2010). Using ubiquitous games in an English listening and speaking course: impact on learning outcomes and motivation. Computers & Education, 55(2), 630–643. https://doi.org/10.1016/j.compedu.2010.02.023.
McLaren, B. M., Adams, D. M., Mayer, R. E., & Forlizzi, J. (2017). A computerbased game that promotes mathematics learning more than a conventional approach. International Journal of GameBased Learning, 7(1), 36–56.
Ministry of Education. (2003). Guidelines of grades 19 curriculum of elementary and junior high school education. Retrieved from https://www.k12ea.gov.tw/92_sid17/%E6%96%B0%E7%B8%BD%E7%B6%B1%E8%8B%B1%E6%96%87%E7%89%88.pdf.
Mullis, I. V. S., Martin, M. O., Foy, P., & Drucker, K. T. (2012). PIRLS 2011 international results in reading. Chestnut Hill: TIMSS & PIRLS International Study Center, Boston College.
Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 International Results in Mathematics. Retrieved from http://timssandpirls.bc.edu/timss2015/internationalresults/
Nilson, L. B. (2014). The secret of selfregulated learning. In Invited article for Faculty Focus: Higher Ed Teaching Strategies from Magna Publications.
OECD. (2013). PISA 2012 results in focus: what 15yearolds know and what they can do with what they know: key results from PISA 2012.
OECD. (2016). PISA 2015 results in focus. Retrieved from: https://www.oecd.org/pisa/pisa2015resultsinfocus.pdf.
Roll, I., Baker, R. S. J. D., Aleven, V., & Koedinger, K. R. (2014). On the benefits of seeking (and avoiding) help in online problemsolving environments. Journal of the Learning Sciences, 23(4), 537–560.
Schraw, G., Flowerday, T., & Lehman, S. (2001). Increasing situational interest in the classroom. Educational Psychology Review, 13(3), 211–224.
Singh, K. (2011). Study of achievement motivation in relation to academic achievement of students. International Journal of Educational Planning and Administration, 1(2), 161–171.
Taub, M., Azevedo, R., Bouchet, F., & Khosravifar, B. (2014). Can the use of cognitive and metacognitive selfregulated learning strategies be predicted by learners’ levels of prior knowledge in hypermedialearning environments? Computers in Human Behavior, 39, 356–367.
Vélez, J., Fabregat, R., Bull, S., & Hueva, D. (2009). The potential for open learner models in adaptive virtual learning environments. In S. D. Craig & D. Dicheva (Eds.), AIED 2009: 14th International Conference on Artificial Intelligence in Education Workshops Proceedings Volume 8 (pp. 11–20). Brighton: International AIED Society.
Yang, E. F. Y., Cheng, H. N. H., Ching, E., & Chan, T. W. (2012). Variation based discovery learning design in 1 to 1 mathematics classroom. In G. Biswas, L.H. Wong, T. Hirashima, & W. Chen (Eds.), Proceedings of the 20th International Conference on Computers in Education (pp. 811–815). Singapore: AsiaPacific Society for Computers in Education.
Acknowledgements
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors would like to thank the Ministry of Science and Technology of the Republic of China, Taiwan, for financial support (MOST 1062511S008003MY3), and Research Center for Science and Technology forLearning, National Central University, Taiwan.
Availability of data and materials
As a series of subsequent research papers are still in progress, for now, it is temporarily impossible to share research data sets.
Author information
Affiliations
Contributions
CYCY contributed to the study design, data acquisition and analysis, mainly drafted the manuscript and execution project. HNHC was involved in data acquisition, revision of the manuscript and data analysis.ZHC was contributed to the study idea and drafted the manuscript. CCYL of this research was involved in data acquisition and revision of the manuscript. TWC was project manager and revision of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Correspondence to Charles Y. C. Yeh.
Ethics declarations
Authors’ information
Charles Y.C. Yeh is currently an PhD student in Graduate Institute of Network Learning Technology at National Central University. The research interests include onetoone learning environments and gamebased learning.
Hercy N. H. Cheng is currently an associate professor and researcher in National Engineering Research Center for ELearning at Central China Normal University, China. His research interests include onetoone learning environments and gamebased learning.
ZhiHong Chen is an associate professor in Graduate Institute of Information and Computer Education at National Taiwan Normal University. His research interests focus on learning technology and interactive stories, technology intensive language learning and gamebased learning.
Calvin C. Y. Liao is currently an Assistant Professor and Dean’s Special Assistant in College of Nursing at National Taipei University of Nursing and Health Sciences in Taiwan. His research focuses on computerbased language learning for primary schools. His current research interests include a gamebased learning environment and smart technology for caregiving & wellbeing.
TakWai Chan is Chair Professor of the Graduate Institute of Network Learning Technology at National Central University in Taiwan. He has worked on various areas of digital technology supported learning, including artificial intelligence in education, computer supported collaborative learning, digital classrooms, online learning communities, mobile and ubiquitous learning, digital game based learning, and, most recently, technology supported mathematics and language arts learning.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Received
Accepted
Published
DOI
Keywords
 Mathematics learning
 Knowledge map
 Gamebased learning
 Construction management games