- Research
- Open Access

# A framework of generating explanation for conceptual understanding based on “semantics of constraints”

- Tomoya Horiguchi
^{1}Email author, - Takahito Tomoto
^{2}and - Tsukasa Hirashima
^{3}

**10**:2

https://doi.org/10.1007/s41039-015-0002-4

© The Author(s) 2015

**Published:**23 June 2015

## Abstract

In science education, conventional problem practice hardly helps students reach “conceptual understanding” with which they can solve various problems by making appropriate models of target systems. Students often superficially read the solution of a problem and apply it wrongly to others without understanding the model. It is difficult to teach how to make appropriate models because model-making expertise includes a lot of implicit knowledge. In this paper, we propose a general framework for systematically describing such knowledge, which makes it possible not only to explain various models and the difference between them but also to design/sequence a set of problems appropriate for promoting conceptual understanding. Our framework was proved useful through a preliminary experiment in which the explanations generated based on our framework promoted subjects’ (15 graduates and undergraduates) conceptual understanding in mechanics. The framework can be the basis for designing intelligent tutoring systems which explicitly help students reach conceptual understanding.

## Keywords

- Science education
- Problem practice
- Conceptual understanding
- Explanation generation
- Semantics of constraints

## Introduction

In science education, it is one of the most important goals for students to acquire the ability to make an appropriate model of the target system and its behavior for a given task. The method most frequently used for achieving this goal is the “problem practice,” in which students are required to make an appropriate model for answering the query in each of a set of problems. They need to identify the structure/state of the target system and the applicable principles/laws for modeling the given situation. We call such ability “conceptual understanding” of a domain. (Note that models are not limited to mathematical ones because not a few problems are solved with non-mathematical models such as qualitative models).

However, it is difficult to reach such an understanding with conventional problem practice. In conventional problem practice, after learning some worked examples, students solve a set of problems one by one. Explanations about the solution mainly focus on the calculation of the required amount from the given ones and rarely focus on the reason why the solution is possible, whether and why the solution is/is not applicable to other problems. Problems are rather arbitrarily sequenced, and the relations between (the solutions of) problems are rarely explained. In such a practice, there are many students who wrongly apply the solution they previously learned (in one problem) to another in which the solution is inapplicable. There are also many students who cannot apply the solution they previously learned to another in which the solution is applicable. Even experience in many problems does not easily improve such difficulties (Bransford et al. 2000; VanLehn 1998; VanLehn and van de Sande 2009). Novice students often generate a naive representation of a problem focusing on its superficial features (called “surface structure” (Chi et al. 1981; Hirashima et al. 1994)). They cannot generate the representation of a problem based on its structural features (called “physical structure”) (Chi et al. 1981; Larkin 1983, 1985). Therefore, instead of applying principles/laws to make the model appropriate for each problem, they often try an inappropriate solution, such as to apply the solution of a problem to another (just as it is) based on their superficial similarity (VanLehn 1998) or to use a general strategy for operating mathematical equations without considering their physical meanings (Larkin 1981). This is mainly because they sometimes succeeded in solving problems with an inappropriate solution by accident in their past practice.

In order to reach conceptual understanding, students therefore need to learn (1) to infer the structural features of problems from the superficial features and (2) to apply appropriate principles/laws to structural features to make models necessary for solving problems. For assisting them, in problem practice, it is necessary to explain not only how each problem is solved but also why the solution is possible and what physical meaning it has. That is, it must be explicit why the principles/laws are applicable to the given situation (i.e., surface structure) and what physical meaning (physical structure) they imply. Additionally, it is important to explain not only the solution of a problem but also the relation (difference) between problems, that is, how the solution (applicable principles/laws) changes when the situation (problem) is changed. Furthermore, it would promote such learning to provide students with an appropriately designed and sequenced set of problems (Scheiter and Gerjets 2002, 2003; VanLehn and van de Sande 2009).

In conventional problem practice, such instruction has been rarely focused on, at most given by a few (experienced) teachers individually and implicitly. Especially, there have been few intelligent tutoring systems (ITS) which can explain the relation between arbitrary two problems and adaptively sequence problems considering the learning effect of order. We think this is because most of the knowledge necessary for such instruction is implicit and difficult to systematize; therefore, there has been no general framework for indexing various types of problems.

In this paper, we propose a general framework for indexing problems. The framework helps authors index problems. Based on such indices, the explanation and sequence of problems mentioned above can be automatically generated. In our framework, making a model in physics is regarded as a process in which various constraints (applied principles/laws and modeling assumptions) are imposed on the target system and its behavior. A model is regarded as the set of constraints. We first formulate the model-making process in physics, then analyze the constraints which compose a model to systematize them based on their physical meanings and roles (functions). After that, we describe the applicable conditions of principles/laws in physics as a set of constraints. The constraints classified/defined in this manner are easily assigned to the situation of a problem. There are also some groups of constraints which are “exclusive” from each other (i.e., cannot be valid simultaneously). Therefore, based on such classification and exclusiveness of constraints, it becomes possible to explain what physical meaning (structural features) superficial features of a problem have, what principles/laws are applicable to them, and how applicable principles/laws change when the situation is changed. By indexing problems with this framework (we call it “semantics of constraints: SOC”), it becomes possible to automatically extract the “differences between problems” necessary for the comparison and sequencing of problems.

The technical contributions of this paper are that a “system of concepts” is developed with which the expertise in solving physics problems can be explicitly described and that an explanation generator is developed which automatically generates explanations based on such descriptions. It is well known that “knowledge engineering” is necessary to represent experts’ knowledge on computers. Ikeda et al. pointed out such engineering (they call it “ontology engineering”) is also important in designing educational systems (Ikeda et al. 1999). We identified various types of modeling assumptions and other constraints (also the relations among them) which constitute models of physical systems and behaviors. Such a system of concepts (or ontology) works as a “conceptual tool” (Ikeda et al. 1999) which assists/guides authors in describing teaching materials (in our case, experts’ model-making process). Without our framework, it would be difficult for authors to describe such expertise explicitly.

The SOC framework originally arose from our research on “error-based simulation (EBS)” (Hirashima et al. 1998; Horiguchi and Hirashima 2001, 2006; Horiguchi et al. 2014). EBS is an educational simulation which is generated with students’ erroneous formulation (i.e., model) of physical systems (Hirashima et al. 1998). In EBS, students observe the behavior of systems which is unnatural/different from the correct one, so they are much motivated to correct their model. That is, EBS works as a counterexample to students’ erroneous ideas (Horiguchi and Hirashima 2001). Here, how the behavior is different from the correct one is important because students do not become aware of their errors if the difference is small. We therefore developed a set of heuristics with which it is judged whether an EBS is useful or not for error awareness (Hirashima et al. 1998; Horiguchi and Hirashima 2001). We also developed a simulator which can identify the constraint violation in the students’ erroneous model and relax some constraints (if necessary) to make the model calculable (Horiguchi and Hirashima 2006). The simulator (called “robust simulator”) uses a set of heuristics for judging what constraint(s) should be relaxed to generate (the most) educationally useful EBS. EBS has been proved useful for correcting students’ misconception through practical use in the classroom (Horiguchi et al. 2014). While the above heuristics were domain-dependent and had the limitation on their applicable area, the SOC framework is the generalization/elaboration of them. Recently, we used the framework to generate feedback to erroneous models the students made in a model-building learning environment (Horiguchi et al. 2012). By using the framework, what constraints are violated in students’ erroneous models is explained and how to correct their models is suggested.

We first discuss the required knowledge and assistance necessary for conceptual understanding based on current research, then introduce the SOC framework. After that, we show the method for generating SOC-based explanations. The results of the preliminary experiment are described which proved the usefulness of our framework. Finally, we conclude this paper and mention our future work.

## Conceptual understanding and assistance

Research on problem solving has revealed the knowledge structure domain experts (Chi et al. 1981; Larkin 1981, 1983, 1985; VanLehn 1998; VanLehn and van de Sande 2009) in science have. Experts can infer the structural features of problems with scientific concepts from the superficial features and generate the representation to which formal operations are applicable. They can also generate an appropriate plan for solving the problem by operating the representation with the knowledge about qualitatively interpreted principles/laws. It is supposed that experts have acquired such knowledge by inducing the essential features through comparison of many problems and by transforming them into (some kinds of) “schemata” or “production rules” (VanLehn and van de Sande 2009). It is, however, difficult for students to reach such an understanding through conventional problem practice. Even instructional innovations based on recent learning science research have limitedly improved students’ understanding (Bransford et al. 2000).

In order to promote such knowledge acquisition, it is effective to appropriately design a set of problems which includes positive/negative examples of various problem categories and “near misses” (near misses are minimally contrasting pairs of examples: one is positive and the other is a negative example of a category, and they differ in only one critical feature), and to provide them in an appropriate order to students (VanLehn and van de Sande 2009) (in fact, it is reported that problem order greatly influences learning (Scheiter and Gerjets 2002, 2003)). In order to do that, it is necessary to explicitly describe the superficial/structural features of problems and their relations, and qualitative interpretations of principles/laws and their means of application. However, since most of such knowledge is implicit, there has been no general framework to systematically describe such knowledge. We think this is the reason of the following fact: Though knowledge structure necessary for expertise was revealed and effective instructional methods were proposed, they have not been widely practiced. The framework we propose makes it possible to systematically describe such knowledge, based on which the design of a set/sequence of problems and explanation generation for promoting conceptual understanding become possible.

## Semantics of constraints

Given a physics problem (which consists of a physical system and query), one makes a model necessary and sufficient for answering the query by embodying an appropriate part of the domain theory. (A “necessary and sufficient” model means that it includes the information necessary for solving the problem and that it does not include the unnecessary (too detailed) information.) Domain theory (Choueiry et al. 2005; Conati and VanLehn 1996; VanLehn and van de Sande 2009) consists of a set of propositions, each of which describes a principle/law, its applicable condition, and resulting constraint(s) on the attribute(s) of the system. Constraints by embodied principles/laws are called the “physical phenomenon constraints (PPCs).”

In making a model, various modeling assumptions are set for selecting appropriate principles/laws. Modeling assumptions define the structure/behavioral range of a system and physical phenomena to be considered. Since embodied physical phenomenon constraints are valid under some modeling assumptions, applicable conditions of principles/laws can be described with a set of modeling assumptions. That is, a PPC always has its corresponding modeling assumptions. Constraints by modeling assumptions are called the “modeling assumption constraints (MACs).”

Boundary condition of a system is given by the “boundary condition constraints (BCCs).” They define the influence from the outside of the system. Making the influence which cannot/need not be calculated with a model given means defining the boundary of the model (i.e., what physical processes are considered/ignored). That is, a BCC always has its corresponding modeling assumptions.

In our framework, a model is the union of physical phenomenon constraints, boundary condition constraints, and modeling assumption constraints. Usually, only the first two constraints are written as a model while the last constraints are remained implicit. However, MACs give the validity to PPCs and BCCs. When modeling assumptions are changed, physical phenomena and boundary conditions also qualitatively change. In order to make a model correctly, it is therefore necessary to understand the physical meaning of the constraints based on modeling assumptions (i.e., why an assumption is set and what role it plays). In most cases, such knowledge is acquired by a few students. In this research, we develop a framework for describing such knowledge explicitly, based on which the function for promoting conceptual understanding is designed. In the following three subsections, we elaborate on each class of constraints to systematize their physical meanings and relations.

### Modeling assumption constraints (MACs)

Modeling assumption constraints define the physical processes considered/ignored in a model. They are classified in two ways from different viewpoints: structural and functional.

Classes of modeling assumption constraints (from structural viewpoint)

Subclasses | Subsubclasses | Definition | Examples | |
---|---|---|---|---|

MAC | Physical structure constraint (PSC) Decision about the perspective and granularity in modeling | Physical object constraint (PO) | Specifies what kind of objects in a physical system are considered | - Consider two blocks in contact as they are or as one |

- Consider two parallel-connected springs/resistors as they are or a compound one | ||||

Physical attribute constraint (PA) | Specifies what kind of relations/attributes of objects in a physical systems are considered | - Consider a block’s net-force or its electric resistance | ||

- Consider the friction between two objects in contact or not | ||||

Operating range constraint (ORC) Decision about the behavioral range of the model | Physical range constraint (PR) | Specifies the range (state space) within which the model is valid by using physical attributes | - A model of two blocks’ motion where one pulls another thru a string assuming the string is taut | |

- A model of a constant resistance assumes its current and voltage are within the proportional range | ||||

Conceptual range constraint (CR) | Specifies the range (state space) within which the model is valid by using conceptual attributes | - A model of a block ( |

Classes of modeling assumption constraints (from functional viewpoint)

Subclasses | Subsubclasses | Definition | Examples | |
---|---|---|---|---|

MAC | Process consideration constraint (PCC) Decision about what kind of physical processes are considered in modeling | Process selection constraint (PS) | Specifies what kind of processes in a physical system are considered/ignored | - Consider (rel-friction( |

- Ignore the change of form/mass of objects which collided PO | ||||

- Consider the heat exchange between two objects ( | ||||

Out-sourcing constraint (OS) | Ignores a physical process by putting it out of the system and regarding its effect as a boundary condition | - An outer tank which supplies water infinitely PR | ||

- An outer power supply which always supplies 5 V PR | ||||

- Initial velocity | ||||

Black-boxing constraint (BB) | Ignores a physical process by putting it in a black box and regarding its effect as a boundary condition | - Consider two parallel-connected springs/resistors as they are or a compound one PO | ||

- Consider two blocks in contact moving with internal force as one PO/CR | ||||

Physical world constraint (PWC) Necessity for maintaining the fundamental law of the physical world | Maintains the fundamental law of the physical world | - Rigid objects never overlap CR | ||

- Mass >0, | ||||

- | ||||

Process simplifying constraint (PFC) Convenience for mathematical calculation with the model | Substitutes the simplified process for a complicated (original) process | - Consider the range within which a resistance’s voltage and current are proportional PR | ||

- Consider the behavior of a pendulum with small amplitude as simple harmonic oscillation PR |

Constraint classes from the structural viewpoint are useful for enumerating modeling assumptions because they rather suggest the components and their relations of a system. For example, when a variable in an equation stands for a physical quantity, it is easy to infer an object and its attribute corresponding to the quantity is considered (which are physical structure constraints). Constraint classes from the functional viewpoint are useful for considering the meaning of modeling assumptions because they rather suggest the process structure (processes considered and their relations). For example, considering/ignoring a physical attribute (which is a physical structure constraint) suggests a physical process concerning the attribute is considered/ignored (which is a process selection constraint). That is, the classes from the structural viewpoint rather concern the surface structure of a problem, while the classes from the functional viewpoint rather concern its physical structure. Furthermore, as shown above, the classes from both viewpoints are related to each other based on their physical meanings. Therefore, with these classifications, it becomes possible to systematically describe the knowledge about the relation between superficial and structural features of problems.

Additionally, there are often sets of modeling assumption constraints which are mutually exclusive (cannot be assumed simultaneously). For example, in the same time interval, “transient state” and “steady state” (which are operating range constraints) cannot be assumed simultaneously. In the same (part of a) system, “consider friction” and “not consider friction” (which are process consideration constraints) cannot be assumed simultaneously. Such exclusiveness between modeling assumptions gives important clues to identify the differences between models/problems (see the Procedure section).

### Physical phenomenon constraints (PPCs)

A relatively simpler physical phenomenon constraint is the “physical device constraint” which arises within a component of a system. That is, it is a “local constraint.” Since it indicates the physical property of the component, each domain has its specific physical device constraints (for example, Ohm’s law constrains the values of current and voltage in an electric device). In contrast, there are “global constraints” which indicate the behavior of multiple components or the whole system. Global constraints are classified as follows.

Classes of physical phenomenon constraints

Subclasses | Subsubclasses | Definition | Examples | |
---|---|---|---|---|

PPC | Dynamic change constraint (DYC) | Differential change constraint (DC) | Specifies the behavior of a physical system while it is dynamically changing (with differential expression) | - |

- | ||||

Integral change constraint (IC) | Specifies the behavior of a physical system while it is dynamically changing (with integral expression) | - | ||

- | ||||

- | ||||

Steady state constraint (SSC) | Specifies the behavior of a physical system in a steady state | - ∑ | ||

- | ||||

Discontinuous change constraint (DCC) | Specifies the behavior of a physical system when it changes discontinuously | - ( | ||

- | ||||

Conservative law constraint (CLC) | Specifies the amount of a quantity which is conserved over the time | - (1/2) | ||

- | ||||

- |

The dynamic change constraint constrains the behavior of a system in a time interval during which it is changing dynamically. It has two subclasses: “differential change constraint” and “integral change constraint.” The former often indicates the relation between the *driving power* of dynamic change and the influences on it, and is represented with differential expression. For example, Newton’s second law (equation of motion) relates an object’s acceleration with the forces applied to it. The latter usually includes a time variable and describes the temporal/integral effect of the driving power, and is represented with integral expression. An example is the expression of linear accelerated motion. The steady state constraint constrains the behavior of a system in a time interval during which it is in a steady state. It indicates the balance/cancelation between influences on the driving power of dynamic change. An example is the equation of balance of forces about an object at rest. The discontinuous change constraint constrains the behavior of a system at a time point on which it changes discontinuously. It indicates the relation between the amounts of a quantity before and after the change. An example is the formula of coefficient of restitution. A quantity is called a “conserved quantity” when its amount is constant during the temporal evolution of a system. The conservation law constraint indicates the amounts of a conserved quantity at arbitrary two time points are the same. An equation of heat exchange between two objects and an equation of conservation of energy/momentum are the examples.

A global physical phenomenon constraint aggregates a set of local physical phenomenon constraints. For example, Newton’s second law (equation of motion), which is a dynamic change constraint in mechanics, includes a set of local constraints, each of which indicates a force applied to the target object (physical device constraints such as elastic force, friction). Such inclusion relation between PPCs gives important clues to identify the dominant principle(s)/law(s) in solving a problem.

Additionally, there are often sets of physical phenomenon constraints of which modeling assumptions (preconditions) are mutually exclusive. These PPCs are never simultaneously valid in the same state of the same system. For example, since “static friction” and “kinetic friction” have exclusive preconditions (operating range constraints) about a contact surface of two objects, they are never valid simultaneously at the same surface. The first three global PPCs (i.e., dynamic change, steady state, and discontinuous change constraints) are exclusive for the same reason. They often entirely change each other when preconditions are changed. For example, suppose a mechanical system is in a steady state by assuming “friction” which cancels other forces. When the assumption is changed to “frictionless,” the system can dynamically change. Such exclusiveness between PPCs gives important clues to identify the differences between models/problems (see the Procedure section).

### Boundary condition constraints (BCCs)

Since the outside of the structure and behavioral range of a system defined with modeling assumptions are not modeled, it is necessary to define the influence from the outside at the boundary in order to calculate the behavior of the system. BCCs give such influence, that is, the boundary values of a set of physical attributes (of objects, relations, and their compound amounts). Therefore, considering BCCs gives important clues to understand the role of corresponding modeling assumptions in defining the system boundary.

Classes of boundary condition constraints

Subclasses | Definition | Examples | |
---|---|---|---|

BBC | Constant boundary constraint (CBC) | Specifies the boundary condition (initial value) of temporally constant attribute | - Mass( |

- g-acc(GF) = 9.8 [m/s/s] | |||

- In ( | |||

- Resistance( | |||

- Compound-capacitance( | |||

Variable boundary constraint (VBC) | Specifies the boundary condition (initial value) of temporally variable attribute | - vel( | |

- Compressed( | |||

- on( | |||

- Variable-resistance( |

Additionally, a BCC sometimes suggests the behavior of a system is assumed to be within a specified behavioral range (operating range constraint). For example, when considering the motion of an artificial satellite, its initial velocity is usually given as it maintains the circular orbit around the earth.

## Explanation generation

### Framework of model-making process description

In our framework, each principle/law is described as a “model fragment” (Falkenhainer and Forbus 1991) which consists of its applicable condition and its consequence(s). An applicable condition is described as a set of modeling assumption constraints, while a consequence is described as a physical phenomenon constraint. A model consists of the union of PPCs given by instantiated model fragments, MACs giving applicable conditions for them, and BCCs given in a problem. (Note that an instantiated “model fragment” is distinguished from “model fragment class” which describes a principle/law itself.) A model-making process (i.e., solution) is described as the procedure in which model fragments are applied (instantiated) in turn to the situations (represented with MACs and BCCs) to yield new consequences (represented with PPCs). (Note that a consequence of a model fragment can be the condition for others.)

For example, in Fig. 1a, the constraint “the block b1 is on the inclined floor p1 (on-floor(b1, p1))” is explicitly described. It is an important condition under which the normal force (*n-force(b1)*) is exerted on the block by the floor. In contrast, in Fig. 1c, since this constraint is not described, it cannot be inferred that the solution becomes invalid when the inclination of the floor exceeds 90° (i.e., the block is no longer “on the floor”). For another example, the constraint on the initial velocity of the block (*velocity(b1) = 0*) is explicitly described in Fig. 1a while it is not in Fig. 1c. With the latter description, it cannot be inferred that the solution becomes invalid when the initial velocity of the block becomes greater than zero. These constraints are rarely explicitly described because they are usually implicit assumptions. Our framework can prompt the authors of the problem/solution to make such constraints explicit.

Additionally, a pair/set of model fragment classes which have similar conditions (situations) but have exclusive MAC(s)/PPC(s) as applicable condition(s)/consequence(s) is called “exclusive model fragment classes.” Grouping such model fragment classes helps the comparison between models.

For example, the model fragments “static friction” in Fig. 1a and “kinetic friction” in Fig. 1b are mutually exclusive because their applicable conditions (assumptions) “the net-force of the block in the tangent direction to the inclined floor is smaller than the maximum static friction” and “the net-force of the block in the tangent direction to the inclined floor is greater than the maximum static friction” are exclusive. Note that the constraint on the value of the coefficient of static friction (*s-cof(b1, p1) = μ*
_{
s
}
*’*) is described in Fig. 1b. It is not necessary for calculating the required amount, but is necessary for judging the block does descend the inclined floor. In the usual description of solution in Fig. 1d, since such a constraint is not explicitly described, it cannot be related to the other solution in Fig. 1c. Therefore, it cannot be inferred what happens when the value of the coefficient of static friction increases.

### Procedure

#### Explanation of the model-making process (solution)

The description of model-making process mentioned above makes it possible to explain why/how each principle/law is applied by explicitly referring to its modeling assumptions. In Fig. 1b, for example, the formula *v*
^{2} − *v*
_{0}
^{2}
*=* 2*ax′* (dynamic change constraint) is used which relates an object’s displacement, velocity, and acceleration in a time interval. Note that the constraint “acceleration is constant in the interval” (operating range constraint) is explicitly described which is an important precondition for this model fragment to be applied. Many students wrongly use this formula when an object’s acceleration temporally varies. The explanation explicitly referring to modeling assumptions would be helpful in avoiding such mistakes.

#### Explanation of the difference between models (problems)

The difference between models (problems) can be inferred by comparing their model fragments. There are two types of relations between problems: (1) the problems which have the same/similar surface structures (situations) but have different physical structures (instantiated model fragments belong to different classes) and (2) those which have different surface structures (situations) but have the same/similar physical structures (instantiated model fragments belong to the same classes). Both relations play an important role for promoting conceptual understanding (Scheiter and Gerjets 2002, 2003). As for the latter, the difference is easily inferred by identifying the corresponding pair of model fragments (each of which belongs to each model) both of which give the (global) PPCs of the same class. The difference can be explained by showing their preconditions (situations) are different.

As for the former, the difference is inferred by identifying the corresponding pair of model fragments (each of which belongs to each model) which belong to exclusive model fragment classes. Since their situations are similar but their modeling assumption constraint(s) and physical phenomenon constraint(s) are exclusively different, they indicate the difference of two models before/after the change of the situation. The type of the difference can be explained by referring to their modeling assumption classes. For example, when two corresponding model fragments have the same physical structure constraints and exclusively different operating range constraints, it is inferred that the difference of the two models is change of the operating range about the partial system they match. The model fragments “static friction” in Fig. 1a and “kinetic friction” in Fig. 1b are in such relation. It can be inferred that the local constraint between a block and slope is changed from “static friction” to “kinetic friction” by changing the operating range, by which the global constraint “balance of forces” (steady state constraint) is changed to “Newton’s second law” (dynamic change constraint) (the generated explanation is shown in Fig. 2c).

## Preliminary experiment

### Design

We conducted an experiment to evaluate the usefulness of our framework. A SOC-based explanation generator was implemented. The purpose was to examine whether the SOC-based explanation promotes students’ conceptual understanding, that is, whether their representation of problems was improved and they became able to solve various types of problems by using correct models.

#### Subjects

Fifteen graduates and undergraduates whose majors are engineering participated.

#### Instruments

(1) Two sets of problems in elementary mechanics: They were called “problem set 1 (PS-1)” and “problem set 2 (PS-2).” Each set included 15 problems of various surface/physical structures. Problems might have similar situations but different solutions, or have different situations but similar solutions. The sets had no common problem. (2) Usual explanation about the solutions of 11 problems in PS-1: The calculation of the required physical amount from the given ones was mainly explained. (3) SOC-based explanation about the solutions of the same problems as usual explanation: In addition to the solution of each problem, the differences between problems were explained about eight pairs of problems which had similar surface/physical structures. (4) Explanation generator used for generating SOC-based explanation: Model-making processes described by the experimenter (first author) were input and their explanations were output, which were rewritten into readable natural language by the experimenter (without changing the point).

#### Procedure

First, subjects were given PS-1 and asked to group the problems into some categories based on some kind of “similarity” they suppose (any number/size of categories were allowed), then asked to label each category they made (called “categorization task 1”). After that, they were asked to solve eight problems in PS-1 (called “pre-test”). After a week, the subjects were divided into two groups: one was the “control group” (seven subjects) and another was the “experimental group” (eight subjects). The average scores of both groups in the pre-test were made equivalent. The subjects in control group were given the usual explanation and asked to learn it. The subjects in the experimental group were given the SOC-based explanation and asked to learn it. After that, by using PS-2, “categorization task 2” was conducted in the same way as above. Finally, subjects were asked to solve eight problems in PS-2 (called “post-test”).

#### Measure

The quality of the representation of problems was measured with the categories, their “frequencies” (number of problems accounted for), and the time required in each categorization task. The ability to solve various types of problems was measured with the scores in each test. The effect of learning with usual/SOC-based explanation on the quality of representation and the ability of problem solving was measured with the comparison of the results of two categorization tasks and pre-/post-tests. The superiority of the SOC-based explanation to the usual explanation was measured with the differences of improvement of categorization and problem solving between experimental and control groups.

## Results

Categories in task-1

Number of subjects using category labels ( | Average size of category ( | Number of problems accounted for ( | Number of problems wrongly accounted for ( | Number of problems correctly accounted for ( | |
---|---|---|---|---|---|

| 12 | 3.1 | 37 | 2 | 35 |

| 9 | 4.1 | 37 | 2 | 35 |

| 12 | 2.0 | 24 | 0 | 24 |

| 12 | 1.9 | 23 | 1 | 22 |

| 3 | 5.7 | 17 | 1 | 16 |

| 7 | 2.0 | 14 | 0 | 14 |

| 5 | 2.2 | 11 | 0 | 11 |

Balance | 5 | 2.4 | 12 | 4 | 8 |

Object only | 1 | 5.0 | 5 | 0 | 5 |

Friction | 3 | 1.7 | 5 | 0 | 5 |

Second law | 2 | 2.5 | 5 | 2 | 3 |

Pulleys | 1 | 2.0 | 2 | 0 | 2 |

Balance of energies | 1 | 4.0 | 4 | 2 | 2 |

Motion of weight | 1 | 2.0 | 2 | 0 | 2 |

Energy of falling object | 1 | 3.0 | 3 | 2 | 1 |

Linear uniform motion | 1 | 1.0 | 1 | 0 | 1 |

Simple harmonic motion | 1 | 1.0 | 1 | 0 | 1 |

Terminal velocity | 1 | 1.0 | 1 | 0 | 1 |

Internal force | 1 | 5.0 | 5 | 5 | 0 |

Others | 1 | 15 | 15 | 2 | 13 |

Categories in task-2 (usual)

Number of subjects using category labels ( | Average size of category ( | Number of problems accounted for ( | Number of problems wrongly accounted for ( | Number of problems correctly accounted for ( | |
---|---|---|---|---|---|

| 4 | 4.5 | 18 | 0 | 18 |

| 4 | 3.3 | 13 | 0 | 13 |

| 3 | 3.7 | 11 | 0 | 11 |

Conservation of energy | 3 | 6.0 | 18 | 9 | 9 |

Second law | 3 | 3.7 | 11 | 2 | 9 |

Pulley and string | 2 | 3.5 | 7 | 0 | 7 |

Circular motion | 4 | 1.5 | 6 | 0 | 6 |

Pendulum | 3 | 1.7 | 5 | 0 | 5 |

Simple harmonic motion | 2 | 2.0 | 4 | 1 | 3 |

Collision | 2 | 1.0 | 2 | 0 | 2 |

Inertial force | 1 | 1.0 | 1 | 0 | 1 |

Friction | 1 | 1.0 | 1 | 0 | 1 |

Others | 2 | 4.0 | 8 | 1 | 7 |

Categories in task-2 (SOC)

Number of subjects using category labels ( | Average size of category ( | Number of problems accounted for ( | Number of problems wrongly accounted for ( | Number of problems correctly accounted for ( | |
---|---|---|---|---|---|

| 7 | 4.4 | 31 | 5 | 26 |

| 7 | 3.6 | 25 | 1 | 24 |

| 8 | 4.1 | 33 | 12 | 21 |

Linear accelerated motion | 3 | 3.3 | 10 | 2 | 8 |

Conservation of momentum | 3 | 1.3 | 4 | 1 | 3 |

Acceleration | 1 | 3 | 3 | 0 | 3 |

Springs | 1 | 3 | 3 | 0 | 3 |

Pulleys | 1 | 3 | 3 | 0 | 3 |

Simple harmonic motion and period | 2 | 1 | 2 | 0 | 2 |

String and tension | 1 | 2 | 2 | 0 | 1 |

Time | 1 | 2 | 2 | 0 | 2 |

Friction | 1 | 1 | 1 | 0 | 1 |

Pendulum | 1 | 1 | 1 | 0 | 1 |

Collision | 2 | 1 | 1 | 0 | 1 |

*t*test

*p*> .10). In the post-test, though there was also no significant difference of average scores between groups (control group 42.7 and experimental group 47.6,

*t*test

*p*> .10), the increase of the average score of the experimental group was larger than that of the control group. This result suggests that the learning with the SOC-based explanation promoted the ability to solve various types of problems, that is, to make appropriate models regardless of their superficial features.

These results suggest that the SOC-based explanation about the solution of problems and their differences can assist students in reaching conceptual understanding.

## Conclusion

Aiming at promoting conceptual understanding through problem practice, we proposed the SOC framework, based on which the knowledge necessary for designing a set of problems, sequencing them, and generating explanations can be described.

Modeling assumptions play an important role in modeling because they give the physical meanings of the other constraints in a model (i.e., physical phenomenon and boundary condition constraints). Clarifying their relations makes it possible to generate schema-based explanation about a solution and the difference between solutions. As we discussed in the Explanation Generation section, the explanation generated by current ITS (Conati and VanLehn 1996; VanLehn et al. 2005; VanLehn and van de Sande 2009) does not focus much on modeling assumptions, or does so at most in an indirect/ad hoc way. (The research by Hirashima et al. (1994) was the pioneering work which tried to make such implicit knowledge explicit in ITS, but its domain was limited to mechanics.) This is because there was no framework for systematically describing modeling assumptions and the other constraints focusing on their roles in models/solutions. Giving such a framework is our contribution.

The importance of modeling assumptions in modeling was first indicated in the research on “compositional modeling” (Choueiry et al. 2005; Falkenhainer and Forbus 1991; Weld and deKleer 1989) in artificial intelligence. The purpose of compositional modeling is to automatically generate the model of a physical system by combining a set of model components (called model fragments) based on the modeling assumptions. While some important assumptions are identified in the research, we elaborated and systematized them into a framework especially aiming at education. Instead of automatic model generation, we enabled the description of problems/solutions with which explanation about experts’ problem solving process is automatically generated. We also enabled the design of a set of problems with which an appropriate sequence of problems can be automatically generated.

Automatic model generation by compositional modeling is not sufficient for automatic explanation generation in education. That is, in automatic model composition, an inference chain is generated which starts with the preconditions (given facts) and ends with the derived consequences (required amounts). However, sequencing the steps in the chain simply in one direction (either forward or backward) results in the explanation which is different from human experts’ explanation. As discussed in the Explanation Generation section, experts first identify the dominant principle/law in solving a problem, then apply other principles/laws to derive its conditions. That is, the explanation goes back and forth. Our explanation generator enables such an explanation by using the SOC-based description of solution. In fact, the description of the solution in our framework is “redundant” as an inference chain of automatic model composition. For example, in Fig. 1 (a), though the fact “acceleration(b1) = 0” is linked to the model fragment “balance of forces,” this fact is not necessary for the automatic application of this model fragment (only the fact “net-force(b1) = 0” is necessary). However, this link plays an important role in explanation generation because it represents experts’ insight “balance of forces is the dominant principle of the solution.” Since such a link must be written manually, it is our important future work to assist the authors in describing such information based on the SOC framework.

Such “redundant” information is also important for designing a set of problems. That is, for promoting conceptual understanding, it is important to show a small/large difference in surface features between problems does/does not lead to the difference in their solutions. Such critical features are often represented as the information which is redundant from the viewpoint of automatic model composition. For example, in Fig. 1a,b, what principle/law is dominant is identified not with the feature “net-force(b1) =/> 0” but with the feature “acceleration(b1) =/> 0” (the former is necessary for applying the model fragment “balance of forces/Newton’s second law” while the latter is not). Additionally, the difference between the (local) model fragments “static friction” and “kinetic friction” arises not because of the difference of the values of coefficients of friction but because of the difference of the dominant principle/law. Such information (i.e., “redundant” links and the weight (importance) of facts/model fragments) is important for comparing/designing a set of problems from the viewpoint of education, while it is not addressed in compositional modeling.

We showed the explanations generated with our framework could promote conceptual understanding through a preliminary experiment. Of course it is preferable to implement an authoring system which assists/guides authors in describing model-making processes with our framework. It is our future work. The SOC-based explanation generator can provide a basic function for designing various instructional methods (e.g., a detailed explanation is gradually simplified (scaffolding-fading), a sequence of problems is given which promotes spontaneous induction). Design of such instructional methods and verification of their effectiveness are also our future work. Additionally, as mentioned in the Introduction section, the SOC framework is developed based on the experience of developing/using intelligent tutoring systems in practice. Therefore, it has the potential to work as a conceptual basis in designing various intelligent functions of educational systems. Exploring such potential is our important future work.

## Declarations

### Acknowledgements

This work is supported in part by Grant-in-Aid for Scientific Research (B) (No. 26280127) from the Ministry of Education, Science, and Culture of Japan.

## Authors’ Affiliations

## References

- Bransford, JD, Brown, AL, & Cocking, RR (Eds.). (2000).
*How people learn: brain, mind, experience, and school*. Washington, D.C.: National Academy Press.Google Scholar - Chi, MTH, Feltovich, PJ, & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices.
*Cognitive Science, 5*, 121–152.View ArticleGoogle Scholar - Choueiry, BY, Iwasaki, Y, & McIlraith, S. (2005). Towards a practical theory of reformation for reasoning about physical systems.
*Artificial Intelligence, 162*, 145–204.View ArticleGoogle Scholar - Conati, C, & VanLehn, K. (1996).
*POLA: a student modeling framework for Probabilistic On-Line Assessment of problem solving performance*(Proc. UM-96, pp. 75–82).Google Scholar - Falkenhainer, B, & Forbus, KD. (1991). Compositional modeling: finding the right model for the job.
*Artificial Intelligence, 51*, 95–143.View ArticleGoogle Scholar - Hirashima, T, Niitsu, T, Hirose, K, Kashihara, A, & Toyoda, J. (1994). An indexing framework for adaptive arrangement of mechanics problems for ITS.
*IEICE Trans. Inf. & Syst., E77-D*(1), 19–26.Google Scholar - Hirashima, T, Horiguchi, T, Kashihara, A, & Toyoda, J. (1998). Error-based simulation for error-visualization and its management.
*International Journal of Artificial Intelligence in Education, 9*(1–2), 17–31.Google Scholar - Horiguchi, T, & Hirashima, T. (2001).
*The role of counterexamples in discovery learning environments: awareness of the chance for learning*(Notes of the 1st Int. Workshop on Chance Discovery, pp. 5–10).Google Scholar - Horiguchi, T, & Hirashima, T. (2006).
*Robust simulator: a method of simulating learners’ erroneous equations for making error-based simulation*(Proc. ITS2006, pp. 655–665).Google Scholar - Horiguchi, T, Hirashima, T, & Forbus, KD. (2012).
*A model-building learning environment with error-based simulation*(Proc. QR2012).Google Scholar - Horiguchi, T, Imai, I, Toumoto, T, & Hirashima, T. (2014). Error-based simulation for error-awareness in learning mechanics: an evaluation.
*Journal of Educational Technology & Society, 17*(3), 1–13.Google Scholar - Ikeda, M, Hayashi, Y, Lai, J, Chen, W, Bourdeau, J, Seta, K, & Mizoguchi, R. (1999).
*An ontology more than a shared vocabulary*(Proc. AIED99 Workshop on Ontologies for Intelligent Educational Systems, pp. 1–10).Google Scholar - Larkin, JH. (1981). Enriching formal knowledge: a model for learning to solve textbook physics problems. In JR Anderson (Ed.),
*Cognitive skills and their acquisition*. Hillsdale, N.J.: Lawrence Erlbaum Associates.Google Scholar - Larkin, JH. (1983). The role of problem representation in physics. In D Gentner & AL Stevens (Eds.),
*Mental models*. Hillsdale, N.J.: Lawrence Erlbaum Associates.Google Scholar - Larkin, JH. (1985). Understanding, problem representations, and skills in physics. In SF Chipman, JW Segel, & R Glaser (Eds.),
*Thinking and learning skills, vol.2: research and open questions*. Hillsdale, N.J: Lawrence Erlbaum Associates.Google Scholar - Scheiter, K, & Gerjets, P. (2002).
*The impact of problem order: sequencing problems as a strategy for improving one’s performance*(Proc. CogSci2002).Google Scholar - Scheiter, K, & Gerjets, P. (2003). Sequence effects in solving knowledge-rich problems: the ambiguous role of surface similarities.
*Proc Cognitive Science, 2003*, 1035–1040.Google Scholar - VanLehn, K. (1998). Analogy events: how examples are used during problem solving.
*Cognitive Science, 22*(3), 347–388.View ArticleGoogle Scholar - VanLehn, K, Lynch, C, Schulze, K, Shapiro, JA, Shelby, R, Taylor, L, Treacy, D, Weinstein, A, & Wintersgill, M. (2005). The Andes physics tutoring system: lessons learned.
*International Journal of Artificial Intelligence in Education, 15*(3), 147–204.Google Scholar - VanLehn, K, & van de Sande, B. (2009). Acquiring conceptual expertise from modeling: the case of elementary physics. In KA Ericsson (Ed.),
*The development of professional performance: toward measurement of expert performance and design of optimal learning environments*(pp. 356–378). Cambridge, UK: Cambridge University Press.View ArticleGoogle Scholar - Weld, DS, & deKleer, J (Eds.). (1989).
*Readings in qualitative reasoning about physical systems*. San Mateo, CA.: Morgan Kaufmann.Google Scholar

## Copyright

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.