Mathematical equivalence is a relation which is reflexive, symmetrical, and transitive (Jourdain 1912, cited in Asghari 2019). The reflexive property of equality states that a number is always equal to itself, transitivity means that if a = b and b = c then a = c, and symmetry means that a = b is identical to b = a. Mathematics education researchers describe mathematical equivalence as the relationship between two quantities which are the same and interchangeable (e.g. 2 + 3 = 1 + 4) (McNeil 2014). Essien and Setati (2006) concur that the equals sign is used to indicate the relationship between quantities or values in mathematics. Earlier research has shown that primary school students have different interpretations of the equals sign. Some view the equals sign as an operation and interpret it as an arithmetic computation. Such a view is called an operational view of the equals sign (Kieran 1981; Knuth et al. 2006). However, the formal understanding of mathematical equivalence involves seeing the equals sign as a relational symbol that has both sameness and substitutive components (Carpenter et al. 2003; Jones et al. 2012; Kieran 1981). This means that students should be able to recognise the equals sign ‘as same as’ and also as meaning ‘can be substituted for’ (e.g. 31 + 40 = 30 + 1 + 40 and so 30 + 1 + 40 = 31 + 40). Such views of the equals sign are called relational views of the equals sign. Relational views thus constitute two aspects: sameness and substitution.

As discussed above, the equals sign has a variety of meaning and purpose which can be confusing for students, and its meaning is highly dependent on the nature of the task and the tools available (Essien & Setati 2006; Jones & Pratt 2005). However, there are some studies which suggest that students who view the equals sign as an operational symbol perform poorly in algebra compared with those who hold a relational view of the equals sign (Hattikudur & Alibali 2010; Knuth et al. 2006). There is also evidence to suggest that viewing the equals sign relationally and flexibly, which correlates with arithmetic competence, can assist with mastery of algebra and other mathematical concepts in the higher grades (Matthews et al. 2012). Thus, we conclude that although viewing the equals sign operationally can help students solve some arithmetic problems (especially traditional arithmetic problems in operations–equals–answer form, a + b = c), students need to hold a relational view of the equals sign to solve more complex equations (for example non-traditional equations different than equations in operations–equals–answer form, such as a = a, a = b + c, a + b = c + d) and for future algebra learning (see McNeil et al. 2016).

Previously, students’ difficulties with equivalence have been attributed to their immature cognitive skills and logical structures, low working memory capacity and lack of proficiency with basic number facts (Kaye 1986; Kieran 1981; McNeil 2014). However, McNeil and Alibali (2005) argued that the early learning environment could hinder students’ understanding of mathematical equivalence. Based on this, they formulated the change-resistance account. This theory claims that students detect and extract (often subconsciously and incidentally) patterns which they encounter regularly in traditional arithmetic computation. These patterns are called operational patterns. McNeil and Alibali (2005) highlight three operational patterns that students develop through over-practising traditional equations (i.e. a + b = c): (1) students think that all operations should be on the left side of the equals sign, (2) operations should be followed by the equals sign and then by a blank for the unknown and (3) all given operations on all given numbers must be used as the equals sign means to ‘do something’ (McNeil & Alibali 2005).

Although some of these operational patterns can help students solve traditional arithmetic problems, these representations become entrenched and students begin to rely on them as their default representations when they encounter novel mathematics problems later (Machaba & Makgakga 2016; McNeil & Alibali 2005). Moreover, students fail to highlight the interchangeable nature of the two sides of an equation (McNeil et al. 2006; Seo & Ginsburg 2003). All these imply that too much exposure to traditional equations in the early years of schooling can impact students’ future understanding of mathematical equivalence. McNeil and Alibali (2005) contended that in the United States, students learn arithmetic in a procedural fashion for years before they learn to reason relationally about equations. Li et al. (2008) argued that students are seldom challenged in their interpretation of the meaning of the equals symbol in the primary school.

There is evidence showing that students in some countries have better understanding of mathematical equivalence than their peers of the same age in other countries (Capraro et al. 2007, Capraro et al. 2010; Jones et al. 2012). Some other studies within the United States showed that practising non-traditional equations and having conceptual instruction (e.g. emphasis on relational interpretation of the equals sign, using the equals sign in different contexts including symbolic and non-symbolic contexts) can improve understanding of mathematical equivalence in some students (e.g. Baroody & Ginsburg 1983; Jacobs et al. 2007). These findings support the change-resistance account and suggest that given the right circumstances, young students can understand mathematical equivalence. Thus, students should be encouraged to solve various non-traditional equations in different formats, and classroom instruction should support the relational meaning of the equal sign which include sameness and substitution components.

Many studies conducted in Western countries reveal that students understand the equals sign not as relational, but rather as operational, requiring them to ‘work out the answer’ (Baroody & Ginsburg 1983; Knuth et al. 2006). There are some studies which show that South African students also have difficulties with viewing the equals sign as a relational symbol and understanding mathematical equivalence. For example, the study by Essien and Setati (2006) explored grades 8 and 9 students’ interpretations of the equals sign in the South African context and found that grades 8 and 9 students viewed the equals sign as a tool to compute the answer rather than a relational symbol to compare the quantities. Machaba and Makgakga (2016) revealed that grade 9 students in one participating school interpreted the equals sign as a ‘do something’ and unidirectional (one-sided) sign, not as the concept that represents an equivalent (concept of keeping both sides of the equals sign equal) of two quantities. Results from the Vermeulen and Meyer (2017) study of grade 5 and 6 students’ understanding of equivalence indicated that few students had a well-developed relational understanding of the equals sign and many could not describe the meaning of the equals sign correctly. Machaba and Makgakga (2016) attributed misinterpretation of the equals sign to how students had been taught the concept of equivalence at lower grades, where greater emphasis was placed on rules rather than the meaning of a concept. Essien (2009) argued similarly in his study of the introduction of the equals sign in grade 1 textbooks. He claimed the textbook examples introduced the equals sign within the context of addition and subtraction entrenching the ‘understanding of the equals sign as an operational rather than symbolic symbol’ (Essien 2009:29). The notion of equivalence was embedded in the operations, making it difficult for students to develop a relational understanding of the concept. He also highlighted the lack of emphasis of equivalence in Foundation Phase curriculum documents which is also true of the current Curriculum and Assessment Policy Statement (Department of Basic Education 2011) and has implications for teaching and learning.

There have been many studies over the past 35 years looking at students’ understanding of equivalence. The Mathematical Equivalence Knowledge Construct Map designed by Rittle-Johnson et al. (2011:3) provides a ‘representation of the continuum of equivalence knowledge’ broken into different levels. In this map, there are four continuous levels. Each level describes what students are likely to do at that level, such as how they perform on specific types of equations and how they interpret the equals sign (see Table 1). However, these levels should not be interpreted as discrete because the literature has shown that students may develop different interpretations of the equals sign synchronously (e.g. Jones et al. 2012), or their equation-solving performance might also depend on the other features of the problems rather than traditional versus non-traditional equations dichotomy (e.g. Hornburg, Wang & McNeil 2018). Matthews et al. (2012) used this Construct Map to develop a valid and reliable assessment to measure students’ understanding of equivalence. The researchers developed the Mathematical Equivalence Assessment (MEA) and this has been frequently used in research focusing on students’ understanding of equivalence (e.g. Fyfe et al. 2018). Not only does the Construct Map depict how most of students develop a sophisticated understanding of mathematical equivalence, but it also provides a useful basis for researchers who desire to develop instruments to reliably measure students’ understanding of equivalence.

We used this Construct Map for the adaptation of the MEA to our study to use the MEA with South African students aged 9–10 years old. We sampled items from different levels of the Construct Map to be able to provide a comprehensive picture regarding a group of South African grade 4 students’ understanding of mathematical equivalence in six schools in the Western Cape.

Despite the vast literature investigating students’ understanding of mathematical equivalence in Western countries in early grades of primary schooling, whether students’ difficulties with equivalence, reported in the literature, are common in South African context has been unclear. To fill this gap in the literature, this study examined grade 4 students’ understanding of mathematical equivalence in South Africa with a particular focus on their interpretations of the equals sign. Through this study, we also aimed to compare our findings from a sample of grade 4 students with the findings of three other studies conducted with grade 5 and 6 as well as grade 8 and 9 South African students (Essien & Setati 2006; Machaba & Makgakga 2016; Vermeulen & Meyer 2017) to see if the same pattern of results of students’ understanding of mathematical equivalence occurs.

A total of 410 grade 4 primary school students, from six different primary schools across the Western Cape, volunteered to participate in the study. The schools are classified quintiles 4 and 5 which means that they are fee paying schools and all have English as their language of learning and teaching (LoLT) from grade 1. Two of the schools have a mixture of students with either Afrikaans or isiXhosa as their home language. The test was administered in the middle of the grade 4 academic year and nine of the students from the group were excluded from the analyses, as there were more than 1/3 of the assessment items unanswered. This left us with 401 students (M_age = 9.91, SD = 0.61, ranging from 9 to 12). This is the largest sample of grade 4 students that have been tested for understanding of equivalence in South Africa to date, as far as we are aware.

All students gave written consent to participate in the study, and parent consents were obtained. The mathematics teacher of each class and the head teacher were informed about the purpose and procedure of this research.

The students were given a paper-and-pencil assessment to complete within their usual lesson time, under test conditions. We used the MEA developed by Rittle-Johnson et al. (2011) and Matthews et al. (2012) in this study. The MEA was formed with 27 items involving three types of items: (1) open-ended equation-solving items, (2) True/False equation-structure items and (3) definition items. These items were developed to assess students’ interpretations of the equals sign, how they perform on different types of equations and whether they use shortcuts to solve equivalence problems.

We made a few adaptations to the original assessment. There were a few reasons for that: (1) we expected grade 4 students to have higher abilities than past international samples, which also include grade 2 and grade 3 students, (2) we used a shorter version of the original assessment for practicality, (3) we increased the difficulty of items by increasing the size of numbers in some questions to make them suitable for our sample (e.g. instead of 7 + 6 = 6 + 6 + 1, we used 70 + 60 = 60 + 60 + 10), (4) we used the substitution definition of the equals sign in one of the questions (i.e. the equals sign means ‘the two sides can be swapped’) (similar to Jones et al. 2012), and (5) were placed letters, used as unknowns, in the questions with boxes given that some students might not have been taught letters as unknowns. Finally, we should note here that there was another difference in the equation-solving items between the original MEA and the adapted version of the MEA that we used in the present study. In the original assessment, Matthews et al. (2012) encouraged students to find a shortcut to solve problems by stating that ‘You can try to find a shortcut so you don’t have to do all the adding’. We did not ask students to use shortcuts as we were interested in students’ spontaneous use of solution strategies. We asked students to show their working for all equation-solving problems.

The assessment instrument used in the present study consisted of 15 items: 4 True/False equation-structure, 6 definition and 5 equation-solving items. Figure 1 depicts examples from the final form of the student assessment for each question type. Through the equation-structure items, we measured whether students were able to judge correctness of an equivalence statement, and which non-traditional equations they did not accept. Definition items assessed students’ interpretations of the equals sign. Finally, equation-solving items measured students’ performance on non-traditional equivalence problems and whether they exploited comparative relational strategies (see Table 1 for the definition of comparative relational strategies) to solve these questions.

The assessment was completed by students on a whole-class basis, within their usual lesson time, under test conditions, and was administered by their regular classroom teachers. To ensure that assessment administration was identical across the schools, we provided detailed information to the teachers in relation to testing procedure.

We used two different scoring schemes to code students’ responses to the assessment items. Firstly, students’ responses to the items were scored for correctness irrespective of the solution method (i.e. 1 for a correct answer and 0 for an incorrect answer). We marked blank items as incorrect unless a participant left more than one third of the assessment blank (more than 5 items). In that case, we excluded the participant altogether as mentioned previously. In the second scoring scheme, for five open-ended equation-solving items, the students received a point if they mentioned the equality relation between values on the two sides of the equation. In other terms, students received a point if they use a ‘comparative relational strategy’ (e.g. 17 + 29 = _ + 28, if students mention that the answer is 18 because 28 is 1 smaller than 29, so the answer should be one larger than 17). We call these strategies relational strategies hereinafter. According to the Construct Map, the use of relational strategies shows that students move to the final and fourth level in their development of equivalence knowledge.

We excluded one of the definition items (item 4b) from the analyses because of this item being used as a distractor (see Jones et al. 2012). The performance of the instrument and coding activity for the rest of the items was checked by estimating internal consistencies. Cronbach’s alpha was 0.715 for all of 14 items for whole sample when the items were scored as correct/incorrect. Cronbach’s alpha was 0.746 for five relational codes for five equation-solving items. These Cronbach’s alpha values are acceptable. We checked the factor structure of the assessment using Mplus 8.1 software, and found that three-factor model had the best-fit indices (similar to Fyfe et al. 2018; Matthews et al. 2012). The 14 assessment items loaded positively onto their relevant factors that were definition, equation-structure and equation-solving.

Approval to conduct this study was received from the Education Faculty Ethics Committee, Ethical Clearance number: EFEC 6-8/2018, 19 August 2018.

We first calculated definition, equation-solving and equation-structure scores for each student, based on students’ accuracy on each item type. We then calculated an overall score for each student based on these three scores. Next, we calculated a relational score for each student, based on our relational codes. Table 2 depicts the descriptive statistics regarding these scores. The results demonstrate that there was good variance in performance on each sub-scale and overall assessment except for use of relational strategies. The mean score for relational showed that students rarely used relational strategies.

Students’ definitions of the equals sign: We asked the students to rate the different definitions of the equals sign. Sixty-six per cent of whole sample rated the sameness definition (i.e. the equals sign means the same as) as good, and 36% of whole sample rated the substitution definition (i.e. the equals sign means the two sides can be swapped) as good. As expected, the number of students who endorsed the substitution definition of the equals sign was lower than those who endorsed the sameness definition. Moreover, 82% of the students who endorsed the substitution definition already rated the sameness definition as good.

In terms of the operational definition, 84% of whole sample rated ‘the equals sign means the answer to the problem’ as good. This shows that most of the students viewed the equals sign operationally. Moreover, 26% (n = 103) of the whole sample did endorse neither the sameness nor the relational definition but endorse only the operational definition, whereas 58% (n = 233) of whole sample endorsed at least one of the relational definitions together with the operational definition. This implies that just above one fourth of the students viewed the equals sign only as an operational symbol.

Three hundred and sixty-one students made a valid decision about the best definition amongst the provided definitions. Fifty-nine per cent of these students chose ‘the answer to the question’ as the best, 29% of them chose ‘the same as’ as the best, and 12% of them chose the substitution definition as the best definition. According to this, most of the students were inclined to think that the operational definition best explains the meaning of the equals sign, and less than half of the students recognised relational definition of equals sign as the best definition.

In terms of generating a definition, only 44% of the participating students provided a relational definition of the equals sign (e.g. such as ‘the same as’, ‘equal to’, and ‘equivalent to’) to at least one of the following two questions ‘What does the equals sign ‘=’ mean? Does it mean anything else?’ This is consistent with the results above that 41% of the students chose one of the relational definitions of the equals sign as the best definition. Choosing and generating a relational definition of the equals sign appear to be necessary to move to Level 3 and further on the Construct Map.

Equation-structure and equation-solving performance, and relational strategy use: We found that 99% of whole sample answered at least one of four equation-structure questions correctly, and 53% of those answered all four questions correctly. These findings illustrate that the students performed successfully on equation-structure problems. This suggested that most of the students fulfilled one of the requirements of level 3 of the Construct Map, which is evaluating successfully mathematical sentences in non-traditional format.

Unlike their high scores on equation-structure items, students performed poorly on equation-solving items. Twenty-nine per cent of whole sample did not answer any of five items correctly, 28% answered only one item correctly, and only 2% of whole sample answered all equation-solving questions correctly. Many of the answers showed that students held an operational view of the equals sign as they interpreted the equals sign as an operation to ‘work out an answer’ (see Figure 2).

When considering relational strategy use, there seemed to be little variance in the scores as the students rarely used relational strategies. Eighty-four per cent of the participating students never used a relational strategy. Relational-strategy use is necessary for the Levels 3 and 4 of the Construct Map. Thus, for these questions, these students currently did not show the knowledge to move them to Level 3 or 4 of the Construct Map.