The effect of playing chess on problem solving was explored using Rasch scaling and hierarchical linear modelling. Subjects were 508 students from Grades 6 - 12 in an Australian Independent boys school, with a strong tradition in the game of chess. Of these 508 students, 64 were regular players of competitive chess. Data from the Australian Schools Science Competition were Rasch scaled and placed on a single scale for all the grades. Multilevel analysis using hierarchical linear modelling was employed to test the effects of the hypothesised variables. No significant effect of the playing of chess on the scholastic performance was found, suggesting that previous results showing positive effects may have been due to other factors such as general intelligence or normal development. It is suggested that this combination of Rasch scaling and multilevel analysis is a powerful tool for exploring such areas where the research design has proven difficult in the past.
Chess enthusiasts have long argued that the playing of chess leads to improved scholastic attainment and greater self-confidence. It is suggested that the playing of chess develops skills of creative thinking, critical thinking and the ability to concentrate and to solve problems. Certainly, there is no doubt that playing competitive chess demands considerable concentration skills. Even at the junior chess level, games are often as long as three hours. Playing chess also demands an ability to project possible positions of pieces and so could help develop visual and spatial abilities. Similarly, chess demands skills of logical thinking. It might be argued that these sorts of skills and abilities should transfer to other scholastic areas. The famous chess master Kasparov has promoted chess in schools and a number of programs have emerged which have been fostered by the various chess associations, particularly in the United States. In an interview reported by Harrell (1999), LaFreniere, the Coordinator of the Washington Chess Federation Scholastic Chess Program said,
Chess is the single most powerful educational tool we have at the moment, and many school administrators are realising that. (Harrell, 1999 on net)Although not always widely published beyond chess circles, there have many research projects on the effects of chess on student performance in the classroom. Frank (1974) explored the relationship between playing chess and other scholastic abilities. He found that there was a significant correlation between chess playing and spatial and numerical abilities and that there was a positive correlation between playing chess and the change in numerical and verbal aptitudes. Christiaen (1976) randomly divided 40 fifth grade students into experimental and control groups. The students were given a number of tests of cognitive development at the end of their fifth grade studies and again at the end of their sixth grade studies. The experimental group received 42 one-hour chess lessons. The tests at the end of both the fifth and the sixth grade showed significant differences in favour of the chess group.
Ferguson (1983) studied the effects of chess treatment and computer treatment on groups of academically gifted students in Grades 7 to 9. The chess treatment group showed a significant difference in the growth of originality. Horgan (1987) advocated the teaching of chess as a means of developing a child's intellect. Ferguson (1987) showed that a group of sixth grade students who had not previously played chess, showed significant development in memory and reasoning skills when they played chess daily over a period of nine months. Margulies (1991) found that elementary pupils, who played chess, showed a significant improvement in reading ability when compared to their non-chess playing fellows. Gaudreau (1992) has reported that in a group of fifth grade students, those who had their mathematics instruction enriched with chess, developed significantly better problem solving abilities than those who had received a normal mathematics program. Much of this material has been summarised by Dauvergne (2000).
In summary, it is argued that chess can be an intrinsically motivating learning tool. Performance at chess cannot be blamed on anyone else and students must accept the consequences of their actions. They must develop skills of planning, of problem solving, evaluating a wide range of alternatives, concentration, and self-discipline. It is clear that the devotees of the game of chess are convinced of its worth as a powerful educational tool. They argue that it is a simple and cheap means of helping students to develop important cognitive skills.
It seems that the traditional pre-test and post-test experimental designs have led to results which while encouraging, have not been conclusive and need further support. The difficulties in experimental design make it desirable to use statistical control, employing regression analysis procedures, to take into account the effects of other factors.
This study seeks to investigate whether there is any effect associated with chess playing in a group of students for whom some data are readily available. This study seeks to avoid the problems mentioned above by the use of statistical control in order to distil out the effects of the other factors. It does this by using data from the Australian Schools Science Competition which is Rasch scaled and then uses hierarchical linear modelling to explain this data in terms of the other variables, including the playing of chess.
It is hypothesised then that students who play chess regularly should perform better in the Australian Schools Science Competition than those who do not, when controlling for the other variables that may be involved.
A number of computer programs have been developed to assist with the analysis of data from test items. One of these is the QUEST program (Adams and Siek-Toon Khoo 1993), which allows the results of tests to be analysed to determine whether they fit the Rasch model and provides estimates, both of the abilities of the students and the difficulties of the test items. In a recent study (Thompson 1998), it was found that the Australian Schools Science Competition data fit the Rasch model well, allowing the estimates of the item difficulties and the student abilities to be plotted on the same scale. The Rasch Scaling process allows the possibility of placing the tests at each of the grade levels on the same scale, thereby allowing direct comparisons between the grade levels. The study by Thompson (1998) showed that it is possible to put the results from the different grade levels on the same scale using concurrent equating, which provides good agreement with the expected results. This involves scoring all of the items and subjects at the one time, relying on the common items to establish the difficulty levels of all of the items across the range. This method was tested in comparison to other equating procedures by Mohandas(1998) and concurrent equating provided good agreement with the expected results. It is therefore possible to put the items and subjects from all the grade levels onto one scale.
These hierarchical linear models are discussed by Bryk and Raudenbush (1992), Raudenbush and Bryk (1997), and Keeves and Sellin (1997). Such models allow a researcher to postulate and subsequently to test statistical hypotheses associated with relationships between the outcome variable and the factors that may affect it. In hierarchical linear models, the researcher can examine the effect of the various factors, both within and between individuals and at the group level and any possible interactions between them. The outcome variable is represented as a function of the various characteristics. Thus in the example of the Science Competition, the outcome variable is the Rasch scaled score of the individual and the variables of IQ and chess playing can become level one variables in an hierarchical model.
First, there is the b etween student within the class group equation
Y_{ij} = _{j0} + _{j1}(IQ) + _{j2}(chess) + r_{ij} (1)In equation (1) Y_{ij} represents the performance of student i in group j and _{j0} represents the baseline performance. Each of the coefficients represents the extent to which the performance of a student is affected by the variable in the brackets. The coefficient _{j1} represents the effect of student IQ and the variable is the measured IQ of the student, whilst _{j2} represents the effect of playing chess and its associated variable is a dichotomous variable indicating whether the student plays chess or not. The term r_{ij} represents the random error. An important feature of hierarchical linear models is that these coefficients will vary from student to student.
At the second or macro level of a hierarchical linear model, the coefficients in the Level 1 equation are expressed as an outcome variable in a linear equation of Level 2 variables at the second or between class group level. For example, the coefficient _{j1}, the effect of IQ on the performance of student i, may be expressed as a function of grade level. Likewise, the intercept _{j0} may be expressed as a function of grade level and other treatment conditions.
Thus, a researcher may build a model as follows in equations (2) and (3), as a between class group equation
_{j0} = _{00} + _{10}(grade) + _{10}(other treatment) + u_{j0} (2)It can be seen then that a layered or hierarchical model is being employed. The values of the various coefficients need to be estimated using the data available from the Australian Schools Science Competition. Recent advances in computational technology make such estimations possible. One program which does this by an iterative method using empirical Bayes estimation procedures based on maximum likelihood estimates is HLM, developed by Raudenbush and Bryk (1996). With this facility, it is possible to estimate the effects of the various parameters and their inter-relationships at each of the levels of the hierarchical linear model._{j1} = _{01} + _{11}(grade) + _{11}(other treatment) + u_{j1} (3)
It follows then, that it may be possible using a hierarchical linear model, to partition out the effects of the variables such as IQ, together with the effect of playing chess, which is of interest in this study, and to estimate the effect of each of these variables on student performance.
The initial model that was explored was as follows in equation (4).
Y_{ij} = _{j0} + _{j1}(IQ) + _{j2}(chess) + r_{ij} (4)In this Level 1 model, the outcome variable (the Rasch scaled performance ability score) is expressed as a function of IQ, and playing chess. At Level 2, the model sought to explain the coefficients at Level 1 in terms of factors associated with the grouping of the subjects as shown in equations (5) and (6).
_{j0} = _{00} + _{10}(grade) + _{10}(other treatment) + u_{j0} (5)In each case, the other treatment was exploring whether the grouping of the students in their classes had any effect on the outcome._{j1} = _{01} + _{11}(grade) + ; _{11}(other treatment) + u_{j1} (6)
and so on.
This model was improved by the elimination of variables that did not prove to have a significant effect. The final model was as follows in equations (7), (8), (9) and (10). Level-1 model
Y = B_{0} + B_{1}*(IQ) + B_{2}*(CHESS) + R (7)In this Level 1 model, the outcome variable Y, the Rasch scaled performance scores measured by the Science Competition test are equal to an intercept or base level B0, plus a term that expresses the effect of IQ, with its associated slope, B1, and a term which expresses the effect of playing chess and its associated slope B2. There is also an error term R. Thus the outcome variable Y is explained in terms of IQ and involvement in chess at Level 1.
In the Level 2 model, the effect of the Level 2 variables on each of the B terms in the Level 1 model is given in equations (8), (9) and (10). Level-2 Model
B_{0} = G_{00} + G_{01}*(GRADE) + U_{0} (8)Thus in equation (8), the constant term B_{0} is expressed as a function of Grade, with an associated slope G_{01}. Values of each of these terms are estimated and the level of statistical significance evaluated to assess the effect of each of the terms.B_{1} = G_{10} + U_{1} (9)
B_{2} = G_{20} + U_{2} (10)
Initially, the HLM program makes estimates of the various values of the slopes and intercepts, using a least squares regression procedure and then using an iterative process improves the estimation using a maximum likelihood estimation and the empirical Bayes procedure. Table 1 shows the reliability estimates of the Level 1 data.
Random Level-1 coefficient | Reliability estimate |
INTRCPT1, B_{0} | 0.664 |
IQ, B_{1} | 0.324 |
CHESS, B_{2} | 0.019 |
Table 2 shows the least -squares regression estimates of the fixed effects.
Fixed Effect | Coefficient | Standard Error | T-ratio | Approx degrees of freedom | P-value | |
For INTRCPT1, B_{0} | INTRCPT2, G_{00} | -1.645 | 0.178 | -9.221 | 504 | 0.000 |
GRADE, G_{01} | 0.217 | 0.200 | 11.034 | 504 | 0.000 | |
For IQ slope, B_{1} | INTRCPT2, G_{10} | 0.040 | 0.002 | 19.089 | 504 | 0.000 |
For CHESS slope, B_{2} | INTRCPT2, G_{20} | 0.120 | 0.091 | 1.323 | 504 | 0.186 |
Table 3 shows the final estimations of the fixed effects.
Fixed Effect | Coefficient | Standard Error | T-ratio | Approx degrees of freedom | P-value | |
For INTRCPT1, B_{0} | INTRCPT2, G_{00} | -1.572 | 0.327 | -4.81 | 20 | 0.000 |
GRADE, G_{01} | 0.208 | 0.057 | 5.69 | 20 | 0.000 | |
For IQ slope, B_{1} | INTRCPT2, G_{10} | 0.036 | 0.003 | 13.67 | 21 | 0.000 |
For CHESS slope, B_{2} | INTRCPT2, G_{20} | 0.056 | 0.091 | 0.619 | 21 | 0.542 |
Table 4 shows the final estimation of the variance components.
Random Effect | Standard Deviation | Variance Component | df | Chi-square | P-value |
INTRCPT1, U_{0} | 0.222 | 0.049 | 15 | 52.09 | 0.000 |
IQ slope, U_{1} | 0.007 | 0.000 | 16 | 29.84 | 0.019 |
CHESS slope, U_{2} | 0.053 | 0.003 | 16 | 20.48 | 0.199 |
Level-1, R | 0.606 | 0.367 |
In order to calculate the amount of variance explained by the model, a null model, with no predictor variables was formulated. The estimates of the variance components for the null model are shown in Table 5.
Random Effect | Standard Deviation | Variance Component | df | Chi-square | P-value |
INTRCPT1, U_{0} | 0.602 | 0.362 | 21 | 357.7 | 0.000 |
Level-1, R | 0.749 | 0.561 |
Using the data from Tables 4 and 5, the amount of variance explained is calculated as follows:
Variance explained at Level 2 | = | 0.362 - 0.049 0.362 | = | 0.865 |
Variance explained at Level 1 | = | 0.561 - 0.367 0.561 | = | 0.346 |
In addition, the intraclass correlation can be calculated.
= | _{00} ----------- _{00} + ^{2} | = | 0.362 ----------------- 0.362 + 0.561 | &nbs p; = | 0.392 ---------------- 0.362 + 0.561 |
This intraclass correlation represents the variance within groups compared to the total variance between and within groups. Thus the model is explaining 33.9 per cent (0.392 x 0.865) of the variance in terms of grade levels. The remaining 21.0 per cent ((1 - 0.392) x 0.346) is explained as the variation brought about by IQ and the playing of chess. In all 54.9 per cent of the variance in scores is explained by the model and 45.1 per cent is unexplained.
Of particular interest in this study is the value G_{20}. This represents the effect of playing competitive chess on the performance abilities of the students. It suggests that, taking into account the effects of IQ and grade level, students who play chess competitively, are performing at a level of 0.056 of a logit better than others, when controlling for the other variables of grade and IQ. This is approximately equivalent to one quarter of a year's work. However this result was not found to be significant. One possible explanation of this lack of significance is that the playing of chess has contributed to the individual student IQ and so the benefits of playing chess have been absorbed into the IQ variable.
This study has examined a connection between the playing of chess and the cognitive skills involved in problem solving. The results have not shown a significant effect of the playing of chess on the scholastic achievement of the students, when controlling for IQ and grade level.
Bryk, A. S. & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Beverly Hills, Ca: Sage.
Bryk, A. S., Raudenbush, S. W., & Congdon, R. T. (1996). HLM for Windows version 4.01.01. Chicago: Scientific Software.
Charness, N., Kampe, R. & Mayr, U. (1996). The role of practice and coaching in entrepreneurial skill domains: An international comparison of life-span chess skill acquisition. In K. A. Ericsson (ed), The road to excellence. The acquisition of expert performance in the arts and sciences, sports and games. Mahwah NJ: Erlbaum.
Chase, W. G. & Simon, H. A. (1973). The mind's eye in chess. In W. G. Chase (ed), Visual information processing. New York: Academic Press.
Christiaen, J. (1976). Chess and cognitive development. Unpublished paper available from U.S. Chess Federation http://www.uschess.org/scholastic/sc-research.html
de Groot, A. D. (1965). Thought and choice in chess. The Hague: Mouton.
de Groot, A. D. (1966). Perception and memory versus thought. In B. Kleinmuntz (ed), Problem-solving. New York: Wylie.
Dauvergne, P. (2000). The case for chess as a tool to develop our children's minds [viewed 26 July 2000, verified 6 Jan 2004]. http://www.auschess.org.au/articles/chessmind.htm
Ferguson, R. (1983). Developing critical and creative thinking through chess. Unpublished paper available from U.S. Chess Federation http://www.uschess.org/scholastic/sc-research.html
Ferguson, R. (1987). The development of reasoning and memory through chess Unpublished paper available from U.S. Chess Federation http://www.uschess.org/scholastic/sc-research.html
Frank, A. (1974). Chess and aptitudes Unpublished paper available from U.S. Chess Federation http://www.uschess.org/scholastic/sc-research.html
Elo, A. E. (1986). The rating of chess players, past and present (2nd ed.). New York: Arco.
Ericsson, K. A. (1996). The acquisition of expert performance. In K. A. Ericsson (ed), The road to excellence. The acquisition of expert performance in the arts and sciences, sports and games. Mahwah NJ: Erlbaum.
Ericsson, A., Tesch-Römer, C. & Krampe, R. (1990). The role of practice and motivation in the acquisition of expert-level performance in real life. In M. J. A. Howe (ed), Encouraging the development of exceptional skills and talents. Leicester: BPS.
Ericsson, K. A. & Lehmann, A. C. (1996). Expert and exceptional performance: evidence of maximal adaptation to task constraints. Annual Review of Psychology, 47, 273-305.
Faulkner, J. (ed) (1991). The Best of the Australian Schools Science Competition. Rozelle, NSW: Science Teachers' Association of New South Wales.
Gaudreau (1992). Étude comparative sur ls apprentissages en mathematics 5e anné Unpublished paper available from U.S. Chess Federation http://www.uschess.org/scholastic/sc-research.html
Harrell, D. C. (1999). Schools try a new gambit: Chess to boost young minds. In Seattle Post-Intelligencer, 31 May 1999. [verified 6 Jan 2004] http://www.seattlep-i.com/local/ches31.shtml
Horgan, D. (1987). Chess as a way to teach thinking. Unpublished paper available from U.S. Chess Federation http://www.uschess.org/scholastic/sc-research.html
Keeves, J. P.and Sellin, N. (1997). Multilevel analysis. In J. P. Keeves, (ed), Educational research, methodology and measurement (2nd ed.), Oxford: Pergamon, pp. 3978-3987.
Margulies, S. (1991). The effect of chess on reading scores. Unpublished paper available from U.S. Chess Federation http://www.uschess.org/scholastic/sc-research.html
Mohandas, R. (1998). Test equating, problems and solutions: Equating English test forms for Indonesian junior secondary school final examinations administered in 1994. Unpublished Masters Thesis, The Flinders University of South Australia.
Raudenbush, S.W. & Bryk, A. S. (1997). Hierarchical linear models. In J. P. Keeves, (ed), Educational research, methodology and measurement (2nd ed.), Oxford: Pergamon, pp. 2590-2596.
Raudenbush, S. W. & Bryk, A. S. (1996). HLM Hierarchical linear and nonlinear modeling with HLM/2L and HLM/3L programs. Chicago: Scientific Software.
Snyder, S. & Sheehan, R. (1992). Research methods. The Rasch measurement model: An introduction. Journal of Early Intervention, 16(1), 87-95.
Thompson, M. J. (1998). The Australian Schools Science Competition - A Rasch analysis of recent data. Unpublished paper, The Flinders University of South Australia.
Weiss, D. J. and Yoes, M. E. (1991). Item response theory. In Hambleton, R. K. & Zaal, J. N. (eds), Advances in educational and psychological testing. Boston: Kluwer, pp. 69-95.
Author: Murray Thompson is a part-time Doctor of Education student at Flinders University. As well he is an experienced secondary school teacher of Physics, Science and Mathematics at Prince Alfred College in Adelaide where he has held a number of positions of responsibility, including pastoral care, curriculum leadership and professional development. His research interests have revolved around educational measurement and cognitive psychology. Email contact address: dtmt@senet.com.au
Please cite as: Thompson, M. (2003). Does the playing of chess lead to improved scholastic achievement? Issues In Educational Research, 13(2), 13-26. http://www.iier.org.au/iier13/thompson.html |