Adams, M., Luke, J., Herzig, A., Barlage, J. & Stein, J. (2005). Thinking Through Art: Isabella Stewart Gardner Museum School Partnership Program Year 2 Report.
Abstract:
The Isabella Stewart Gardner Museum and an outside evaluation company, ILI, conducted research on the Gardner’s multiple-visit program for elementary school students, the Museum’s School Partnership Program (SPP). The research examined how participation in the program affected the development of students’ critical thinking skills. The research was carried out over three individual phases, each building on and informed by the previous year’s research findings. The first phase in Year One focused on strengthening programmatic goals. A rubric for defining critical thinking skills was created in this phase, and subsequently refined in other phases. This summary focuses on the subsequent year, Year Two, when the researchers began to assess student outcomes. In this year of the evaluation, the researchers compared the critical thinking skills of students who had participated in the SPP program with a control group of students who had not participated in the program. They found that students who participate in the program showed greater use of critical thinking skills than those in the control group when analyzing and discussing works of art in a museum setting. They found no difference between the groups when analyzing and discussing reproductions of artwork in a school setting.
Key Findings:
- When the researchers compared students’ use of critical thinking skills in a museum setting, they found that students who participated in the School Partnership Program (SPP) used a broader range of the seven critical thinking skills identified by the researchers. Specifically students used the critical thinking skills of analysis, revision, association, and interpretation more often than the control group students.
- Treatment students also used more descriptive language and a higher quality of evidence in discussing art works in the museum setting.
- The researchers found no significant differences between treatment and control students’ use of critical thinking skills in the school setting. They also found no significant difference in standardized comprehensive academic achievement test scores, measuring both English Language Arts and mathematics skills, between treatment and control group students in Year Two. Because critical thinking skills are typically not being tested on such tests, researchers used this as further evidence of the comparability between the treatment and control groups.
Significance of the Findings:
The findings suggest that the School Partnership Program (SPP) was successful in enhancing students’ use of critical thinking skills while analyzing and discussing art in the museum. The rigorous and quasi-experimental design of the study sets it apart from most of the studies being conducted in museum education. This research provides a solid base for the larger museum field on which to better understand critical thinking skills in an art museum multiple visit program and, as a result, apply these findings to the development of school programs that more consciously and deliberately facilitate critical thinking.
Methodology:
The researchers used a quasi-experimental research design to compare the critical thinking skills of students who participated in the Gardner Museum’s School Partnership Program (a treatment group) with those of students who did not participate in the program (a control group). Treatment and control students were in the third, fourth, and fifth grades and attended schools that were comparable in regard to race, socio-economic status, and student test scores. Their schools were considered low to middle income and had a high proportion of students labeled “at-risk.” The control students did not receive regular arts instruction. One hundred and twenty-six students participated in the study.
The researchers used the Critical Thinking Skills rubric developed in the first year of the evaluation to assess the impact of SPP on students’ critical thinking skills. The seven critical thinking skills identified in the rubric are: observation, analysis, revision, association, interpretation, problem-finding, and evaluation. Researchers examined: 1) students’ application of critical thinking skills to other contexts such as classroom settings and standardized tests, 2) students’ use of critical thinking skills in the school context, and 3) students’ use of critical thinking skills in the museum setting. The researchers collected data by conducting open-ended interviews with pairs of students about a reproduction of a work of art at the school site. At the museum site, small groups of six students walked around the museum discussing works of art. To collect data in this site, the unguided conversation was recorded. At each site data was collected from 57 students (32 from the treatment group and 25 from the control group). Researchers also collected student test scores on state-wide standardized tests.
The researchers used a statistical test called ANOVA to analyze student test scores. The interviews and group discussions were transcribed and coded and then scored by trained scorers against the Critical Thinking Skills rubric developed by the research team including museum professionals at the Gardner.
Limitations of the Research:
Although this multi-year, multi-phase study was comprehensive, the researchers may have relied too heavily on the rubrics that they created and revised based on students’ responses in their own program. It is unclear if the rubric would work in another institution with another student population. Department of Education funding required the researchers to focus on low-income, “at-risk” student populations. This research does not tell us if critical thinking skills would improve in middle- to high-income students not at-risk for school failure.
Likewise, it is interesting in Year Two that the site of the student discussions about art (school vs. museum) had an impact on the extent to which students utilized critical thinking skills. Researchers acknowledged that this might imply the need to refine the sensitivity of the rubric and coding system.
Finally, this research does not address what specific components of the SPP led to higher instances of critical thinking in treatment students – was it the consistency of the program, the pedagogy of the museum, follow up by teachers in the classroom, etc.?
Questions to Guide New Research:
How would the increase or decrease of the quantity of visits affect students’ critical skills development? How would students who receive regular in-school visual arts instruction have responded to this study? How would this study be different in a different institution with a different student demographic?