||Volume 19, Number 5, March 2007
The great debate on school measurement
by Janet Amsden
Peter Cowley, director of School Performance Studies at the Fraser Institute and Dr. Paul Shaker, dean of education at Simon Fraser University argued on the topic of school measurement at the Newlands Golf Club in Langley on January 18. Organized by the BC Association for Supervision and Curriculum Development, the event was classic debate with each speaker giving timed statements and rebuttals.
Peter Cowley opened saying that schooling is a service and taxpayers need access to standardized results so that they can compare schools. Paul Shaker countered with "Real science is not present in the Fraser Institute testing process" and suggested alternatives to the Fraser Institute’s BC School Report Card. Shaker’s alternatives, which involved external evaluations, do not meet Cowley’s criteria for school measurement; data collection that includes all schools, is easily reportable, and allows parents to compare schools.
They clashed on the topic of low scores for schools. Shaker called the practice demoralizing; Cowley saw a zero score as a catalyst for improvement. He told the audience he has been on the receiving end of tears, anger, and frustration from some "very good people."
"It’s hard," he admitted.
That the two men viewed the question from different beliefs was apparent in their language throughout the discussion. Shaker used the language of the research facility and Cowley that of the marketplace.
Shaker said the Fraser Institute reports fail to meet scholarly standards because no hypothesis is tested, no peers review the process, no experts are consulted, and tests are used for purposes for which they were not intended. He criticized Cowley for not including factors such as parent education, numbers of ESL, and students with special needs in the calculation of school scores. (Those data are presented but are not part of the score computation.)
Cowley rebutted, "We could do that...but I have an issue about whether we should modify the data because parents’ education, etc. will not factor into a prospective employer’s decision to hire a student, or a university’s decision to admit a student."
When asked what he would do if he were minister of education Cowley mused, "I would like to open things up a bit, to organizations and people who would try things differently." In this context he made a distinction between the term ‘public education’ and ‘education of the public.’
In response to a request to include a confidence interval for the school rankings, one that would show how far apart scores need be to have any meaning, Cowley replied that they decided not to include estimates of error. Instead, they chose to report results over a five-year period so that parents can see the results over time.
"Cherry picking," was Paul Shaker’s response. "That’s why we have peer review."
Shaker was also critical of The Vancouver Sun for ‘distorting the value of the data to the public by devoting 29 pages to the report card. He argued that the Fraser Institute encourages ‘phenomenological primitives’—that people attach themselves to primitive notions about schools generated from fragmented knowledge (see Phi Delta Kappan, March 2005) and that the press has responsibility to prevent this. He commended The Sun for responsible journalism in its recent unmasking of the RCMP’s ‘scientific criticism’ of Vancouver’s safe injection site as opinion.
Although the audience, mainly teachers, administrators, and trustees appeared more supportive of Shaker’s arguments, Cowley summed up the outcome, saying, "We agree to disagree on the benefits to ranking to students." It was clear that their disagreement was deep.
Janet Amsden teaches at Fairview Elementary School, Maple Ridge.
Quotes from the debate:
"Rankings crystallize focus on schools and energize them." (Peter Cowley)
"Reducing school scores to a single indicator is the ‘Enronization’ of education... It demoralizes people. It is wrong, it is corrupt!" (Paul Shaker)
"The Province issue that compares schools is in its second highest seller every year." (Peter Cowley)
"The question is whether or not we should rank school rather than whether or not we should measure them. We agree to disagree on the benefit to students of whether we should rank schools." (Peter Cowley)
"The medium is the message. Measuring or ranking is not a cure-all, but if you are going to do ranking, at least do it with scientific accuracy." (Paul Shaker)