|
Home > Web 2.0 > Research Blog
Permanent linkThe New York Times has published a very good summary of the L.A. Times database controversy and of the further use of value-added modeling as a means of evaluating teachers in the U.S.
The Los Angeles Times has compiled excerpts from comments that teachers posted when they had the initial opportunity to review their scores, before the database was made public. It is interesting to note that of the teachers quoted here, most of those who express concern with the database are ranked highly on the Times' effectiveness scale. The full comments left by teachers can be found tied to their individual pages within the searchable database.
And further to my post of August 30th, another recent study - this one from the U.S. Department of Education - questions the merits of using value-added models to evaluate teachers:
Permanent linkSunday also saw the release of a new study by the Economic Policy Institute, co-authored by an impressive list of names:
Problems With the Use of Student Test Scores to Evaluate Teachers
EPI Briefing Paper #278
August 29, 2010
Co-authored by scholars convened by the Economic Policy Institute: Eva L. Baker, Paul E. Barton, Linda Darling-Hammond, Edward Haertel, Helen F. Ladd, Robert L. Linn, Diane Ravitch, Richard Rothstein, Richard J. Shavelson, and Lorrie A. Shepard
The paper directly challenges the use of value-added modeling to evaluate teachers:
"For a variety of reasons, analyses of VAM results have led researchers to doubt whether the methodology can accurately identify more and less effective teachers. VAM estimates have proven to be unstable across statistical models, years, and classes that teachers teach.
VAM's instability can result from differences in the characteristics of students assigned to particular teachers in a particular year, from small samples of students (made even less representative in schools serving disadvantaged students by high rates of student mobility), from other influences on student learning both inside and outside school, and from tests that are poorly lined up with the curriculum teachers are expected to cover, or that do not measure the full range of achievement of students in the class.
For these and other reasons, the research community has cautioned against the heavy reliance on test scores, even when VAM methods are used, for high stakes decisions such as pay, evaluation, or tenure."
Valerie Strauss has provided a good summary of the report in The Washington Post's Answer Sheet blog.
Other studies have also cautioned against using value-added to make high stakes decisions regarding teachers:
"the use of VAMs does not obviate the need to collect other types of information for the evaluation process.
Most importantly, VAM results should not be used as the sole or principal basis for making consequential decisions about teachers (concerning salaries, promotions and sanctions, for example)."
"Although VAM holds great promise, it also raises many fundamental and complex issues. ... If these issues are not adequately addressed, VAM is likely to misjudge the effectiveness of teachers and schools and could produce incorrect generalizations about their characteristics, thus hampering systematic efforts to improve education."
Permanent linkThe Los Angeles Times' database of "value-added" teacher ratings went live yesterday. Approximately 6,000 third-, fourth-, and fifth-grade teachers and 470 elementary schools are now searchable by name. According to the Times:
"Although value-added measures do not capture everything that goes into making a good teacher or school, The Times decided to make the ratings available because they bear on the performance of public employees who provide an important service, and in the belief that parents and the public have a right to the information."
The paper published another related article on Saturday - No gold stars for successful L.A. teachers - about outstanding teachers in the L.A. Unified School District who are apparently not recognized for their good work or asked to share the secrets of their success. When asked to comment on what she thought made her so effective, the top-ranked teacher replied, "The ones that love their students and love their job do well. You can't bottle that, and you can't teach it."
Another teacher, Aldo Pinto, "like most other teachers interviewed, said his good results had not been recognized.
'No one is ever really singled out, neither good nor bad,' said Pinto. 'The culture of the union is: Everyone is the same. You can't single out anyone for doing badly. So as a result, we don't point out the good either.'"
Walt Gardner presents his arguments against making these ratings public in the latest post on his "Reality Check" blog -Why Not Name and Shame Teachers?
Further elucidating the Obama administration's stance on the use of the value-added model in education, Arne Duncan gave a lecture last week, sponsored by the University of Arkansas Clinton School of Public Service and the Clinton Presidential Library, in which he called on more schools across the U.S. to disseminate information about student achievement and teacher effectiveness - not necessarily to post the results online, but to make the information available to teachers and parents. Permanent linkA new report has been released by the Canadian Council on Learning, sounding a cautionary note with regards to the state of the Canadian education system and its implications for the future well-being of the country.
Taking Stock of Lifelong Learning in Canada (2005-2010): Progress or Complacency?
Canadian Council on Learning, August 2010
The report aims to "provide an overview of the current state of learning across Canada in all phases of life."
The President and CEO of CCL, Dr. Paul Cappon, is quoted in the press release:
"as our report shows, by continuing to fall behind in some key areas of learning, Canada may be creating a national knowledge disadvantage. Unlike Canada, competitor countries have developed, or are in the process of developing, coordinated approaches to education and lifelong learning."
The Vancouver Sun has also printed an article about the report: .
"More 'chronically unemployable' people likely product of Canada's education system, observers say"
Giuseppe Valiante, 25 August 2010
Permanent linkThe results of two polls gauging public opinion on a variety of education issues in the United States have just been released.
Permanent linkThe Los Angeles Times is getting set to launch a database of teacher effectiveness data, using standardized test scores to come up with value added analyses of individual teachers. When the database goes live later this month, performance data for more than 6,000 teachers in the Los Angeles Unified School District (the second largest school system in the United States) will be made available to the public.
From the Times:
"No one suggests using value-added analysis as the sole measure of a teacher. Many experts recommend that it count for half or less of a teacher's overall evaluation.
And in Los Angeles, the method can be used for only a portion of the district's roughly 14,000 elementary school instructors: California students don't take the test until second grade and teachers must have had enough students for the results to be reliable.
Nevertheless, value-added analysis offers the closest thing available to an objective assessment of teachers. And it might help in resolving the greater mystery of what makes for effective teaching, and whether such skills can be taught."
United Teachers Los Angeles has voiced strong opposition to the Times' reporting and database. The union released a member alert urging members to write letters to the editor to express their disapproval, and the president of the union, A.J. Duffy, is now calling on members to boycott the paper. This hard-nosed stance has been criticized both in the blogosphere and in letters to the Los Angeles Times editor.
Leaders of the Los Angeles Unified School District are now asking UTLA to consider making the value-added analysis a part of teachers' evaluations. On Saturday, UTLA agreed to reopen negotiations on evaluations with the district, but Duffy would not indicate whether or not the value-added method would be a part of those negotiations.
U.S. Secretary of Education Arne Duncan has come out in support of the Times' reporting and the database. According to the newspaper, "Duncan's comments mark the first time the Obama administration has expressed support for a public airing of information about teacher performance - a move that is sure to fan the already fierce debate over how to better evaluate teachers."
American Federation of Teachers president Randi Weingarten has also weighed in on the debate, arguing that while parents have a right to know how their children's teachers have fared on evaluations, that information should not be made available to the general public.
In Washington, D.C., Michelle Rhee, the influential chancellor of District of Columbia Public Schools, has indicated that she will consider making value-added scores available to the public. She recently made news by firing 76 teachers for poor performance, using a new evaluation system that holds teachers accountable for their students' scores on standardized tests.
In Nevada, a state law that prohibited test scores from being used in teacher evaluations was changed by the Legislature in February. In the wake of the reports coming out of Los Angeles, the Nevada Education Department seems to be taking steps towards the creation of a similar, publicly-accessible database of student test scores tied to individual teachers - though the Department says it will be another three years before they are able to implement such a database.
Related articles from the Los Angeles Times:
More from the blogs:
Other related news and commentary:
Permanent linkThe Elementary Teachers' Federation of Ontario is calling for a two-year moratorium on provincial standardized tests for grades three and six. This comes on the heels of new research from Environics Research Group, commissioned by ETFO to hold focus groups with teachers throughout Ontario to find out about their experiences with EQAO testing and other assessment strategies. According to ETFO President Sam Hammond, "Teachers told us EQAO testing does little to improve learning. It was originally set up to test the system as a whole, but now it is driving what gets taught in the classroom. We're asking for a moratorium and review of the testing regime and a reduction in the number of Ministry initiatives driven by the test so that teachers can have the time to get back to providing a balanced education for every student."
So far, the Ontario Ministry of Education has been unsupportive of the idea.
More information can be found on the ETFO EQAO Testing page.
Permanent linkA new study from the Institute of Education at the University of London shows that children do better on exams when their teachers focus on learning rather than solely on test performance and results.
The full report by Chris Watkins is titled, "Learning, Performance and Improvement," and appears in the most recent issue of IOE's Research Matters.
In his conclusion, Watkins notes that the "twin challenges" for schools are "to recognize that passing tests is not the goal of education, but a by-product of effective learning," and "to recognize that even when we want pupils to do their best in tests, pressure and performance orientation will not achieve it."
A Guardian article about the study can be found here. Permanent linkThe Globe and Mail recently reported that a four-year pilot study conducted by the Peel District School Board in Ontario has shown that "children who have only a one-month summer break do better in math, retain more of their lessons and need less time for review."
The full report is available here. Permanent linkA recent study conducted by researchers from Harvard, Berkeley and Northwestern found that good kindergarten teachers can have a big impact on how well students do later in their lives. As other researchers have found, the positive effects of effective kindergarten classes largely disappear by the time students reach junior high school. However, the new study shows that these effects re-emerge in adulthood.
As reported by the New York Times:
"Students who had learned much more in kindergarten were more likely to go to college than students with otherwise similar backgrounds. Students who learned more were also less likely to become single parents. As adults, they were more likely to be saving for retirement. Perhaps most striking, they were earning more.
All else equal, they were making about an extra $100 a year at age 27 for every percentile they had moved up the test-score distribution over the course of kindergarten. A student who went from average to the 60th percentile - a typical jump for a 5-year-old with a good teacher - could expect to make about $1,000 more a year at age 27 than a student who remained at the average. Over time, the effect seems to grow, too."
A presentation of the findings, titled "How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project STAR," can be found here.
|