It looks like you're new here. If you want to get involved, click one of these buttons!
VAM Analysis and Merit Pay
Displaying all 12 posts.
yes"> Robert Valiant
yes"> Here you will find articles and datato refute the claims of proponents of VAM and merit pay.
yes"> about 10 months ago · Delete Post
or, if you prefer,
Guy Brandenburg analysis of NYC VAM evaluations
yes"> Director, UCLA's Institute forDemocracy, Education, and Access
yes"> From Huffington Post
yes"> Posted: August 24, 2010 11:05 AM
yes"> Value Added is No Magic: AssessingTeacher Effectiveness
yes"> Read More: Lausd , Los Angeles Times, Los Angeles Unified School District , School Reform , Teacher Effectiveness ,Teacher Rankings , Value Added , Los Angeles News
yes"> That old sorcerer has vanished
yes"> And for once has gone away!
yes"> Spirits called by him, now banished,
yes"> My commands shall soon obey.
yes"> In Goethe's classic, the apprenticeuses a sorcerer's spell to ease his daily chores. Chanting the master's words,he brings a broomstick to life and tells it to fetch water to clean theworkshop. The broomstick obeys, only too well. It races between the well and backuntil the workshop begins to flood. Although the apprentice had enoughknowledge to set magic in motion, he could not think ahead to what he did notknow.
yes"> I worry about a similar flood ofunintended consequences if the Los Angeles Times moves forward with its plansto publish a database that places 6,000 Los Angeles third- to fifth-gradeteachers on a spectrum from "least effective" to "mosteffective." The Times believes that the data will be a powerful tool toforce better teaching, but it cannot anticipate all of the consequences. Forexample, consider that capable prospective teachers might avoid a profession inwhich they risk public embarrassment based on an undeveloped science. Considerthe well-documented estimates that 25% of the value-added assessments arelikely to be in error.
yes"> Publishing the database might easilyundermine parent and teacher morale and make it more difficult for principalsto advance school improvement. Being told that their child's teacher is"ineffective," or even marginally less effective than a teacheracross the hall, may lead some parents to pressure the principal to place theirchild with a "high-scoring" teacher. Pitting parents against oneanother or against their principal is not a recipe for school improvement.
yes"> The Times' teacher effectivenessrankings are based on an elaborate statistical model created by Richard Buddin,a senior economist and education researcher at the Rand Corporation. (Significantly,Buddin did not attach teachers' names to his analysis; that was done by theTimes.)
yes"> Buddin is one of many researchersacross the country exploring so-called value-added approaches to assessingteacher quality. The assessments measure gains that students make on standardizedtests from one year to the next. For example, researchers compare test scoresof fourth graders with their scores as third graders to determine the"value added" by the fourth grade teacher. Proponents believe thatthe "value added" reliably distinguishes between more and lesseffective teachers. And they think that school officials would use suchcomparisons to target support to struggling teachers and motivate them to dobetter.
yes"> Yet value-added analyses focusnarrowly on standardized tests, usually in math and English Language Arts.These tests give important information about student learning, but they ignoremuch learning that matters to students, parents, and teachers. That's why itcan be a useful tool, but cannot possibly stand alone as a measure of"effectiveness." The National Academy of Sciences has identifiedseveral of the problems posed by value-added methods. These cautions should betaken seriously.
yes"> * First, student assignments toschools and classrooms are rarely random. As a consequence it is not possibleto definitively determine whether higher or lower students test scores resultfrom teacher effectiveness or are an artifact of how students are distributed.
yes"> * Second, it is difficult to comparegrowth of struggling students with the growth of high performers. In technicalterms, standardized tests do not form equal interval scales. Enabling studentsto move from the 20th percentile to the 30th is not the same as helpingstudents move from the 80th to the 90th percentile. These test score numbersare not like inches along a tape measure that have the same value regardless ofwhere they occur.
yes"> * Third, estimates of teachereffectiveness can range widely from year to year. In recent studies, 10-15% ofteachers in the lowest category of effectiveness one year moved to the highestcategory the following year while 10-15% of teachers in the highest categoryfell to the lowest tier.
yes"> The National Academy of Sciencesconcluded that value-added analysis "should not be used as the sole orprimary basis for making operational decisions because the extent to which themeasures reflect the contribution of teachers themselves, rather than otherfactors, is not understood."
yes"> And yet, the Los Angeles Times isabout to publish a database with the teacher effectiveness rankings of 6,000elementary school teachers. The Times argues that its role is to provide"parents and the public ... information that would otherwise bewithheld" about the "performance of public employees." The Timesshould not believe in the magic of this data, and should realize that it cannotforesee or control all of the consequences.
yes"> Follow John Rogers on Twitter:www.twitter.com/UCLA_IDEA
yes"> Evidence about the use of testscores to evaluate teachers: Economic Policy Institute, 2010
yes"> “…there is broad agreement amongstatisticians, psychometricians, and economists that student test scores aloneare not sufficiently reliable and valid indicators of teacher effectiveness tobe used in high-stakes personnel decisions, even when the most sophisticatedstatistical applications such as value-added modeling are employed.
yes"> For a variety of reasons, analysesof VAM results have led researchers to doubt whether the methodology canaccurately identify more and less effective teachers. VAM estimates have provento be unstable across statistical models, years, and classes that teachersteach. One study found that across five large urban districts, among teacherswho were ranked in the top 20% of effectiveness in the first year, fewer than athird were in that top group the next year, and another third moved all the waydown to the bottom 40%. Another found that teachers’ effectiveness ratings inone year could only predict from 4% to 16% of the variation in such ratings inthe following year. Thus, a teacher who appears to be very ineffective in oneyear might have a dramatically different result the following year. The samedramatic fluctuations were found for teachers ranked at the bottom in the firstyear of analysis. This runs counter to most people’s notions that the truequality of a teacher is likely to change very little over time and raisesquestions about whether what is measured is largely a “teacher effect” or theeffect of a wide variety of other factors.”
yes"> Neither Fair Nor Accurate •Research-Based Reasons Why High-Stakes Tests Should Not Be Used to EvaluateTeachers
yes"> By Wayne Au
yes"> A pitched battle raged in myhometown of Seattle this fall. Superintendent Maria Goodloe-Johnson and theSeattle Public Schools district fought with the Seattle Education Association overtheir most recent teachers’ union contract. At the heart of the dispute: Shouldteacher evaluations be based in part on student scores on standardized tests?
yes"> Seattle is not unique in thisstruggle, and it is clear that Superintendent Goodloe-Johnson takes her cuefrom what is happening nationally.
yes"> In August, for instance, the LosAngeles Times printed a massive study in which LA student test scores were usedto rate individual teacher effectiveness. The study was based on a statisticalmodel referred to as value-added measurement (VAM). As part of the story, theTimes published the names of roughly 6,000 teachers and their VAM ratings (seesidebar, p. 37).
yes"> In October the New York CityDepartment of Education followed suit, publicizing plans to release the VAMscores for nearly 12,000 public school teachers. U.S. Secretary of EducationArne Duncan lauded both the Times study and the NYC Department of Educationplans, a stance consistent with Race to the Top guidelines and PresidentObama’s support for using test scores to evaluate teachers and determine meritpay.
yes"> Current and former leaders of manymajor urban school districts, including Washington, D.C.’s Michelle Rhee andNew Orleans’ Paul Vallas, have sought to use tests to evaluate teachers. Infact, the use of high-stakes standardized tests to evaluate teacher performanceà la VAM has become one of the cornerstones of current efforts to reshapepublic education along the lines of the free market.
yes"> On the surface, the logic of VAM andusing student scores to evaluate teachers seems like common sense: The moreeffective a teacher, the better his or her students should do on standardizedtests.
yes"> However, although research tells usthat teacher quality has an effect on test scores, this does not mean that aspecific teacher is responsible for how a specific student performs on astandardized test. Nor does it mean we can equate effective teaching (or actuallearning) with higher test scores.
yes"> Given the current attacks onteachers, teachers’ unions, and public education through the use of educationalaccountability schemes based wholly or partly on high-stakes standardized testscores and VAM, it is important that educators, students, and parentsunderstand why, based on educational research, such tests should not be used toevaluate teachers.
yes"> Although there are manywell-documented problems with using VAM to evaluate teachers, I’ve chosen tohighlight six critical issues with VAM that are so problematic they aloneshould be enough to stop the use of high-stakes standardized tests for suchevaluations. I hope these will be helpful as talking points for op-ed pieces,blogs, and discussions at school board meetings, PTA meetings, and in thebleachers at basketball games.
yes"> Statistical Error Rates
There is a statistical errorrate of 35 percent when using one year’s worth of test data to measure ateacher’s effectiveness, and an error rate of 25 percent when using data fromthree years, researchers Peter Schochet and Hanley Chiang find in their 2010report “Error Rates in Measuring Teacher and School Performance Based on TestScore Gains,” released by the U.S. Department of Education’s National Centerfor Education Statistics.
yes"> Bruce Baker, finance expert atRutgers University, explains that using high-stakes test scores to evaluateteachers in this manner means there is a one-in-four chance that a teacherrated as “average” could be incorrectly rated as “below average” and facedisciplinary measures. Because of these error rates, a teacher’s performanceevaluation may pivot on what amounts to a statistical roll of the dice.
yes"> Year-to-Year Test Score Instability
yes"> As Tim Sass, economics professor atFlorida State University, points out in “The Stability of Value-Added Measuresof Teacher Quality and Implications for Teacher Compensation Policy,” testscores of students taught by the same teacher fluctuate wildly from year toyear. In one study comparing two years of test scores across five urbandistricts, more than two-thirds of the bottom-ranked teachers one year hadmoved out of the bottom ranks the next year. Of this group, a full third wentfrom the bottom 20 percent one year to the top 40 percent the next. Similarly,only one-third of the teachers who ranked highest one year kept their topranking the next, and almost a third of the formerly top-ranked teachers landedin the bottom 40 percent in year two.
yes"> If test scores were an accuratemeasurement of teacher effectiveness, “effective” teachers would rate highconsistently from year to year because they are good teachers; and one wouldexpect “ineffective” teachers to rate low in terms of test scores just asconsistently. Instead, the year-to-year instability that Sass highlights showsthat test scores have very little to do with the effectiveness of a singleteacher and have more to do with the change of students from year to year(unless, of course, one believes that one-third of the highest ranked teachersin the first year of the study simply decided to teach poorly in the second).
yes"> Day-to-Day Score Instability
yes"> Fifty to 80 percent of anyimprovement or decline in a student’s standardized test scores can beattributed to one-time, randomly occurring factors, according to Thomas Kane ofHarvard University and Douglas Staiger of Dartmouth College in their researchreport “Volatility in Test Scores.”
yes"> This means that factors such aswhether or not a child ate breakfast on test day, whether or not a child gotinto an argument with parents or peers on the way to school, which otherstudents happened to be in attendance while taking the test, and the child’sfeelings about the test administrator account for at least half of any givenstudent’s standardized test score gains or losses. Some factors, such as a dogbarking outside an open window, can affect an entire class.
yes"> Kane and Staiger’s findingsillustrate that using tests to evaluate teachers ignores the reality that ahost of individual daily factors that are completely out of a teacher’s controlcontribute to how a student performs on any given test. To reward or punish ateacher based on such scores could literally mean rewarding or punishing ateacher based on how well or poorly a student’s morning went.
yes"> Nonrandom Student Assignments
yes"> The grouping of students—eitherwithin schools through formal and informal tracking or across schools throughrace, socioeconomic class, and linguistic (ELL) segregation—greatly influencesVAM test results, as 10 leading researchers in teacher quality and educationalassessment highlight in their policy brief “Problems with the Use of StudentTest Scores to Evaluate Teachers,” published by the Economic Policy Institute.
yes"> These researchers note that“teachers who have chosen to teach in schools serving more affluent studentsmay appear to be more effective simply because they have students with morehome and school supports for their prior and current learning, and not becausethey are better teachers.”
yes"> Even when VAM models attempt to takeinto account a student’s prior achievement or demographic characteristics, themodels assume that all students will show test gains at an equal rate. Thisassumption, however, does not necessarily hold true for groups of students whohistorically have performed poorly on tests, for English language learners whoare asked to become proficient in both a new language and a tested subjectarea, or for students with disabilities whose test-based rates of progress maybe incomparable to any other student.
yes"> Nonrandom student assignment meansthat a teacher could be punished, dismissed, or lose tenure purely because thecourse they teach or the school they teach in has a significant population oftraditionally low-scoring students who may show variable or slower test scoregains.
yes"> Imprecise Measurement
yes"> High-stakes, standardized tests arealso unable to account for the complexities of learning (and, by extension,teaching). For instance, we know from the linguistic research of Steven Pinkerand others that learning often happens in a U-shape—that making mistakes is anintegral part of the learning process. When children are tested, we never quiteknow where on the U-shaped learning curve they might be, nor do we realize thattheir mistakes could be a vital part of a natural learning process. When testsare used to evaluate teachers, it is possible that highly effective teacherswho push students out of their cognitive comfort zones are penalized forprovoking the deep learning that requires students to make mistakes on the wayto greater understanding.
yes"> Standardized tests are also toocrude to account for the possibility of cognitive transfer of skills thatstudents learn across different subjects. Using VAM, as the researchers in theabove-mentioned Economic Policy Institute policy brief explain, means that “theessay writing a student learns from his history teacher may be credited to hisEnglish teacher, even if the English teacher assigns no writing; themathematics a student learns in her physics class may be credited to her mathteacher.” In other words, we can never be certain which class and which teachercontributed to a given student’s test performance in any given subject.
yes"> Out-of-School Factors
yes"> Out-of-school factors such asinadequate access to health care, food insecurity, and poverty-related stress,among others, negatively impact the in-school achievement of students soprofoundly that they severely limit what schools and teachers can do on theirown, explains David Berliner, Regents Professor of Education at Arizona StateUniversity, in his report “Poverty and Potential.”
yes"> Although it is clear from theresearch of Stanford University’s Linda Darling-Hammond and others thatteachers play an absolutely pivotal role in student success, when we usehigh-stakes tests to evaluate teachers, we incorrectly assume that teachershave the ability to overcome any obstacle in students’ lives to improvelearning. Although good teachers are critically necessary, they are not alwayssufficient.
yes"> To assume otherwise is to think thatteachers (and schools) can somehow make up for the lack of housing, food,safety, and living wage employment, among other factors, all on their own. Thesocial safety net is the responsibility of a much broader socioeconomicnetwork—not the sole responsibility of the teacher.
yes"> Politics, Not Reality
yes"> The reality of standardized tests isthat they are too imprecise and inaccurate to measure the effectiveness ofindividual teachers. The sad thing is that testing experts, researchers, andpsychometricians have known this for quite some time. In 1999, for instance,the expert panel that made up the Committee on Appropriate Test Use of theNational Research Council cautioned that “an educational decision that willhave a major impact on a test-taker should not be made solely or automaticallyon the basis of a single test score.”
yes"> Yet two short years later, abipartisan Congress and the presidential administration of George W. Bushpassed No Child Left Behind and its test-and-punish approach to school reforminto law.
yes"> Although the Bush administrationseemed to ignore educational research as a matter of policy (as illustratedthrough NCLB’s Reading First program and the advocacy of using phonics-onlyteaching methods that had little basis in research), many hoped for somethingdifferent with the election of President Obama.
yes"> Unfortunately, the Obamaadministration has sent a clear message: When it comes to high-stakesstandardized testing, the research doesn’t matter.
yes"> It hasn’t mattered that, accordingto the above cited U.S. Department of Education report, “More than 90 percentof the variation in student gain scores is due to the variation in student-levelfactors that are not under control of the teacher.”
yes"> It hasn’t mattered that the NationalResearch Council of the National Academy of Sciences has stated that “VAMestimates of teacher effectiveness should not be used to make operationaldecisions because such estimates are far too unstable to be considered fair orreliable.”
yes"> It hasn’t mattered that even theresearchers who completed the Los Angeles Times study acknowledged that VAMdata were too unreliable to use as the sole measure of teacher performance (apoint that the Times neglected to clearly articulate in their article).
yes"> Sadly, with Bush, now with Obama,politics and ideology trump educational research.
yes"> One would think that all of thepolicy makers, politicians, pundits, superintendents, talk show hosts,documentary movie makers, business leaders, and philanthropic foundations so inlove with the idea of using test score data to evaluate teachers would beequally as passionate about accuracy. People’s lives are at stake, and yet the“data” underlying important decisions about teacher performance couldn’t beshakier.
yes"> The shakiness of test-based VAM dataillustrates that the current fight over teacher “accountability” isn’t reallyabout effectiveness. The more substantial public conversation we should behaving about rising poverty, the racial resegregation of our schools,increasing unemployment, lack of health care, and the steady defunding of thepublic sector—all factors that have an overwhelming impact on students’educational achievement—has been buried. Instead, teachers and their unionshave become convenient scapegoats for our social, educational, and economicwoes.
yes"> Yes, teachers’ performance needs tobe evaluated, but in a manner that is fair and accurate. Using high-stakesstandardized tests and VAM to make such evaluations is neither.
yes"> A former high school teacher, WayneAu is a Rethinking Schools editor and assistant professor at the University ofWashington, Bothell Campus.
yes"> about 9 months ago · Delete Post
School District Citizens
yes"> One of the best compendiums ofarguments against VAM can be found here:<http://rdsathene.blogspot.com/2011/02/are-value-added-methods-vam-new-flat.html>
yes"> about 8 months ago · Delete Post
yes"> School District Citizens
yes"> Here is another great source forarguing against VAM: http://www.njspotlight.com/ets_symposium/
yes"> Read the EPI study of VAM here.Theirfindings: VAM is a SCAM.
yes"> Financial incentives for teachers toincrease student performance is an increasingly popular education policy aroundthe world. This paper describes a school-based randomized trial in overtwo-hundred New York City public schools designed to better understand theimpact of teacher incentives on student achievement. I find no evidence thatteacher incentives increase student performance, attendance, or graduation, nordo I find any evidence that the incentives change student or teacher behavior.If anything, teacher incentives may decrease student achievement, especially inlarger schools. The paper concludes with a speculative discussion of theoriesthat may explain these stark results.
yes"> Roland G. Fryer Department ofEconomics Harvard University
yes"> about 7 months ago · Delete Post
yes"> Of course we should hold teachersaccountable,
yes"> but this does not mean we have topretend
yes"> that mathematical models can dosomething they
yes"> cannot. Of course we should rid ourschools of
yes"> incompetent teachers, butvalue-added models are
yes"> an exceedingly blunt tool for thispurpose. In any
yes"> case, we ought to expect more fromour teachers
yes"> than what value-added attempts tomeasure.
yes"> John Ewing
yes"> I came across this article byMathematician John Ewing and wanted to share it with you.
yes"> Mathematical Intimidation: Driven byData
yes"> by John Ewing
yes"> Mathematicians occasionally worry
yes"> about the misuse of their subject.
yes"> G. H. Hardy famously wrote about
yes"> mathematics used for war in his
yes"> autobiography, A Mathematician’s
yes"> Apology (and solidified hisreputation as a foe of
yes"> applied mathematics in doing so).More recently,
yes"> groups of mathematicians tried toorganize a boycott
yes"> of the Star Wars project on thegrounds that
yes"> it was an abuse of mathematics. Andeven more
yes"> recently some fretted about the roleof mathematics
yes"> in the financial meltdown.
yes"> But the most common misuse ofmathematics
yes"> is simpler, more pervasive, and(alas) more
yes"> insidious: mathematics employed as arhetorical
yes"> weapon—an intellectual credential toconvince
yes"> the public that an idea or a processis “objective”
and hence better than othercompeting ideas or
yes"> processes. This is mathematicalintimidation. It is
yes"> especially persuasive because somany people are
yes"> awed by mathematics and yet do notunderstand
yes"> it—a dangerous combination.
yes"> The latest instance of thephenomenon is
yes"> valued-added modeling (VAM), used tointerpret
yes"> test data. Value-added modeling popsup everywhere
yes"> today, from newspapers to televisionto
yes"> political campaigns. VAM is heavilypromoted with
unbridled and uncriticalenthusiasm by the press,
yes"> by politicians, and even by (some)educational experts,
yes"> and it is touted as the modern,“scientific”
yes"> way to measure educational successin everything
yes"> from charter schools to individualteachers.
yes"> Yet most of those promotingvalue-added
yes"> modeling are ill-equipped to judgeeither its
yes"> effectiveness or its limitations.Some of those
yes"> who are equipped make extravagantclaims without
yes"> much detail, reassuring us thatsomeone
yes"> has checked into our concerns and weshouldn’t
yes"> worry. Value-added modeling ispromoted because
yes"> it has the right pedigree—because itis based on
yes"> “sophisticated mathematics”. As aconsequence,
yes"> mathematics that ought to be used toilluminate
yes"> ends up being used to intimidate.When that happens,
yes"> mathematicians have a responsibilityto
yes"> speak out.
yes"> Value-added models are all abouttests—standardized
yes"> tests that have become ubiquitous inK–12
yes"> education in the past few decades.These tests have
yes"> been around for many years, buttheir scale, scope,
yes"> and potential utility have changeddramatically.
yes"> Fifty years ago, at a few key pointsin their education,
schoolchildren would bringhome a piece of
yes"> paper that showed academicachievement, usually
yes"> with a percentile score showingwhere they landed
yes"> among a large group. Parents couldtake pride in
yes"> their child’s progress (or fret overits lack); teachers
yes"> could sort students into those whoexcelled
yes"> and those who needed remediation;students could
yes"> make plans for higher education.
yes"> Today, tests have more consequences.“No
yes"> Child Left Behind” mandated thattests in reading
yes"> and mathematics be administered ingrades 3–8.
yes"> Often more tests are given in highschool, including
yes"> high-stakes tests for graduation.With all that
yes"> accumulating data, it was inevitablethat people
yes"> would want to use tests to evaluateeverything
yes"> educational—not merely teachers,schools, and
yes"> entire states but also newcurricula, teacher training
yes"> programs, or teacher selectioncriteria. Are
yes"> the new standards better than theold? Are experienced
teachers better thannovice? Do teachers
yes"> need to know the content they teach?Using data
yes"> from tests to answer such questionsis part of the
yes"> current “student achievement”ethos—the belief
yes"> that the goal of education is toproduce high test scores. But it is also part of a broader trend in modern
yes"> society to place a higher value onnumerical
yes"> (objective) measurements than verbal(subjective)
yes"> evidence. But using tests toevaluate teachers,
yes"> schools, or programs has manyproblems. (For a
yes"> readable and comprehensive account,see [Koretz
yes"> 2008].) Here are four of the mostimportant problems,
yes"> taken from a much longer list.
yes"> 1. Influences. Test scores areaffected by many factors,
yes"> including the incoming levels ofachievement,
yes"> the influence of previous teachers,the
yes"> attitudes of peers, and parentalsupport. One
yes"> cannot immediately separate theinfluence of a
yes"> particular teacher or program amongall those
2. Polls. Like polls,tests are only samples. They
yes"> cover only a small selection ofmaterial from
yes"> a larger domain. A student’s scoreis meant to
yes"> represent how much has been learnedon all
yes"> material, but tests (like polls) canbe misleading.
yes"> 3. Intangibles. Tests (especiallymultiple-choice
yes"> tests) measure the learning of factsand procedures
yes"> rather than the many other goals of
yes"> teaching. Attitude, engagement, andthe ability
yes"> to learn further on one’s own aredifficult
yes"> to measure with tests. In somecases, these
yes"> “intangible” goals may be moreimportant
yes"> than those measured by tests. (Thefather of
yes"> modern standardized testing, E. F.Lindquist,
yes"> wrote eloquently about this [Lindquist1951];
yes"> a synopsis of his comments can befound in
yes"> [Koretz 2008, 37].)
yes"> 4. Inflation. Test scores can beincreased without
yes"> increasing student learning. Thisassertion has
yes"> been convincingly demonstrated, butit is widely
ignored by many in theeducation establishment
yes"> [Koretz 2008, chap. 10]. In fact,the assertion
yes"> should not be surprising. Everyteacher knows
yes"> that providing strategies fortest-taking can
yes"> improve student performance and thatnarrowing
yes"> the curriculum to conform preciselyto the
yes"> test (“teaching to the test”) canhave an even
yes"> greater effect. The evidence showsthat these
yes"> effects can be substantial: One candramatically
yes"> increase test scores while at the sametime actually
yes"> decreasing student learning. “Testscores”
yes"> are not the same as “studentachievement”.
yes"> This last problem plays a largerrole as the stakes
yes"> increase. This is often referred toas Campbell’s
yes"> Law: “The more any quantitativesocial indicator
yes"> is used for social decision-making,the more
yes"> subject it will be to corruptionpressures and
yes"> the more apt it will be to distortand corrupt the
yes"> social processes it is intended tomeasure” [Campbell
1976]. In its simplestform, this can mean
yes"> that high-stakes tests are likely toinduce some
yes"> people (students, teachers, oradministrators)
yes"> to cheat…and they do [Gabriel 2010].But the
yes"> more common consequence ofCampbell’s Law
is a distortion of theeducation experience, ignoring
yes"> things that are not tested (forexample, student
yes"> engagement and attitude) andconcentrating on
yes"> precisely those things that are.
yes"> The remainder of this paper can beread at Mathematical Intimidation:
yes"> Driven by the Data.
yes"> about 5 months ago · Delete Post
yes"> May 15, 2011 To The New York StateBoard of Regents:
yes"> As researchers who have doneextensive work in the area of testing and measurement, and the use ofvalue-added methods of analysis, we write to express our concern about thedecision pending before the Board of Regents to require the use of state testscores as 40% of the evaluation decision for teachers.
yes"> As the enclosed report from theEconomic Policy Institute describes, the research literature includes manycautions about the problems of basing teacher evaluations on student testscores. These include problems of attributing student gains to specificteachers; concerns about overemphasis on “teaching to the test” at the expenseof other kinds of learning; and disincentives for teachers to serve high-needstudents, for example, those who do not yet speak English and those who havespecial education needs.
yes"> Reviews of research on value-addedmethodologies for estimating teacher “effects” based on student test scoreshave concluded that these measures are too unstable and too vulnerable to manysources of error to be used as a major part of teacher evaluation. A report bythe RAND Corporation concluded that:
yes"> The research base is currentlyinsufficient to support the use of VAM for high-stakes decisions aboutindividual teachers or schools.1
yes"> The Board on Testing and Assessmentof the National Research Council of the National Academy of Sciences stated,
yes"> ...VAM estimates of teachereffectiveness ... should not be used to make operational decisions because suchestimates are far too unstable to be considered fair or reliable.
yes"> Henry Braun, then of the EducationalTesting Service, concluded in his review of research:
yes"> VAM results should not serve as thesole or principal basis for making consequential decisions about teachers.There are many pitfalls to making causal attributions of teacher effectivenesson the basis of the kinds of data available from typical school districts. Westill lack sufficient understanding of how seriously the different technicalproblems threaten the validity of such interpretations.2
yes"> According to these studies, theproblems with using value-added testing models to determine teachereffectiveness include:
yes"> 1 Daniel F. McCaffrey, DanielKoretz, J. R. Lockwood, Laura S. Hamilton (2005). Evaluating Value-Added Modelsfor Teacher Accountability. Santa Monica: RAND Corporation. 2 Henry Braun,Using Student Progress to Evaluate Teachers: A Primer on Value-Added Models(Princeton, NJ: ETS, 2005), p. 17.
This is 2 years old, but still relevant. http://www.washingtonpost.com/blogs/answer-sheet/post/the-myths-of-standardized-testing/2011/04/14/AFNxTggD_blog.html