A Blog

What is the Best Teacher Effectiveness Assessment Out There?

I was thinking about a card in the  Creative Wack Pack by Roger Van Oech that spoke of a town where people were struck down by a terrible plague that put its victims into a death-like coma that they would either wake up or die from.  When the townspeople discovered that they had buried some people alive, they naturally wanted to find a solution.   The solutions that they came up with were determined by the questions they asked themselves in order to solve it.  One group asked the question  What if we bury someone who is alive? Their solution was to put food and water in the coffin.  In this way, the person would be saved presumably until someone came to check. Some others asked a different question,  How can we be sure that everyone we bury is dead? They came up with another solution. Put a stake in the lid of the coffin over the person’s heart and when the coffin was closed, all doubt would be erased.

High stake testing is a similar kind of problem.  It answers a particular question (inaccurately) : How can we be sure that teachers are doing their jobs? The solution? Make a test that students will take each year and determine whether teachers are doing their jobs by how their students do.  There are significant issues with this approach, not the least of which is that a test that determines whether teachers keeps their jobs will very quickly become the goal of teaching which narrows the curriculum and poisons relationships.  

It is true that there are mediocre teachers and some truly lousy ones. It’s true that we can always do better.  But, it’s also true that many schools are not broke and that many teachers do good work, even really great, inspiring work.  Evaluation through high stakes testing cannot hope to measure or inspire real quality of teaching.  It can’t even guarantee to identify those teachers who aren’t doing good work since teaching to a test is something we can all do.  It can only give that lovely false sense of security that one gets in front of data. there’s something with a number to hold on to.

To be fair, some have asked, How can we attract and retain the best people? Their solution here seem to miss the dock, as well.  Wendy Knopp thought to create an elite cadre of teachers (TFA) who would teach for two years and then leap frog into policy positions where they would enjoy better status and have a greater impact on the problems of education.  (Short term teaching with a high status escape hatch hasn’t really done that much for teaching as a profession)     And they talked about raising salaries to attract better students who might otherwise go for more lucrative work and offering merit pay on the premise that more money would naturally inspire teachers to do a better job.

All of these solutions are thuggish ones. First, they assume that most teachers are inadequate and need some sort of incentive to do their jobs.  I would argue that this is not the case.  And secondly,  they suffer from a lack of understanding of human motivation.   Daniel Pink, in his book Drive , argues that people are motivated by intrinsic reward.  As he points out, it isn’t Encarta created by paid personnel that is the gold standard of online encyclopedias.  It’s Wikipedia: open source, created by volunteers that is the go to reference site.  If you want to have the best people and get the best work out of the people you have, you have to aim at the best motivation that people have: the ability to use their talents, to feel like they are the owners of their work, that they have a purpose. Teachers don’t do better work because you scare them to death or because you give them a cookie.  Pink says that people are motivated by autonomy, mastery and purpose.   I think he’s right.

Michael McGill, Superintendent of Scarsdale Schools  is doing it the way I think it should be done.  He understand that the best assessment is a local one.  He begins with different questions: What kind of learning do we want children to have?  How can we create a climate in schools where that kind of learning takes place?  How do you encourage autonomy, mastery and purpose?  How will you know it when you see it?  It’s the exact opposite of the carrot and a stick.  It recognizes that when the work itself IS the carrot, there is no stick.  It isn’t VAM; It isn’t a better VAM.  It’s training, trust, purpose and expectation built into school culture.  It’s opportunity and witness rather than gotcha.  This, not so coincidentially, is also a key to good teaching.  It’s not that you don’t measure, or that you don’t have expectations and standards.  If you look at Scarsdale’s scope and sequence for its Advanced Topics (which replaced AP exams) and at its Center for Innovation, you can see what’s possible.  If you create a climate that encourages self direction, mutual respect and commitment, you can let the climate do its work.  It’s a question of leadership.

Dr. Michael McGill, Superintendent of Scarsdale Schools is a hero superintendent guiding  a hero school district with a long history of thoughtful school policy and innovation, sometimes in the face of powerful and regressive political forces (including successful resistance to attempts to limit access in the schools to information and ideas by advocates for McCarthyism in the 1950s).  Continuing that long tradition, Mr. McGill is a vocal opponent of high stakes testing and tying teacher evaluation to tests.

But, he’s not only a hero because of what he opposes; he’s a hero because of what he advocates. Under McGill’s leadership, Scarsdale is providing a thoughtful road map for school improvement that others might well emulate.  He consults with this teachers and community.  Programs and changes are considered thoughtfully and evaluated for their merit before being launched.  His expertise and thoughtful approach to the training and retaining of high quality teachers, his vision for the authentic education of children and his leadership style in the building of his community make him the kind of superintendent any school district would hope to have.

Admittedly, he’s in a district that supports education over measurement. In that way, Scarsdale is already at odds with the testing über alles approach.  In 2001, his parents stood up to the obsessive testing culture created by NCLB and refused to allow their children to take the test.  In 2005, in concert with his faculty, and after two years of researching the value and consequences of the changes under consideration, McGill led the Scarsdale community as it became the first high school in America to drop out of the AP program.  Some might question that policy. Doesn’t that mean that Scarsdale is foregoing a challenging curriculum?  No, it doesn’t. It means the exact opposite.  As he wrote in an article for the AASA, McGill realized that the imperative of test preparation determined both what and often how teachers taught. Although AP courses undeniably met a high standard, teachers wanted their pupils’ experience to be even better and not a cynical process of strategizing to amass the right number of points. Instead of continuing to pursue a test of excellence, he decided to build a course of excellence.  He brought in experts to work with his faculty to build advanced topics that would be superior to the AP and no longer beholden to the AP exams.  Students were still free to take AP exams, but he understood what those in the forefront of the reform movement do not, that learning is compromised when driven by a high stakes testing culture. Scarsdale set their standard higher than the state.

Under the leadership of Dr. McGill, Scarsdale continues a long tradition of educational excellence and innovation.  He surrounds himself with thoughtful, experienced educators and a  School Board dedicated to building better learning environments.  who work with him to continues to innovate and point the way for other school systems that want to set their sights higher as well.  This year, Scarsdale opened its Center for Innovation, the first innovation center to be hosted and supported by a K-12 school district… [b]ased on successful models of university and corporate technology R&D programs, such as the MIT Media Lab and Apple Advanced Technology Group, the Center… provide[s] opportunities for Scarsdale to continue its leadership role in demonstrating innovative instructional practices. The Center plans to partner with concerned others, fostering conversation and collaboration with teachers, students, community members, university researchers, corporate and university R&D departments and other school districts.

Scarsdale provides a principled and meaningful response to the misguided policies of the non-educator driven reform movement.   They are blazing a trail for public educators: a sustainable, well considered approach to practice and innovative, built on experience, expertise and an understanding of the challenges and opportunities ahead in public education.  What better reason for excitement? Informed reform you can believe in.

Dear Student: You seem to be making fun of our test, and we don’t appreciate it. We do not need your implications. We’re looking for that sweet spot between illiteracy and insight…. the text based detail. This is not it. As for any inferences that you might wish to make in the future…. those will be part of a series of wholly interpretative multiple choice questions you’ll get later. Your teacher fails.

Sincerely,
Test Company

ps. You look like money to me.

The primary focus of the New York State English test  is  weighted toward multiple choice answers which require one right answer.  I wouldn’t argue that there are no critical thinking skills involved in multiple choice tests, because there surely are, especially since multiple choice tests are designed by the evil test makers to include distractors that there to trick students into picking partially correct or not quite as right answers instead of the correct ones.  A deep reading of the text is often necessary to tease out which response is really the right one.  So, yes.. as heretical as that may be to some, there are things to teach and to learn about multiple choice tests.  However, it grates against another kind of teaching I find important and that is the notion that there is more than one right answer.  Language, connotation, persuasion, slant, intepretation are not venues for a single response.

In fact, I find that students who focus on a single answer are usually my most literal students, often because they equate intransigent grade grubbing with success and  inevitably they stop their thinking process where they think the grade may be.  My challenge is to integrate test taking and the fact of the one right answer into a curriculum that preferences growth and assertion and support as the more important skills.  Here’s what I tell my incoming 7th graders currently:

First Principle: There is often more than one right answer and more than one way to get to a right answer. Depending on the kind of answers we’re looking for, your answer can be right even when it is different from mine. This means: go out and find your answer… but don’t forget to use the second principle

Second Principle: Your answers are as right as your ability to back them up. If you can back your assertions up with compelling and persuasive evidence, you usually can use them. That means: do your research, find your evidence, make your argument, let the chips fall where they may.

Third Principle: Real thinkers think. Grade grubbers grub. The grade you get is the grade you earn.  It is assessing what you’ve learned.  It is not a prize or a cookie, it is information.  Use the grade you get to inform your actions for the next go round. That being said, if you think I assessed you incorrectly, make your case (refer back to the second principle)

Fourth Principle: Even in circumstances where there is one right answer, the process by which you get to your answer (right or not) is often more useful than the answer itself. Yes, I mean this. (except on the State Test)

Yes, But.. Aren’t Right Answers Important?

Of course, they’re important. It’s not very useful to NASA if they don’t get the right answer on the cold weather capabilities of their ORings but the process of getting to the right answer does include failure (hopefully, not as spectacularly disasterous as the Challenger’s).  Actually, failure is essential.  Sometimes failing is the  means to a better success.

As a former art teacher of mine put it, “You have to be willing to do bad work if you ever want to do good work.”  This is a good lesson for students.  The dilemma of the artist (or writer or scientist or hockey player) is that as they develop, their standard of excellence moves in front of their ability. They develop an educated eye long before they develop the ability to master their art.  Once students realize this simple reality of learning, they can allow themselves to accept their best bad work and do it faithfully every day in their inevitable approach of the standard they hope to achieve.

Randy Pausch also put it very well.  We learn most of what we learn indirectly or by what he called a “head fake.” He said, ”… we send out kids out to learn football or soccer or swimming or whatever it is… we actually don’t want our kids to learn football… we send our kids out to learn more important things… teamwork,sportsmanship, perseverence… and you should keep your eye out for [these lessons], because they’re everywhere.”

Teaching to the test teaches  students that getting the right answer is the only measure of success, and that anything but the “right answer” is failure.  It also teaches them that failure is unacceptable.   Once the right answer becomes coded as the single measure of their personal value, they find it hard to learn the other more subtle lessons that are embodied in a more nuanced and laboratory approach. They’re too focused on superficial learning and measurement… they, along with everyone else, become victims of the all important God of Data.