Lately the NYT has been alive with stories and commentary about college students cheating using amazing new technological techniques like CTRL-C and CTRL-V.

I came across Goodhart’s Law in my web wanderings several weeks ago and it’s been knocking about in my mind ever since. Basically it states that when you attempt to pick a few easily defined metrics as proxy measures for the success of any plan or policy, you immediately distract or bait people into pursuing the metrics, rather than pursuing the success of the policy itself.  The mythical example is Soviet factories:

“When given targets on the basis of numbers of nails produced many tiny useless nails, when given targets on basis of weight produced a few giant nails.”

This is hard stuff because it’s human nature to want to distill big complicated goals down into a few easy to understand numbers, and it seems efficient from a change-making perspective as well. Yet we can all see the bad outcomes from an overreliance on the numbers: Police districts (ok, on The Wire) manipulating murder cases to come out better on COMSTAT ; School districts and states lowering standards and encouraging learning disabled kids to stay home on test days, so they look better under No Child Left Behind tests. I also see how it works in my own life: I have a log on my iPod nano of how many times I’ve used the stopwatch to time my regular 2.8 mile run over the bridge. But then I started to turn it on when I go to the gym, or on lazy days when I only run half as far, because it makes the stats (number of times I worked out this month) look better.

In the case of college cheaters, we methodically train students for years to define their worth and their tasks in school extrinsically by grades and test scores (see No Child Left Behind, above). Then we give them boring assignments–test questions that aren’t updated from year to year, and papers that don’t require introspection or individual response. Then we pretend to be shocked when they respond just like Stakhanovites in a Soviet factory, turning out more and more of shoddier and shoddier product.

The answer is simple: we’re measuring the wrong things.

Remember the Woody Allen joke? I cheated on my metaphysics midterm–I looked into the soul of the student sitting next to me.

If professors were looking into students’s souls, and truly asking students to look into their own souls, then cheating might be less of an issue.  Would you still turn in a shoddy, cut-and-pasted paper  if it wasn’t just between you and your professor–your work was out there on the web for friends and family members and future employees to see?  What if it was a collaborative project where you were responsible for other team members’ grades as well as your own? The interpersonal stakes are certainly raised then. Or what if the topic of the class was one that you chose to study, one that was close to your heart? What if there was real trust and a bond between you and your professor?

I really liked what Alfie Kohn had to say about this on the Room for Debate blog, and I plan to download one of his books.

3 Responses to “Cheating and Goodhart’s Law”

  1. Goodhart’s law also applies to Universities chasing position in World Rankings. I heard of one institution that jumped a bunch of places when they put someone in charge of working the rankings and figuring out what metrics they had to chase to improve their score.

  2. Emily B. says:

    I couldn’t agree more. Not that I ever condone cheating…but the very structure of the school system sends students the message that what matters is a product that meets certain superficial expectations, not whether they actually internalized any mastery or curiosity of a subject.

    I had a college professor (of special topics in theater history) of whom it became known rather widely that his grading was, let us say, pretty flexible. Your paper could wind up being on a thesis, or topic, completely different than the one he’d approved. If you stumbled across something tangential in your research that you were more interested in, you could just ditch your actual topic and go with that. I saw classmates’ presentations of papers in which they said, essentially, that they felt they completely failed to argue what they had set out to. Basically, if you wound up saying something interesting or original, you’d get an A, even if what you did didn’t particularly conform to the assignment.

    It would be easy to think that this would be an encouragement to slack off and half-ass the work for this class, but that’s not what I saw happening. I absolutely agonized over my papers for this class–I wanted to write something true and meaningful and well-supported, even though I knew that he was going to be pretty lax about how the paper itself turned out. What he cared about was whether you wound up more knowledgeable and a better reader, and most importantly, more interested in the subject.

  3. Anya,

    Love the book, the blog, and all of your work. Please keep it up!

    We spend far to much money and time teaching our youth how to take standardized test. In his recent book, Linchpin, Seth Godin wrote: “We train the factory workers of tomorrow. Our graduates are very good at following instructions. [...] So, it it any surprise that people have learned to fit in, do the standardized test, keep heads down, obey instructions? … ” He goes on to say that, “Well-intentioned teachers don’t want to do this, but the system often gives them no choice. The work of creating positive change in a class room is daunting, and without enough time and support, it’s a tough slog.”

    It is very frustrating. Class time is not available to invest in innovative teaching because it has to be used for programmed teaching and preparation to take ‘the test’. I believe change is possible, but it will take a lot of work. We must be engaged!

    Great post … thanks for sharing.

    Rick

Leave a Reply