Clickers for the mind

I had taken for granted that feedback is a critical part of learning: it’s information that we use to adjust our performance and incrementally get better. However, Dan Meyer gave an excellent example of when feedback goes wrong. When working through an algebraic equation in a computer program, the student writes a step and the equation turns red: the equation is wrong. They flip a sign and then it’s green: the equations is correct! And yet the student doesn’t understand anything.

I finally have a better framework for thinking about feedback: reinforcement learning, the theory behind animal training, particularly via a technique called clicker training.

In clicker training, animal trainers help an animal associate the “click!” of a clicker (just a small object that makes a click when pressed by the trainer) with positive reinforcement like a small treat for a dog or a fish for a killer whale. The purpose of the clicker is that the trainer can time the click exactly to when the animal performs a correct step. The positive reinforcement is a very powerful way to instill the behavior in the animal, and it works from household pets to performance animals.

Back to our algebra software: the program turns green, click!, positively reinforcing the student’s step. The problem is that we’re reinforcing the wrong action: “keep flipping signs until it’s right”. How Children Fail is a whole book of these kind of training failures in the classroom setting. The author explores how students he’s observed fall into patterns of trying to get to right answer, whether that is saying “I don’t know” or probing for the right answer like our student in the computer program. Anything but doing the hard work of understanding and working out the real problem!

It’s like reinforcing the dog for dragging every item in the house to your lap because those happened to include her fetch stick. Instead, we can break down the actions into a chain of tiny parts, and reinforce these one at a time. The principles of reinforcement learning, which I’ve been reading about (after getting a dog of my own) in the book Don’t Shoot the Dog, tell us how to do this kind of training. Here’s one example from the book:

We were watching a horse being trainer to bow, or kneel on one knee, by a traditional method involving two men and a lot of ropes and whips; the horse under this method is repeatedly forced onto one knee until it learns to go down voluntarily.

I said it didn’t have to be done that way and asserted that I could train a horse to bow without ever touching the animal. (One possibility: Put a red spot on the wall; use food and a marker signal to shape the horse to touch its knee to the spot; then lower the spot gradually to the floor so that to touch it correctly and earn a reinforcer the horse has to kneel.)

This act of shaping is a subtle art. Even training my puppy to sit wasn’t a straightforward procedure — I almost wanted to reach for the ropes and whips after twenty minutes without her ever getting in the right position. In animal training, trainers understand that verbal communication is starting from scratch. The dog has no idea what “sit” means when we start. The math student likewise has no idea what the concept of “equality across the sign” means. (Actually the task is even harder because we don’t know whether the observed behavior of flipping the sign comes from understanding the mistake or from trying all the possibilities. Meanwhile a sit is a sit.)

I believe that successful teaching practices are those that use effective shaping. It applies no matter what perspective you bring to education. Discovery learning shapes using affordances in the learning environment as I’ve talked about before with Portal. Explicit instruction shapes through worked examples that slowly build on previous understanding and have clear points of failure when applying misconceptions like the sign of a unit.

The link to animal training and its behavioral history has been quite surprising to me. Behavioralism has been a relegated branch of psychology, particularly by the cognitive science training that I had. The example of applied behavioralism that I see is in the design of addictive but meaningless games like Cow Clicker, where you are reinforced for clicking an invisible cow. But there’s no question that human minds respond to the same effects and they can be used for good. (Another amusing application: relationships.) There is still more to explain in terms of when concepts are understood versus when we’re just grinding through procedures, but this is where I’m at for now.