Pots and marshmallows
Bill Buxton has an oft-referenced story about a pottery class where one group was graded for their final pot and the other group was graded for the number of pots they produced (regardless of their quality). The second group made better pots — they were iterating and improving while the first was preoccupied with making the one perfect pot. Similarly, Tom Wujec talks about groups trying to build towers out of dry spaghetti and tape with a marshmallow on top. The most successful groups start with the marshmallow on top and iterate to increase the height of the tower. Other groups fail because they try unsuccessfully to add the marshmallow at the end.
The lesson I want to draw here is the importance of starting with the constraint and keeping it always in the picture. I believe that one way to drive a successful PhD career is to have a single good research problem that understands an important constraint in the world. That’s the constraint that guides all of your iterations. I’ve been thinking about this since coming across the biography Andrew Ko’s website, a very successful former HCII student. I noticed that his description of his voluminous work at HCII was a single problem: “trying to find out what made debugging so difficult, and inventing technologies to make it easier”.
After thinking about it for a few weeks, I finally have a one-liner that I like: “How can we make the learner come back tomorrow?” Seems simplistic, but there are a few reasons I like it:
- There’s a user-centered value: if the user is coming back tomorrow, they probably like what’s going on.
- There’s a learning theory value: more than anything, learning takes persistance and consistency. We are naturally wired to learn, but there is a lot of information to absorb and skills to master.
- There’s even a business value: if the user is coming back every day, they’re probably going to be willing to pay or check out a sponsor or are at least looking at a lot of ads.
- A solution can be evaluated without requiring inter-rater reliability on some abstract construct from the 80s (but any papers submitted to CHI 2013 doing this should be accepted without question!) — literally just: did they try to learn more the next day?
- It doesn’t assume a technological solution.
- What may be part of a good answer is my favorite technological intervention, spaced repetition!
- As simple as this is, I think a lot of people working in intelligent tutoring systems aren’t asking this question. They may just assume the technology is going to be integrated into the classroom. Most experiments are looking at what students do in a single class period. The closest close-to-home example is Nudge, but that’s still relying on the motivational structure of a traditional course.
- It motivates some cross-domain work. Why does a gamer play every day? Are more substantial reasons than the addictive features of the game? Can we learn anything from why gamers quit (see also Koster, 2005)? Replace gamer with anyone serious about their craft. They’re all learning something.
A naive observer might conclude that the definition of a reader is someone who slowly, day-by-day, moves a bookmark from one side of a book to the other. They might not be too far off. Assuming someone knows how to read, when a book is well-written, the reader can just look at the words on the page and the brain does the rest fairly automatically. Millions of kids didn’t need comprehension monitoring strategies, concept maps, or highlighting to understand transfiguration spells in the made-up universe of the Harry Potter books.
Now imagine a book where the content is automatically generated with exactly what you need to learn. Reminders appear in the following pages just as you are about to forget what you’ve encountered, sections carefully repeat and rephrase what is difficult for you, assignments at the end of chapter suggest exactly the skills you need to work on. And a bookmark traces your progress as you come back tomorrow to move it a little further down. That’s what we can do with educational technology. That’s all we need to do.