Human Brains Are Curious Things

We are running an experiment in my lab right now that asks (randomly) some participants to "try hard" and some to "go fast." Mostly, it seems to work. The try hard people take about two seconds longer to answer each question than the "go fast" people. Cool.

Then, after that part of the experiment, we tell them (basically), "OK, we're done with that part. Ignore those instructions and just answer the rest of these questions like you normally would."

But they don't. The "try hard" people keep taking longer to answer the rest of the questions than the "go fast" people. And they seem to be answering the questions differently. Arrgh. Since they are randomly assigned to a treatment group, they should not differ on the other variables.

For other reasons, we have to ask the questions in the order they are presented. And we need both the "try hard" instructions and the "go fast" instructions.

Hence, experimental design remains an art.