In the random-lottery incentive system, a subject is asked to choose in several choice situations. At the end, a random procedure selects only one of those several situations to be played for real. The other situations are not played for real. Experimental economists and others have often and extensively discussed the pros and cons of this system. The fundamental reason to play only one situation for real, and not two or all, is that under multiple payments a choice and payment in one situation, if for real, can be accompanied by a payment from another situation, and these can interact and distort. The house-money effect and the income effect concern such interactions. Under the random-lottery incentive system, each choice situation, if played for real, is not accompanied by any payment or play in another situation. Then no interactions between different choice situations can arise. Another, less fundamental, reason for implementing only one choice can be that then bigger stakes can be considered.

Holt (1986) described a problem for the random-lottery incentive system that might arise theoretically. Subjects may not perceive every choice situation as isolated, but may perceive their situation as a grand meta-lottery over many choice situations, which it is indeed. The assumption that an isolated perception and the grand meta-lottery perception lead to the same behavior is close to the assumption of expected utility and its independence condition, which is empirically questionable. Starmer & Sugden (1991) were the first to empirically test whether the problem pointed out by Holt arises in experiments. It did not in their study. Subjects treated every choice situation as isolated and did not treat them as a grand meta-lottery.

Since that time, the random-lottery incentive system has become the almost exclusively used incentive system for individual choice, and numerous studies have used and tested it. It is used by people well recognized in experimental economics (Harrison, Lau, & Williams 2002; Holt & Laury 2002; Myagkov & Plott 1997). Unfortunately, experimental economics referees who emphasize the importance of real incentives but who do not know the field of experimental individual choice well, will again and again open up the debate of this incentive system. People working on individual choice have become annoyed about the time lost. Thus, Hey & Lee (2005) explicitly referred to referees in their closing sentence (p. 263):
"The conclusion seems to be that experimenters can continue to use the random-lottery incentive mechanism and that this paper can be used as a defence against referees who argue that the procedure is unsafe."

In the
annotated bibliography
the key word
random-lottery incentive system
will give many references that used this system.

My experience at this moment of writing, January 2007, is that still more than half of the referees of economics journals open up this debate. Their comments usually start with "I have serious doubts about ..." They, however, never suggest a good alternative incentive system. During several years, in every experimental paper I would write half a page describing in detail the history and defense of the incentive system. It is, however, a waste of time and journal space if every paper on individual choice again must do this. In some recent papers I tried if I could stop and write only that the system is the nowadays almost exclusively used system for individual choice, hoping that referees will sense that they should not enter a well known debate in a field that they are not very familiar with. But then again a majority of referees opens up the good old debate, I am experiencing now.

How can authors avoid this waste of human resources? In the short run, it cannot be stopped I am afraid. Many referees decide in the first minute that they want to reject a paper, and from that moment on just search for negative things to say taking them as little time as possible. The random-lottery incentive system serves referees in this state of mind well. (Which is not to say of course that every referee opening up the debate did so for the reason mentioned here.) Thus it will continue to appear in the near future. I hope that many others will join John Hey, as I am now doing, in efforts to minimize the loss of human resources that is involved. I hope that editors will stop such debates. Remember that the random-lottery incentive system is the only real incentive system for individual choice known today that can avoid the income effect. Without it, real incentives for individual choice are no longer well possible.

Added Feb. 2008: Lee (2008) is a nice empirical study showing that the random lottery incentive system works well, avoiding income effects that occur with repeated payments.

     Harrison, Glen W., Morten I. Lau, & Melonie B. Williams (2002), “Estimating Individual Discount Rates in Denmark: A Field Experiment,” American Economic Review 92, 1606–1617.

     Hey, John D. & Jinkwon Lee (2005), “Do Subjects Separate (or Are They Sophisticated)?,” Experimental Economics 8, 233–265.

     Holt, Charles A. (1986), “Preference Reverals and the Independence Axiom,” American Economic Review 76, 508–513.

     Holt, Charles A. & Susan K. Laury (2002), “Risk Aversion and Incentive Effects,” American Economic Review 92, 1644–1655.

     Lee, Jinkwon (2008), “The Effect of the Background Risk in a Simple Chance Improving Decision Model,” The Journal of Risk and Uncertainty 36, 19–41.

     Myagkov, Mikhail G. & Charles R. Plott (1997), “Exchange Economies and Loss Exposure: Experiments Exploring Prospect Theory and Competitive Equilibria in Market Environments,” American Economic Review 87, 801–828.

     Starmer, Chris & Robert Sugden (1991), “Does the Random-Lottery Incentive System Elicit True Preferences? An Experimental Investigation,” American Economic Review 81, 971–978.