Sometimes when I complain about the horrible state of the art in social science, and especially in human resource management, senior colleagues tell me that it’s near impossible to convince companies to do proper large randomized trials. Perhaps, but if that’s how science needs to get done, then work on convincing them. Fortunately, there are some who had success. I showcase this particular study not because of content interest to readers of this blog, but because I want to show that such studies are indeed done from time to time. I suggest researchers summarize these when approaching companies to convince them to let researchers do experiments.

• Friebel, G., Heinz, M., Krueger, M., & Zubanov, N. (2017). Team incentives and performance: Evidence from a retail chain. American Economic Review, 107(8), 2168-2203.

In a field experiment with a retail chain (1,300 employees, 193 shops), randomly selected sales teams received a bonus. The bonus increases both sales and number of customers dealt with by 3 percent. Each dollar spent on the bonus generates $3.80 in sales, and $2.10 in profit. Wages increase by 2.2 percent while inequality rises only moderately. The analysis suggests effort complementarities to be important, and the effectiveness of peer pressure in overcoming free-riding to be limited. After rolling out the bonus treatment, and control shops’ performance converge, suggesting long-term stability of the treatment effect.

The design is a pretty straightforward randomized, clustered design:

The firm entails 193 bakery shops with a total of 1,300 employees. Like many of its competitors, the chain had developed its business model since 1980, by exploiting the benefits of attractive locations and economies of scale. In 2011, however, discount retailers Aldi and Lidl began to sell freshly-baked bread and related products in their dense network of existing shops, with significant success. Their bread is considered to be of similar quality to that of the chains, but is sold at much lower prices, which forces the incumbent chains to rethink their strategy. Our study firm moved into the market for snacks, cakes, sandwiches, and beverages traditionally covered by cafés and fast-food chains. Substantial investments in shop design were carried out and additional marketing instruments introduced, some of them in pilot studies. As a complement to the strategic shift into more service-oriented product lines, HR practices were reconsidered with the goal of motivating employees to engage more actively with their customers. After intensifying training and experimenting unsuccessfully with hiring more qualified employees to try to improve customer service, the firm approached us for help.

…

An average shop employs a team of 7 employees (4 full-time equivalents), aver-age monthly sales are around €28,000, and there are around 10,000 customer visits per month (see column 1 in Table 1). In a typical shop, the workers carry out a variety of interconnected and often simultaneous tasks, such as handling goods, operating the oven, serving customers, etc. The volatility of demand makes task specialization expensive (Friebel and Yilmaz 2017), as workers would be idle much of their time. Instead, the workers are expected to help each other. For instance, consider a worker in a not-so-busy shift who can prepare sandwiches for the colleague(s) in a busier shift or do nothing. Or, consider a worker who could clean the oven or help a col-league deal with a queue of customers. Workers helping each other, both within and across shifts, is an important source of complementarities, resonating with Itoh’s (1991) theoretical argument that help efforts are cost efficient and give rise to team organization and, if anything, team-based compensation. Indeed, detailed inter-views carried out in several randomly picked shops in December 2014 revealed that employees perceived their work as teamwork. Furthermore, there is only one cash register per shop, creating substantial congestion in peak times and thus requiring seamless coordination to serve customers quickly.

Convincing the company:

It is interesting to note that it was relatively straightforward to deal with the mild forms of organizational resistance described above. In particular, we took care of a substantial part of the administrative work related to the implementation of the bonus. We ran simulations of the bonus effects on sales and personnel costs, showing that the team bonus payments were likely to be lower than €20,000 per month when half of the shops were treated and the monthly shop bonus was capped at €300. To deal with district managers’ concerns about their bonuses, senior management decided that bonus payments to sales staff would be paid from a separate budget in order not to affect the district managers’ cost KPI. District managers were quick to realize that they were likely to benefit from increased sales in their shops. The workers’ council was also in favor of the bonus, because it was designed as a pure add-on payment and was a result of the high level of trust between the council and management.

To further gain trust we built up a record of research engagement with the firm prior to the experiment (see also List 2011). We achieved an “early success” by showing the company the limitations of their existing subjective performance evaluation system. We also reinforced trust through constant communication with managers at all levels of hierarchy. We received the shops’ sales, financial and accounting, geographical, compensation, and personnel data since January 2012, which allowed us to conduct a very precise randomization procedure (explained in detail in Section III). We offered our advice free of charge and covered most of the research costs. The company provided the data and administrative support needed. Our main interfaces were the CEO, HR, the sales director, and a small group of district managers. The support of the workers’ council turned out to be crucial. It assured legitimacy and commitment for the bonus, suggesting that institutions that one might have expected to be obstructive to change and experimentation, when convinced, will actually assist the experimenter by boosting trust and legitimacy within the firm.

The monetary reward structure:

The results

Changes in wages for workers in the treatment group:

Modeling of the results:

From my reading, “POTS” is a simple regression without the pre-treatment sales, “ANCOVA” is the same except they add this covariate. The addition of this covariate removes some of the variation in the data allowing for greater precision of the treatment variable: the SE in first model is 0.034 and it’s 0.014 in the third model. The middle model is a more effective difference-in-differences approach (note the changed sample size because it’s using person-time units instead of person units, I think).

Graphically:

So, all in all, they find this incentive adds about 3% sales, does not produce counter-productive gaming of the system, raises wages by 2.2%