Tyler Cowen asks why pay across software programmers is not more unequal. He cites John Cook, who argues that differences in programmer productivity are difficult to identify and measure. Since I do know something about this (though not a comprehensive answer), I thought I would comment.
Cook contrasts programmers to salespeople: “In some professions such a difference would be obvious. A salesman who sells 10x as much as his peers will be noticed, and compensated accordingly. Sales are easy to measure, and some salesmen make orders of magnitude more money than others.” Although this is the most-cited example around (except perhaps for traders), I don’t actually think it’s true.
OK, I’ll concede that sales are easy to measure. But I won’t concede that sales bear more than a loose and passing relationship to sales productivity (unless you make the circular definition of sales productivity as sales). I’ve been a part of dozens of major, high-dollar sales efforts, and there are many, many factors other than how “good” the sales rep is: the sales consultants (pre-sales, sales engineers, application engineers, whatever you call them) involved; how happy the customer is with previous products from the same company; executive involvement; random personality fit; recommendations by industry analysts (who are primed by marketing); the user group meeting; and on and on. If you simply reward successful sales, you end up rewarding people who are the best at monopolizing the best sales consultants and the best at getting top executives to come along on their deals; is that really what you want? And of course, by far the biggest factor in any big deal is luck. In enterprise software, where the deals are so big and so few that a salesperson can make his or her quota with one big deal, luck trumps everything.
That said, most companies do pay salespeople on commission anyway, because they can’t think of another way to do it, and the salespeople want it that way. In addition, the people in charge of most companies, at least technology companies, came up on the sales side, so they are disinclined to realize that a lot of their success was due to luck, and they are inclined to keep the same metrics that they succeeded on.
Turning back to programmers, there are a lot of reasons why differences in pay don’t match differences in ability. One is that most programs of any significance are written by teams, and the productivity of a team is to some extent limited by the productivity of its least-productive member. Another problem is that quality of code is very hard to measure up front; you really don’t know how good any component is until it’s been used by real customers in lots of ways you didn’t anticipate it being used.
That said, if you walk around any good development team and ask who the superstars are, you’ll find out that people really do know who the great programmers are. So why don’t the top people make more?
Well, to some extent they do. Although pay for employees of, say, Oracle goes pretty much by seniority and responsibility (like for most jobs), there is also a big market for independent contractors, who typically get paid much more (per hour) than employees. Second, as Cook says, “Someone who is 10x more productive than his colleagues is likely to leave, either to work with other very talented programmers or to start his own business.” There are many software programmers in the world; some do maintenance for decades-old internal systems at midsized manufacturing companies in the Midwest and some work for Google or Microsoft. The latter make a lot more than the former. Finally, historically in Silicon Valley the most sought-after jobs were not the ones with the highest salaries; they were the jobs at new hot startups, where you could get a non-trivial amount of equity as an early employee. I assure you, those jobs do not go to mediocre programmers.
But still, within a given “hot startup,” differences in pay do not fully reflect differences in ability. And I think the reasons are more sociological, or cultural, than anything else. (As I said above, not only do I think it would be easy to identify the top programmers on a given team, generally it would be completely uncontroversial who those people are.) One factor is that major pay discrepancies can have a corrosive effect on a team of people who have to work together. Most sales departments, by contrast, have such a strong individualist ethos that it’s assumed from the beginning that everyone is in it for himself, so there is nothing to corrode. In this sense, software programming is like almost every other job other than sales or trading; pay differences don’t reflect differences in ability, because the social convention is for them not to.
Another factor that is probably more important is the way salaries are set at companies. At a high level, they are set by management teams, headed by a CEO (who is rarely a former developer), and supervised by a board of directors. The people making these decisions do not understand the nature of software development, and are often prone to idiotic ideas like thinking that you can ship a product in half the time if you have twice as many people. My observation is that top executives, for all they say about the importance of people, revert to a bean-counter mentality when it comes time to deciding how much to pay those people.
Now, there are some companies that go out of their way to pay top developers more than they would make on the open market. But they don’t have to pay them very much more, because there aren’t very many of those companies. The market price is set by companies run by bean-counters, who are like the famous “noise traders” of Larry Summers, Brad DeLong, Fischer Black, and others. So a developer who is 10x as productive as the average will make 20% more than average and probably be content, because he or she doesn’t expect to make 10x the average and is happy to be recognized. If not, he or she can go be a hired gun.
This is neither an elegant nor a complete explanation, but I think it’s pretty realistic.