Some perspectives on Information Security

This paper by Ross Anderson gives an economic perspective on why information security is hard. It talks a little bit about the security of bank teller machines, computer networks, and web sites. Along the way, it makes some really insightful comments about the information economy in general.

"In general, when the party who is in a position to protect a system is not the party who would suffer the result of security failure, then problems may be expected," Anderson writes. This may sound obvious, but if you think about it, there are quite a number of systems in use today where this is the case. Some would argue that the widespread incidence of identity fraud today is a result of such perverse incentive systems. An identity thief ruins the credit rating of the person whose identity he steals, NOT of the bank or credit card company whose lax security allowed the attack to happen in the first place!

Anderson then spends some time discussing economic pressures that lead to insecure systems. He begins by talking about "network externalities" and Metcalfe's Law. Basically this law states that the more people use certain networks, the more valuable the networks become for everyone. For example, "the more merchants take credit cards, the more useful they are to customers, and so more customers will buy them... and the more customers have them, the more merchants will want to accept them." In my opinion, this is almost the guiding principle behind the rise of Microsoft and many other computer companies. Popular technologies tend to get more popular, even if they aren't perfect. Customers get "locked-in" to using a particular technology because of the high cost of switching technologies.

Given all these network externalities, Anderson continues, software companies have a huge incentive to get products out the door quickly, even if that means sacrificing security or stability. In many cases, a rational manager will not want his product to ship later than those of other companies, even if waiting would make his product better.

Companies often use a proprietary, obscure architecture instead of a standard, well-analyzed and tested one. This helps to promote lock-in, and also make differentiated pricing possible. In short, differentiated pricing is when a company prices its product not on its cost, but on its value to the consumer, in order to optimize profit. "This is familiar to the world of air travel," Anderson writes: "you can spend $200 to fly the Atlantic in coach class, $2000 in business class, or $5000 in first." Obviously, it doesn't cost the airline $1800 more to seat a business class passenger than a coach class passenger. But by pricing this way, they can optimize their profit by charging each customer what he can afford to pay, rather than the true value of what he is being sold.

In this context, "digital rights management" schemes start to seem even more sinister. By promoting even more customer lock-in than is currently the case, DRM would allow companies to engage in even more differentiated pricing. This has already happened: one of the first DRM schemes to see the light of day, the CSS scheme which protects DVD movies, was designed to allow movie studios to price movies differently (and release them at different times) in different parts of the world. That is the function of the so-called "region codes."

Anderson then moves on to discuss some peculiarities of information technology itself. The technology favors attack over defense, he maintains, because even if "white hat" security consultants find 100 times the bugs that a lone "black hat" hacker finds, the hacker might only need one bug to compromise the system! "Defending a modern information system can... be likened to defending a large, thinly-populated territory like the nineteenth-century Wild West," he comments. "The men in black hats can strike anywhere, while the men in white hats have to defend everywhere."

The politicization of information security only makes this situation worse. Government agencies engaged in economic intelligence missions have every incentive to use security bugs to spy on foreign nations, and little incentive to fix the bugs. Unless people are aware there is a security problem, successful fixers seldom get credit, while successful attackers always do.

In my opinion, this is only a specific example of a broader phenomenon where doing preventive maintenance on systems often earns you little respect, even if the maintenance is vital. Before Hurricane Katrina, how many voters in New Orleans voted for politicans based on their record of strengthening the levies? Voters are seldom aware of the most important things that politicans do for them; instead, they focus on media circuses and talking points which are of little real consequence.

Anyway, to continue: there are other reasons why information technology buyers often don't get a secure system. They may not be knowledgeable enough to evaluate the system that they are buying. Security certifications often mean little, and, at least in the past, most companies had little in-house security know-how. (Maybe this will improve in the future.)

I should finish with Anderson's conclusion.
In an ideal world, the removal of perverse economic incentives to create insecure systems would de-politicize most issues. Security engineering would then be a matter of rational risk management rather than risk dumping. But as information security is about power and money ­, about raising barriers to trade, segmenting markets and differentiating products, the evaluator should not restrict herself to technical tools like cryptanalysis and information flow, but also apply economic tools such as the analysis of asymmetric information and moral hazard. As fast as one perverse incentive can be removed by regulators, businesses (and governments) are likely to create two more.