The Human Flaw of Heartbleed

The announcement on April 7 was alarming. A new Internet vulnerability called Heartbleed could allow hackers to steal your logins and passwords. It affected a piece of security software that is used on half a million websites worldwide. Fixing it would be hard: It would strain our security infrastructure and the patience of users everywhere.
It was a software insecurity, but the problem was entirely human.
Software has vulnerabilities because it's written by people, and people make mistakes -- thousands of mistakes. This particular mistake was made in 2011 by a German graduate student who was one of the unpaid volunteers working on a piece of software called OpenSSL. The update was approved by a British consultant.
In retrospect, the mistake should have been obvious, and it's amazing that no one caught it. But even though thousands of large companies around the world used this critical piece of software for free, no one took the time to review the code after its release.
The mistake was discovered around March 21, 2014, and was reported on April 1 by Neel Mehta of Google's security team, who quickly realized how potentially devastating it was. Two days later, in an odd coincidence, researchers at a security company called Codenomicon independently discovered it.
When a researcher discovers a major vulnerability in a widely used piece of software, he generally discloses it responsibly. Why? As soon as a vulnerability becomes public, criminals will start using it to hack systems, steal identities, and generally create mayhem, so we have to work together to fix the vulnerability quickly after it's announced.
The researchers alerted some of the larger companies quietly so that they could fix their systems before the public announcement. (Who to tell early is another very human problem: If you tell too few, you're not really helping, but if you tell too many, the secret could get out.) Then Codenomicon announced the vulnerability.
One of the biggest problems we face in the security community is how to communicate these sorts of vulnerabilities. The story is technical, and people often don't know how to react to the risk. In this case, the Codenomicon researchers did well. They created a public website explaining (in simple terms) the vulnerability and how to fix it, and they created a logo -- a red bleeding heart -- that every news outlet used for coverage of the story.
The first week of coverage varied widely, as some people panicked and others downplayed the threat. This wasn't surprising: There was a lot of uncertainty about the risk, and it wasn't immediately obvious how disastrous the vulnerability actually was.
The major Internet companies were quick to patch vulnerable systems. Individuals were less likely to update their passwords, but by and large, that was OK.
True to form, hackers started exploiting the vulnerability within minutes of the announcement. We assume that governments also exploited the vulnerability while they could. I'm sure the U.S. National Security Agency had advance warning.
By now, it's largely over. There are still lots of unpatched systems out there. (Many of them are embedded hardware systems that can't be patched.) The risk of attack is still there, but minimal. In the end, the actual damage was also minimal, although the expense of restoring security was great.
The question that remains is this: What should we expect in the future -- are there more Heartbleeds out there?
Yes. Yes there are. The software we use contains thousands of mistakes -- many of them security vulnerabilities. Lots of people are looking for these vulnerabilities: Researchers are looking for them. Criminals and hackers are looking for them. National intelligence agencies in the United States, the United Kingdom, China, Russia, and elsewhere are looking for them. The software vendors themselves are looking for them.
What happens when a vulnerability is found depends on who finds it. If the vendor finds it, it quietly fixes it. If a researcher finds it, he or she alerts the vendor and then reports it to the public. If a national intelligence agency finds the vulnerability, it either quietly uses it to spy on others or -- if we're lucky -- alerts the vendor. If criminals and hackers find it, they use it until a security company notices and alerts the vendor, and then it gets fixed -- usually within a month.
Heartbleed was unique because there was no single fix. The software had to be updated, and then websites had to regenerate their encryption keys and get new public-key certificates. After that, people had to update their passwords. This multi-stage process had to take place publicly, which is why the announcement happened the way it did.
Yes, it'll happen again. But most of the time, it'll be easier to deal with than this.