via NY Times
Over the past few months, there has been a steady trickle of alarms from the Department of Homeland Security concerning a threat to the nation’s power grid, and hardly anyone has noticed.
The advisories concern vulnerabilities in the communication protocol used by power and water utilities to remotely monitor control stations around the country. Using those vulnerabilities, an attacker at a single, unmanned power substation could inflict a widespread power outage.
And yet, the two engineers who discovered the vulnerability say little is being done.
Adam Crain and Chris Sistrunk do not specialize in security. The engineers say they hardly qualify as security researchers. But seven months ago, Mr. Crain wrote software to look for defects in an open-source software program. The program targeted a very specific communications protocol called DNP3, which is predominantly used by electric and water companies, and plays a crucial role in so-called S.C.A.D.A. (supervisory control and data acquisition) systems. Utility companies use S.C.A.D.A. systems to monitor far-flung power stations from a control center, in part because it allows them to remotely diagnose problems rather than wait for a technician to physically drive out to a station and fix it.
Mr. Crain ran his security test on his open-source DNP3 program and didn’t find anything wrong. Frustrated, he tested a third-party vendor’s program to make sure his software was working. The first program he targeted belonged to Triangle MicroWorks, a Raleigh, North Carolina based company that sells source code to large vendors of S.C.A.D.A. systems. It broke instantly.
Mr. Crain called Mr. Sistrunk, an electrical engineer, to see if he could help Mr. Crain test his program on other systems.
“When Adam told me he broke Triangle, I worried everything else was broken,” said Mr. Sistrunk.
Over the course of one week last April, the two tested Mr. Crain’s software across 16 vendors’ systems. They did not find a single system they couldn’t break.
By the end of the week, the two had compiled a 20-page report replete with vulnerabilities in 16 different system vendors for the Department of Homeland Security’s Industrial Control Systems Cyber Emergency Response Team, I.C.S.-C.E.R.T., which notifies vendors of vulnerabilities and issues public advisories.
And then, they waited. It would take I.C.S.-C.E.R.T. another four months to issue a public advisory for Triangle MicroWorks’ system.
Triangle MicroWorks’ engineering manager Greg Godlevski said that during those four months, the company developed a number of its own tests to look for defects in its software and fix them. Mr. Godlevski said the company waited for confirmation from Mr. Crain that the problem had been fixed, then met with I.C.S.-C.E.R.T. several times to review and comment on the government advisory.
“We take any reported problems discovered in our products very seriously,” Mr. Godlevski said. “We expend a lot of effort adding levels of security to our protocols and ensuring that they comply to the published specifications.”
Over the course of those four months, Mr. Crain and Mr. Sistrunk found vulnerabilities in an additional nine vendors’ systems.
“We haven’t found anything we haven’t broken yet,” Mr. Crain said in an interview. At minimum, the two discovered that they could freeze, or crash, the software that monitors a substation, thereby blinding control center operators from the power grid. Mr. Crain likened that capability to “a bank robber being in a bank vault with the camera frozen.”
In the case of one vendor, Mr. Crain found that he could actually infiltrate a power station’s control center from afar. An attacker could use that capability to insert malware to take over the system, and like Stuxnet, the computer worm that took out 20 percent of Iran’s centrifuges, inflict actual physical harm.
“This is low-hanging fruit,” said Mr. Crain. “It doesn’t require some kind of hacker mastermind to understand the protocol and do this.”
What makes the vulnerabilities particularly troubling, experts say, is that traditional firewalls are ill-equipped to stop them. “When the master crashes it can no longer monitor or control any and all of the substations,” said Dale Peterson, a former N.S.A. employee who founded Digital Bond, a security firm that focuses on infrastructure. “There is no way to stop this with a firewall and other perimeter security device today. You have to let DNP3 responses through.”
Even more troubling, Mr. Peterson said, is that most DNP3 communications aren’t regulated. The original version of DNP3 worked on serial communications — a way of transmitting data usually found in things like coaxial cables — and is still widely deployed in large systems, particularly substations around the country. But current cybersecurity regulations, governed by the North American Electric Reliability Corporation’s (N.E.R.C.) Critical Infrastructure Protection Committee (C.I.P.C.) are focused on Internet Protocols, or I.P. protocols, and specifically exclude serial communications and the equipment that uses them from meeting any security requirements.
“Why isn’t D.H.S., N.E.R.C., and the DNP3 committee telling vendors they need to fix this now and utility owners they need to get this patched A.S.A.P.?” Mr. Peterson said.
Like most security alerts, there are some caveats to this concern for the safety of electric facilities: Mr. Peterson’s company, Digital Bond, sells consulting services to assess and improve the security of S.C.A.D.A. systems.
Mr. Crain also has an interest. In March, he plans to release a free version of his security test, but for now he is charging vendors to use his program. (Mr. Crain would not disclose pricing, since it differed for each vendor based on vendor size, saying only that he charged in the “thousands” though he said he charged far less than commercial services like WurldTech Security, which charges tens of thousands of dollars for similar programs.)
To date, D.H.S. has posted nine advisories, several of them for software used by major players in the electric sector.
“This is a systemic problem,” Mr. Crain said. “Most of the top five utilities use this software and just because a patch is available, doesn’t necessarily mean that utilities are applying them.”