The summer of 1991 was, in the words of one Bell Atlantic Corp. engineer, "a summer that people around here will never forget. It was disastrous."
Indeed. One year ago today, Bell Atlantic had the first of a summer's worth of high-tech headaches from repeated telephone network outages.
The first outage crippled phone service in Maryland, Washington, D.C., Virginia and parts of West Virginia, leaving as many as 5 million customers without local phone service for most the day.
Others followed in rapid succession, leaving Bell Atlantic customers in Western Pennsylvania, and Pacific Bell customers in Los Angeles and San Francisco, without phone service. The outages, which hit in waves throughout July, left millions of people out of touch for periods of a few minutes to more than eight hours.
The outages were traced to a minor software error -- three bits of errant code buried in 3 million lines of computer coding. By the time the source of the problem had been found, public confidence had been shaken, Congress was alarmed, and state and federal regulators were asking, "Could this happen again?"
One year later, the short answer is yes.
But industry reforms make the chances of a replay of last summer more remote, experts say.
"That's not to say it's fail-safe from now on, but the chances of a major outage are somewhat less," says James Spurlock, special assistant to the chief of the common carrier bureau at the Federal Communications Commission in Washington.
In the past year, the industry has made changes aimed at reducing the likelihood of outages and, in the event of a breakdown, ensuring that solutions can be found faster.
* Not relying on one software or equipment vendor. The idea is to shore up "redundancy," or backup, in the network.
In last summer's outages, Bell Atlantic and Pacific Bell were using the same hardware and software, provided by the same vendor. The upshot: Both fell victim to the same, errant software.
* More testing. Before last summer's failures, the Bells relied heavily on outside vendors to test software and hardware. Now the Bells test extensively themselves and scrutinize test data submitted by outside vendors more closely.
* Tightened security. A close review of the Bells' electronic systems revealed weak spots, company officials said. Those systems, always a security priority, have been tightened even more. Security also has been tightened at buildings that house critical equipment.