WASHINGTON -- A key piece of software that has been linked to recent phone outages, including one in Maryland on June 26, was not adequately tested by DSC Communications Corp. before it was released, a DSC official told a House subcommittee yesterday.
Frank Perpiglia, DSC's vice president for technology and product development, said that software developed by DSC usually undergoes three to four months of rigorous testing in the lab before it is released to customers.
But he said that usual test procedures weren't used with a software modification that has since been linked to half a dozen phone outages across the territories served by Bell Atlantic Corp. and Pacific Telesis. The outages, which began last month, have lasted anywhere from several minutes to more than eight hours.
"We had a small modification to make, a small change. We felt that the change itself did not require three to four months of testing," Mr. Perpiglia told the House subcommittee on telecommunications and finance at an emergency briefing yesterday afternoon.
He said the decision, "in hindsight, was flawed."
Mr. Perpiglia told the subcommittee that the software modification resulted in the omission of algorithms -- problem-solving instructions for the computer -- that DSC now believes contributed to the recent spate of outages.
The telephone outages that have dogged Bell Atlantic and Pacific Telesis have been linked to DSC-made computers that route and sort calls. During outages, DSC computers have been overloading themselves with "maintenance messages," which are internal messages computers send themselves to confirm their components are working properly.
DSC computers became so gummed up with maintenance messages that they could no longer route and sort calls, causing the local networks of Bell Atlantic and Pacific Telesis to shut down.
Bell Atlantic's first major outage, on June 26, left up to 5 million customers in Maryland, Washington, Virginia and West Virginia without local phone service for most of the day. An outage in western Pennsylvania less than a week later left about 1 million people without phone service.
Pacific Bell, which serves California, experienced a three-hour outage on June 26 in Los Angeles. Less than a week later, PacTel engineers had to manually reroute traffic to get around outages on successive days in the San Francisco Bay area.
DSC has since made available a software "patch" -- which is essentially a revision of a portion of software -- to all of its customers.
Though the software patch is seen as a temporary solution to the outage problems, representatives of Bell Atlantic, Pacific Telesis, DSC and Bellcore, the research arm of the seven Baby Bells, said yesterday that they still don't know what caused the software to go into an overload mode in the first place.
Since the outages began, a task force of 200 top engineers and technicians have been trying to track down the root cause of the outages.
James Young, vice president for regulatory and industry relations for Bell Atlantic, said yesterday that Bell engineers are being hobbled in their investigative efforts by the 1984 consent decree that regulates the Bells' actions. The court-ordered restriction against manufacturing precludes the Bells from designing telecommunications hardware or software that is integral to operation of the equipment. Last week, Bell Atlantic asked U.S. District Court in Washington for an emergency waiver to that restriction. A hearing on that request is scheduled today.