County e-mail, data server restored, but some data lost

The replacement of a faulty computer controller on Wednesday evening, Sept. 10, triggered a series of events that caused the failure of the county’s e-mail system, the county’s web site and lost a full day’s worth of e-mail and accounting system data entry. Full functionality for those systems was restored shortly before 8 a.m. on Friday, Sept. 12.

September 16, 2008 2:23 am

The replacement of a faulty computer controller on Wednesday evening, Sept. 10, triggered a series of events that caused the failure of the county’s e-mail system, the county’s web site and lost a full day’s worth of e-mail and accounting system data entry. Full functionality for those systems was restored shortly before 8 a.m. on Friday, Sept. 12.

“We were on the phone with the manufacturer, following their technician’s directions throughout the replacement process,” said County Information Systems Director George Johnson. “When we powered the system back up at about 8 p.m. Wednesday, the controller only recognized about 80 percent of the devices it was supposed to control.” The storage systems not recognized included the data disc arrays that contained e-mail, data for the county website and accounting system data used by the Auditor’s office and the Treasurer’s office.

After several hours of discussions with the vendor, Hewlett-Packard, and attempts to recover the data on the storage devices, the decision was made to restore data from Tuesday night’s data backup tape.

“The failure occurred before the Wednesday evening data backup ran,” Johnson said, “so we lost all e-mails that arrived between about 8 p.m. on Tuesday night and Wednesday evening.” All data entered into the accounting system on Wednesday will have to be re-entered as well, however neither the Auditor’s office nor the Treasurer’s office believes that information about any transactions will be lost.

“HP [the manufacturer] told us this was a unique circumstance,” Johnson said. “That’s not much consolation, but at least for now everything seems to be working properly.” He noted that the county had already purchased, and was in the process of installing additional equipment which would “mirror” the database server to prevent service disruptions in case of a future server failure.

In the meantime, anyone who e-mailed information or requests to county offices on Wednesday, Sept. 10 and has not received a response should re-send the e-mail to make sure it is received.