This past July the City of San Francisco's chief network engineer was arrested and charged with multiple counts of computer tampering after refusing to divulge the passwords to critical routers on the city's wide area network. (He subsequently did reveal the passwords to the Mayor of San Francisco.) There was no destruction of city data nor was there any attempt to extort money from the city. The network engineer in question is still being held in jail with bond set at $5M. The trial has yet to begin and several motions for bail reduction have been denied.
There is a lot we don't know yet about this episode but the obvious question is what were the city's IT managers thinking when they allowed one network engineer to claim ownership to the "security keys" of the city's network?
Even if this network engineer was a Cisco Certified Internet Expert (CCIE), which is the "gold standard" in Cisco certifications, it stands to reason that you do not let one person gain exclusive control to important network devices.
The passwords and configurations of the city's routers are not the personal property of any network engineer no matter what they think they are doing. And it was the City of San Francisco's lack of appropriate IT policies and procedures that contributed to the creation of this security problem.
So, what can be done to prevent a trusted network engineer from going rogue? Well, regular IT management communication with the network engineer would have been a good start. A network engineer working in isolation with little management contact could easily imagine that he is the only one smart enough to do the job right. And having policies and procedures regarding securing network device passwords and proper storage of network device configurations would have been a good thing to have in place.
In the vacuum created by the lack of of such policies and procedures, this network engineer decided that he knew best how to protect his network from those who he likely deemed to be less competent or potentially destructive.
Once again, IT management failed. Most network engineers take pride in their work, but when pride leads to jealous guarding of information and mistrust of IT management you wind up with a serious personnel and security problem.
Whatever this particular network engineer feared may have been completely justified in his own mind, but the fact that he was able to unilaterally take action to secure his network the way he wanted to is an indictment of the IT management of the City of San Francisco.
Should this network engineer be punished? Only if he is found guilt of the charges filed against him, but IT management is equally culpable for allowing this security abuse to exist in the first place. Heads need to roll in order to establish that there was incompetence in the IT management of the city's network. The city's IT administration also needs to demonstrate to other network engineers that appropriate IT policies and procedures are going to be designed and implemented to prevent further abuse.
The City of San Francisco, which is short drive from Silicon Valley, was publicly embarrassed by the news this event generated and may have to spend up to $1M to rectify the security problems exposed by the actions of this network engineer. It is also a good story of what not to do with talented and trusted employees.
Do you have IT policies and procedures in place to prevent a similar situation from happening in your organization?
For those interested in the outcome of the trial and additional news related to this security lapse, read the blog posts of my former co-worker, Paul Venezia, which are published at http://weblog.infoworld.com/venezia.
SET SOAPBOX = OFF
Comments