Should IG Reports be Treated as Gospel?
John Gilligan doesn't think that inspector general audits of agencies' IT security should be treated as gospel. The flaws they identity may be factual, he says, but they're not always put in the perspective of the agencies' overall approach to cybersecurity.
Gilligan is the former chief information officer of the Air Force and Energy departments, and for the past couple of years been the major force is getting government agencies to adopt the Consensus Audit Guidelines, 20 key automated controls that when implemented could go a long way in securing agencies' IT systems.
His basic gripe about the IG information security audit is that it's not placed in context. In a conversation I had with Gilligan late last week, here's what he said about the process:
"The whole IG review process is one that has not really provided a lot of value because the IGs come in without criteria, and all they do is that they have to find potential problems. So, the agency says, 'It doesn't matter what I do; the IG is going to find some problems.' But the Consensus Audit Guidelines says, wait a minute, you can define a subset and focus on them (steps to secure IT), and here are the criteria, here's how you evaluate how you're going to be successful. The IG may say this other stuff, but the response could be, 'Yes, I'm focused on the most important things.'"
Gilligan doesn't contend the IG audits are valueless. If anything he says, some agencies IGs do a better job than others in identifying problems with IT security. "I don't cast all of them with the same brush," he says, adding:
"But I think that a lack of objective criteria to some extent is a lack of experience in many of the IG shops and that serves to create a situation where even well-performing organizations can find that their IG gives them a poor report. On the other hand, sometimes you can find there's an organization that is not really doing so well gets a good IG report, not because there's an objective evolution against consistent criteria, just this particular IG organization perhaps is not as experienced or not as focused."
Gillian says government IGs as a group need to provide better IT security training to government auditors and develop consistent criteria in evaluating the security of IT systems because many constituencies, including Congress, give significant weight to their findings:
"They're looking for independent corroboration; it's an important part of our governing system. But right now I don't think it's often adding the value it could if we mature it a bit more."