Evan Schuman's StorefrontBacktalk

In the Visa data theft case, much has been made of a security audit that cleared the clearinghouse that improperly handled the data. But security experts see audits rarely shedding much light on what the CIO doesn’t already know.

Security audits?often conducted by the same firms that handle financial audits?are supposed to be an outsider’s expert view of how safe and secure a company’s systems are. But in reality, many security audits today are executed under such tight restrictions that they reveal little that the CIO didn’t already know.

In the recent massive Visa data theft case, Visa and CardSystems officials tried hanging some of the blame on the company?Cable & Wireless Security?that conducted an audit for CardSystems long before the data loss, saying that they should have identified then some sloppy practices.

Should they have? Could they have? Security experts agree that audits are only as strong as the instructions the auditors are given.

“Some 95 percent of the organizations that I go to [for security audits] believe they are getting more than they are,” said Craig Wright, the manager of computer assurance services with the BDO Chartered Accountants and Advisers firm in Sydney, Australia.

A typical example, Wright said, involves the company that hires so-called white hat hackers who get paid to try to break into systems and then report discovered flaws to that company?and sees the results of a bunch of automated tests that show no vulnerabilities. But when Wright’s team looks at the system, they discover users who had full network access four years after leaving the company.

Not only are most data thefts internal, but “the people inside your network know what to take and what is the most valuable,” Wright said.

The vast majority of those internal assaults are simply authorized users who exceed their authority. “One user might try transferring a lot of money to himself. ‘Gee, it worked. I’m leaving,'” said Fred Cohen, the CEO of Fred Cohen & Associates and the man who is most widely credited with the earliest anti-virus efforts in the 1980s.

Audits frequently overlook another common security weak point: system interdependencies. A manufacturing plant, for example, might be wisely cut off from the rest of the network to guarantee that viruses and other problems can’t get through.

But those same companies that were smart enough to isolate their manufacturing plants often link crucial financial systems with an easy to break-into domain name server, Cohen said.

Monkeying with the DNS is a very easy way to engage in identity theft and data theft because it takes traffic intended for a legitimate company and sends it to a duplicate site to grab passwords and many other pieces of information.

“DNS, everyone forgets about it. If you compromise your DNS server, you basically own the site,” Wright said. “With Citibank.com, do you remember the IP address or Citibank.com?”

A smart hacker wouldn’t be greedy and might redirect only a small portion of the site traffic to the dummy site?say, perhaps one out of a thousand customers?making it almost impossible for site or law enforcement authorities to recreate the hack, Wright said.

How can audits be used more effectively by companies? First, executives must understand what they are buying. Security audits are often confused with security assessments.

Wright points to the Cable & Wireless Security incident as a good example of the confusion. Cable and Wireless said it had “completed an audit of the systems at CardSystems. This is not correct. They have had a vulnerability assessment. A vulnerability assessment is in no way an audit,” Wright said. “Vulnerability tests are generally about 13 to 15 percent as effective as an audit. Also, a well defined vulnerability test takes just as long as an audit. The level of skills required for an audit is far greater than (those needed for) a vulnerability test.”

Adding to the confusion is that another well-regarded security expert?Mark Rasch, the former head of the U.S. Justice Department’s computer crime unit and the government’s prosecutor against Robert Tappen Morris, who created the first major Internet worm?agrees that the terms are confusing and took the opposite position. “To me, an assessment can be more comprehensive than an audit,” said Rasch, who today serves as senior VP and chief security counsel for Solutionary Inc.

Cohen comes in right down the middle, arguing that neither an audit nor an assessment is better, that they are merely different and it depends on how both are executed.

But both approaches suffer from the same problem, which they also share with their financial audit cousins: the investigators are told what to look at and what to ask by the very people they are trying to evaluate.

“There are big problems with this. Management really wants you to say good things about them,” Cohen said. “They might want you to only look at the things that are problem-free. The auditors only measure what they’re asked to measure.”

Are company executives?including CIOs?being fooled into thinking that their security audits are doing more than they’re doing? Or are the executives knowingly taking these measures to cynically convince their boards and shareholders that more is being done than really is?

Attorney Rasch argues it’s a bit of the second, but it’s more apathy then ill intentions.

These audit-purchasing executives “are not buying security. They are buying a piece of paper to wave around to say, ‘We’ve met these standards,'” Rasch said. “A lot of companies are spending money to pass the audit because they have to. They don’t want to be secure. They don’t care about being secure. You can be incredibly vulnerable and still pass an audit or assessment.”

Asked if he agrees with Rasch about the lack of concern about security preparedness, Cohen said that he did. “They don’t care and maybe they shouldn’t. If you’re running a business, the goal is to make the business run right,” he said, adding that a lack of security can threaten severe damage to the company, but not necessarily to the executives in charge of security.

“The punishment doesn’t come back to the executives. It comes back to the shareholders,” Cohen said.

But Rasch stressed that he applauds the efforts that Visa has undertaken because?although certainly not comprehensive?the audit requirements are making companies aware of some issues, which is more than they were before.

“The goal of the Visa standards is to make sure that people are doing something. In the absence of the standards, they didn’t have to do anything,” Rasch said.

A key CIO who has been aggressive on security auditing procedures is William Morgan, CIO for the Philadelphia Stock Exchange. Morgan says that now may be the time for customers to consider performing?and paying for?their own security audits of contractors’ systems.

Had that happened with CardSystems and had Visa paid for its own audit and issued the instructions for that audit, the final outcome might have been avoided, assuming an audit had been performed after CardSystems began retaining data improperly.

“It’s a question of corporate independence,” Morgan said. “Maybe [the audit] has to come from the customer’s side.” Morgan added that his exchange routinely conducts security audits on customers and suppliers and that the SEC routinely audits his team’s security.

At his exchange, Morgan said that he finds audits of his operations?operating on the instructions from his company’s audit committee?quite useful.

“I think it’s a good idea to have an independent source. I see it as free consulting, and I’m not really threatened by it,” Morgan said.

“A lot of times, we’re all so busy, making sure that we’re leveraging our technology. Sometimes, you need someone from the outside to look at these security issues more, as long as you get competent people doing it.”

But Cohen does find fault with many of the audit firms. For example, when the auditing firms needed to certify that their audits were accurate with the new threat of imprisonment, auditing fees more than doubled, increasing on average $4.5 million, Cohen said. If the new costs are to perform the audits properly, what, he asked, were companies paying for during all of those earlier years?

Cohen also faults auditing firms for often using lower-level COBIT standards and not the more extensive COSO ones. “The auditing firms are auditing to the wrong standards?and they know it?because they have checklists that junior auditors can fill out,” Cohen said.

Companies should also create chief security officer positions and make sure that they do not report into the CIO, Cohen said. “Managing the information technology risks is a new area and the CIO is not the person who should be in charge,” he said.

Why should the CIO not oversee the security issues? First, part of the role is to perform oversight on how data is being managed, so the overseer can’t report to the overseen, Cohen said.

Secondly, there are many areas of security management?such as building security and HR hiring/firing functions?that have little to do with a CIO’s jurisdiction. A company’s acquisition, for example, brings in tons of new people who will be given varying levels of access to confidential company data. Who is running the background checks on these hires?

Also, Cohen said, some of the security decisions will be based on corporate priorities that would also typically be outside the scope of the traditional CIO.

Experts also point to constantly growing extranets and intranets as an area often ignored by audits and assessments. With the soaring number of contractors, suppliers, distributors and key customers with varying levels of network access, the internal threat is truly towering over external break-ins as an area of concern.

Wright adds that extranets are even more frightening as consolidation makes many of those contractors and subcontractors also the contractors for a company’s competition. Add to that the coopertition projects?where a rival works with a company on one project while competing against it on another?and there are lot of ways internal data isn’t looking that internal anymore.

And an audit that focuses solely on known external threats is looking less and less helpful.