Okay, keep in mind that I’m a Security Architect. I’ve been one since 2000 and I’ve been in IT since 1988. So I’ve been around and I’m damn good. But more and more lately, I’m being asked to Security Technical Risk Assessments on solutions and I’m starting to ask the question that EVERY Architect should be asking:
How are you justifying this expenditure? What’s the ROI?
I know, I know – you’re going to say that you need all these various technologies to reduce risk. I’m going to turn around now and ask you to prove it. And please don’t say that it’s ‘industry best practice’. If that’s your answer, please point me to the specific document that says that you implement this technology and that technology. Because you can’t.
I can design with the best of them and my favourite security implementations are anything to do with Identity and Access Management or SIEMs. Because there’s an actual Return on Investment. I have the calculators to show the cost savings associated with those solutions. But I cannot, in all good conscience, say how much the risk is reduced by using those solutions or encrypting or using Anti-Virus. Because, at the end of the day, it’s just gut feel.
Remember there are 2 types of risk; Inherent Risk and Residual Risk. Everyone always points to Inherent Risk – “you’ll get hacked” or “you need to protect against Ransomware”. Those are Inherent Risks. Putting in place all these various technologies will reduce that Inherent Risk and result in your Residual Risk. But what is that? Can you actually say what that Residual Risk is?
Here comes the arguments! I can just here them. “Neil, you should know better! Risk is Impact vs Likelihood”. True. What’s the likelihood of an attack actually succeeding if there isn’t a SIEM in place? .5% increase? 10% increase? You don’t know and neither do I. What’s the likelihood of something happening if there isn’t encryption for traffic in the clear? 5% more? 15% more? less? Again, you don’t know. No one does.
And here’s the kicker – we tell everyone you have to have both Encryption in place and a SIEM in order to reduce risk. Okay – is Encryption more or less valuable than using a SIEM? They can’t be the same. I would assume that sending something across the Internet unencrypted is going to be much more dangerous than if a SIEM isn’t in place internally. But that’s an assumption and you know what they say about assumptions (as I point at my butt).
So I had this brilliant idea to go looking for Insurance Actuarial Tables associated with Cyber Risk Insurance. Surely those organizations are basing their premiums on what controls are in place since those controls will have statistics of how successful they are in reducing risk. Guess what – they don’t have Actuarial Tables for Cyber Risk! And that’s not surprising considering how much IT has changed over the last 30 years. If the environment isn’t stable and the technologies aren’t stable, how do you know what the risk of a technology is?
Okay, I thought. How do the more mature organizations measure risk. So I looked at the Risk Matrix that a Utility that I worked with had. They had a lovely color coded table that had the ‘Frequency of Occurrence’ on one axis and the ‘Impact’ on the other axis. The Frequency was taken from known issues like Hurricanes hitting, which is measurable. The Impact was based on allegorical things like human injuries or reputational impact. The impact aspect works well for cyber risk. But frequency? How do you measure frequency of occurance, which is another term of Likelihood of occurance?
What we do now in the industry isn’t measure risk when we look at technical solutions. What we are doing is measuring against Security Policies and Standards and THOSE are just Approved Architectural Patterns. Not risk.
What is really needed is the research of companies that are impacted by a cyber security incident on what solutions that were in place and what solutions that weren’t but could have been useful. Break those down so that you can actually have actuarial tables on specific technologies like DLP or RBAC (I have a Reference Security Architecture which you could use if you want). But, until that research is completed, don’t tell me that I’m doing a Risk Assessment on a solution.
Interestingly, I just had a conversation with an Architect NOT in Security Architecture and we’ve both agreed that the term “Architecture Risk” is also NOT appropriate. So we need more in terms of measurements of what risk actually is, regardless of whether it’s a security risk or an architectural risk.
In the meantime, I’ll look for architectural patterns that are approved for organizations. But please don’t say that you are measuring risk. You’re not unless you can codify WITHOUT personal opinion what the risks are.
Hope that helps …