This blog entry was originally written by John Hoffoss, I am just migrating the post to the new SIRA site.
Mairtin contacted the board in November of 2011 with a question about sharing his thesis:
A bit over a year ago I did my MSc thesis on optimizing Information Security Investment, which effectively turned into looking primarily at quantitative risk assessment using the usual FAIR/Monte Carlo type approach.
While the conclusions aren’t anything new to people involved in SIRA, I thought it might be a good introduction read for those who are interested in the area but haven’t a clue where to start.
I was wondering if you’d be interested in linking to it or including it on the SIRA site? It’s mainly just sitting in my folders doing nothing so if it was helpful to others, I’d be thrilled.
This is the writeup he created for us that we’re finally getting published here.
-jth
If anyone has been following the SIRA mailing list for the last few months, you have seen some fantastic debate over the approaches to dealing with information security risk. While there is obviously a lot of incredibly talented people on the list, the common information security guy in the trenches may often be scared off by a lack of understanding of what on earth everyone is talking about! So I thought I’d share my brief story of how an information security guy like myself, who was originally more at home with penetration testing and reviewing tcpdump packet captures, ended up in the world of Monte Carlo simulations, aggregated risk and statistics!
Like most information security professionals, ever since I studied for my CISSP, I’ve read about Annual Loss Expectancy (ALE) and how it can be used to estimate the amount you should be spending on security controls. I subsequently saw references to ALE time and time again in other information security books and exams such as the CISM. As I came from a highly technical background, I just accepted this as management type material that was interesting but wasn’t all that relevant to what I did day-to-day.
However, as I started moving more towards management positions and started to help client companies (I work in consultancy) with information security management challenges, I started to ask a very simple question. If ALE is being referenced in the majority of books and certificates tailored to information security professionals, why haven’t I ever seen anyone actually use it? That started me thinking, and around five years ago I started digging for books that could help me understand why this was.
My first area of research was in metrics, particularly kickstarted by reading one book: Security Metrics by Andrew Jaquith. Straight away this book showed me what the key flaws were within ALE, although at the time I had no idea what on earth an “outlier” was! By the way, Andrew highlights the following problems with ALE: the inherent difficulty in modelling outliers, the lack of data for estimating probabilities of occurrence or loss expectancies, and sensitivity of the ALE model to small changes in assumptions. Read the book for more info!
The next key book for me came the following year and it was probably the most important book I’ve ever read in information security. That book was The New School of Information Security by Adam Shostack and Andrew Stewart. In this book, Adam and Andrew highlighted the need for evidence-based information security decisions and raised ideas around economics, physiology and even sociology and how they apply to information security. Now at this stage I knew I was starting to stray far from my usual comfort zone!
Following on from the areas of economics of information security highlighted in The New School of Information Security, I started doing more research in this area and came across the great research what Ross Anderson and his team were doing in the University of Cambridge. This started me thinking that perhaps the answer actually lied in more academic research that may not always make it into the mainstream of information security books and magazines. So I started reading, leading me to many very detailed papers that outlined models around security risk and optimising investment written by mathematicians and economics throughout the world’s universities.
Trying to understand these quickly became almost impossible due to my lack of statistics and deep mathematical background. A shame, but unfortunately I just simply didn’t understand what was going on, and didn’t have the time with work to dedicate to learning yet another new area!
And that’s where I left it, until I started to think about topics that I would like to write my MSc thesis on when completing my part time MSc in Information Security at Royal Holloway, University of London. Straight away I thought that this was the perfect time for me to spend time exploring this area in more detail, with the support of people who understand the academic area and would be capable of providing me further assistance in how to interpret it all! And that’s exactly what I did.
My basic objective for the thesis was not necessarily to find any ground-breaking new discoveries, but very simply to compile all the different types of research I could find in the area of optimising information security spending and try to make it understandable to someone with a background like myself; not an economist, not a mathematician, not a professor but a simple information security professional.
During this journey I came across many hugely interesting books and research that changed my outlook on information risk by people such as Doug Hubbard, Dylan Evans, Sam Savage, Jack Jones and a plethora of academic research by too many people to mention!
I looked at work done in the areas of risk management, corporate finance, economics and reliability to try and identify how other disciplines are dealing with similar challenges and found that a number of problems existed in the area of optimising information security investment, namely education, concept of return, lack of information, rating systems and ordinal scales, uncertainty and risk appetite.
Using these as a starter, I then went on to review each of these in order to further explain the problems I saw, and attempt to identify some high level solutions to these problems.
Now I don’t for a second claim that I’ve identified every possible bit of literature, nor that my analysis is flawless, but what I do hope is that if you’re working in information security and you’re interested in SIRAbut don’t have a clue what people are talking about, then my thesis might help out in giving a bit of background to what these guys are talking about.
What’s most interesting for me is that when I wrote this thesis around two years ago, SIRA didn’t exist and getting information around of information risk management was difficult to say the least! Now we haveSIRA with daily running discussions on almost everything I’ve covered in my thesis! It’s great to see how far things have advanced in terms of discussion and availability of information in such a short time.
Now, if only SIRA had existed three years ago before I started so I could have gone a lot further in my thesis!
Máirtín
Thesis: Optimising Information Security Investment
