Qualitative risk assessments are a cornerstone security management tool. This type of assessment process is characterized by estimates of asset values, threats, vulnerabilities, and costs from anticipated exposures. Risk management frameworks are a way for managers to determine where to allocate resources when risk is at an unacceptable level. It is a process which relies heavily on the opinions, suppositions, and hunches of its participants. It is not formal, in the mathematical sense, and is very subjective. Essentially, it cannot be measured. It is fascinating that qualitative risk assessments are accepted with little opposition when their results are based on few if any facts. In an era when security metrics have a spotlight, why is it that we rely on something which is primarily subjective as a valid method of security management?
In contrast, quantitative risk assessments attempt to put a value on most, if not all, aspects of the risk assessment process. This requires an enormous effort to compile statistics and values which most likely produces a result that is no better than a qualitative assessment. It is also worth noting that statistics in quantitative risk assessments rely on probabilities, which are ultimately a mathematical best guess. This essentially returns us to where we started with a qualitative assessment. In this regard, the value of a qualitative assessment is the cost savings of avoiding a quantitative assessment. Real value indeed!
Do qualitative risk assessments provide any real value? Certainly they help managers subjectively identify areas where countermeasures are weak. But, are they really weak? Perhaps risk assessments help reduce costs by avoiding the allocation of resources in areas with low risk. But, are the threats truly mitigated by the existing controls? The unscientific answer is risk assessments are a perceived value that defies rational quantification. We are now approaching the same problem we face when forced to estimate the return on investment (ROI) for security processes and tools.
A subjective process lacking formalism and measurable attributes defies quantification. If it cannot be measured then its value cannot be proven. Thus, the value of risk assessments at best are difficult to ascertain and at worst a frivolous expenditure of resources.
Interestingly enough, the security profession embraces qualitative risk assessments. We will most likely continue to use them into the foreseeable future. They are an accepted management tool. However, we should consider improvements in risk based frameworks which move away from assertions and toward those things that are tangible. Associating risk assessments with security metrics is one possibility. At least we may start to incorporate measurable quantities which provide value whereas opinions have limited usefulness.
A change in the risk assessment paradigm may increase their value. I invite security professionals and researchers to consider this problem and propose solutions aimed at reducing opinionated estimates. By identifying attributes which are measurable we can improve security in a quantifiable manner as opposed to simply guessing.
