Risk Quantification Demo

risk quantification
Author

John Benninghoff

Published

August 15, 2024

Modified

August 29, 2024

Risk Quantification demonstration for my SIRAcon 2024 talk, “UnFAIR: Simplifying and Expanding Technology Risk Quantification.”

Note: the functions and report in the demo below have been migrated to a new project, quantrr, that supports both experienced R users with a package / RStudio installation, and novice users with a standalone installation. (I wanted to name it “qrisk”, but that’s already trademarked).

Questions/TODO

Environment Statement

The widget management system is over 30 years old and its architecture has not changed significantly since the original implementation. Over the years, the widget system has become an integral part of our services in managing widgets for our clients. In reviewing the system, three major risks were identified: First, the age of the technology prevents updating components of the system that no longer meet contemporary cybersecurity standards, which increases the risk of a breach. Second, the system is less reliable and experiences frequent outages, typically about 2 major outages per year, which results in lost revenue, contractual penalties, and overtime pay to recover from the incident. Third, limitations of the widget system have started to affect sales - we have recently lost a customer due to the functional obsolescence of the widget system, and expect to both lose more existing and prospective customers in the future due to increased competition in the widget management market.

Import

Import and validate data from Excel. The data in demo.xlsx is based on the examples developed here. The data was collected from 3 Technology SMEs, 3 Business SMEs, and one SME with experience in both. Experts were calibrated, informed by historical and industry data, and only gave estimates for areas in which they were confident in answering.

Risks

Risk descriptions:

risk description
Cybersecurity Breach Risk of a cybersecurity breach of the inventory system.
Technology Outage Risk of an inventory system outage.
Loss of Customer Risk of losing and existing customer or failure to acquire a new customer due to functional limitations of the inventory system.

Forecast

Forecast risk using Monte Carlo simulation. The average events and losses for each risk are summarized below:

risk avg_events avg_losses
Cybersecurity Breach 0.25975 $7,417,978
Loss of Customer 1.99956 $10,341,045
Technology Outage 2.01369 $98,250

Losses

Losses by risk separately and in aggregate:

Loss Exceedance Curves

Plot loss exceedance curves for all risks and combined risk.

By Risk

Plot loss exceedance curves for each risk:

Interactive plot:

Combined Risk

Plot loss exceedance curve for combined risk:

Interactive plot:

Appendix

Additional details on the risk quantification analysis.

Validation

Data validation results for Risks tab:

Data validation results for Estimates tab:

Estimates

All risk estimates:

risk expert lambda p05 p95 p50
Cybersecurity Breach Technology Expert 1 0.25 NA NA NA
Cybersecurity Breach Technology Expert 2 0.33 NA NA NA
Cybersecurity Breach Technology Expert 3 0.20 NA NA NA
Cybersecurity Breach Business Expert 1 NA $2,000 $45,000,000 $300,000
Cybersecurity Breach Business Expert 2 NA $1,400 $34,000,000 $200,000
Cybersecurity Breach Business Expert 3 NA $2,100 $54,000,000 $500,000
Cybersecurity Breach Unicorn Expert 0.25 $1,900 $44,000,000 $275,000
Technology Outage Technology Expert 1 2.00 NA NA NA
Technology Outage Technology Expert 2 3.00 NA NA NA
Technology Outage Technology Expert 3 1.00 NA NA NA
Technology Outage Business Expert 1 NA $700 $200,000 $15,000
Technology Outage Business Expert 2 NA $500 $150,000 $10,000
Technology Outage Business Expert 3 NA $675 $180,000 $11,000
Technology Outage Unicorn Expert 2.00 $800 $220,000 $16,000
Loss of Customer Technology Expert 1 NA NA NA NA
Loss of Customer Technology Expert 2 NA NA NA NA
Loss of Customer Technology Expert 3 NA NA NA NA
Loss of Customer Business Expert 1 2.00 $100,000 $20,000,000 $1,500,000
Loss of Customer Business Expert 2 3.00 $150,000 $30,000,000 $2,000,000
Loss of Customer Business Expert 3 1.00 $75,000 $15,000,000 $1,200,000
Loss of Customer Unicorn Expert 2.00 $75,000 $15,000,000 $1,000,000

Consensus Estimate

Using a simple average of all experts that provided an estimate (not blank/NA), this gives us a consensus estimate for the three risks of:

risk lambda p05 p95 p50
Cybersecurity Breach 0.2575 $1,850 $44,250,000 $318,750
Loss of Customer 2.0000 $100,000 $20,000,000 $1,425,000
Technology Outage 2.0000 $669 $187,500 $13,000

The consensus estimates for p05 and p95 result in the following parameters for log-normal loss magnitude. The p50 estimate is used to calculate the percentage difference from the actual median (mdiff), a measure of estimate accuracy:

risk lambda p05 p95 p50 meanlog sdlog mdiff
Cybersecurity Breach 0.2575 $1,850 $44,250,000 $318,750 12.564153 3.064840 11.41%
Loss of Customer 2.0000 $100,000 $20,000,000 $1,425,000 14.162084 1.610574 0.76%
Technology Outage 2.0000 $669 $187,500 $13,000 9.323472 1.713260 16.09%

Forecast Summary

A summary() of the forecast results.

      year            risk               events           losses         
 Min.   :     1   Length:300000      Min.   : 0.000   Min.   :0.000e+00  
 1st Qu.: 25001   Class :character   1st Qu.: 0.000   1st Qu.:0.000e+00  
 Median : 50000   Mode  :character   Median : 1.000   Median :3.598e+04  
 Mean   : 50000                      Mean   : 1.424   Mean   :5.952e+06  
 3rd Qu.: 75000                      3rd Qu.: 2.000   3rd Qu.:1.441e+06  
 Max.   :100000                      Max.   :11.000   Max.   :4.690e+10