0wnage and the null hypothesis

H₀: Our systems are not hacked.

That is what management wishes to hear all the time and expects to hear it with absolute certainty.

– But …

There are no buts in such matters for management, right? Oh but there are…

In reality there is no way to know whether the systems we maintain are hacked or not. We can only know with absolute certainty that they are owned and this only when the fact is detected. To help management understand this, use a “simpler” example:

H₀: This message is not spam

Work with the not-spam example and the table above. It seems fairly straight forward that if your anti-spam measures are relaxed you receive a lot of undetected spam (Type II error) and if you tighten the controls you risk having legitimate messages characterized as spam (Type I error).

In a similar fashion you can detect that your systems are hacked and therefore you can reject H₀. You can have your Intrusion detection systems, monitoring systems, processes or other controls “cry wolf” (a Type I error) or they may stay silent while in fact infiltration has happened (a Type II error). A Type II error means that an opportunity to detect a breach was lost.

So you see management, we cannot under absolute certainty assure you that we are and will remain unbreakable till the end of time. After all, if you really think about it hard, time is on the side of the blackhats. We can only provide you with data that we are doing our best with the tools you are providing.