There is a fundamental problem with many cyber-security products involving detection.
Bob Klein and Ryan Peters did a brilliant job outlining this problem in their Black Hat presentation, “Defeating Machine Learning - What Your Security Vendor is Not Telling You.”
To borrow Mr. Klein’s and Mr. Peters’ analogy, while introducing a twist:
Imagine a town with only one locksmith. Everyone buys the same lock from this locksmith. The lock looks strong and has good reviews, so why not? And the lock really is strong - all the latest technologies are inside, ensuring cracking it is as difficult as possible.
Then along comes a thief, intent on robbing a house. He approaches the first home and sees the lock on the door. No sense in sticking around and picking the lock where he can be seen; that’s risky. Instead, he makes his way down the street and one-by-one, he notices every house has the same lock. He realizes, “If I crack this lock, I can rob every house in town." So the thief goes to the locksmith and buys the same lock. He then heads to his basement with a two-liter bottle of Mountain Dew and gets to work on breaking the lock.
Days go by. Since the thief is determined and sophisticated, he finally cracks the lock. The effort was worth it. His next question is, “Which house do I rob first?”
The point is not that the lock was too easy to break. Better locks can always be built. The point is that if we accept the premise that an unbreakable, silver-bullet security product does not exist, and frankly, we’d be naive to believe otherwise, a sophisticated adversary obtaining an identical lock to ours makes our lock utterly useless against him.
It doesn’t take much imagination to apply this idea to cybersecurity.
As cyber-security defenders, how might we approach this problem?
So you’re going to settle for second- or third-rate products? I don’t think so. A better idea is to develop your own, in-house tools. If you’re capable of this, do it. Unfortunately, it’s out of reach of most organizations. Why? Lack of time, money, and resources are the reasons first cited, but equally important is lack of data. Getting detection right is hard. Really hard. To do it properly, a significant quantity of diverse data is required for analysis. This is an area vendors with many customers and partnerships have an advantage. Also, there really are vendors out there with new, innovative ideas with real-world experience to back it up.
This doesn't refer to new feature updates (those are nice though.) It means updates to the signatures / rules / algorithms / models the product uses for detection. By updating frequently, you cause the attackers some pain because time is no longer on their side. Think The Imitation Game, where Alan Turing and team attempt to break the Enigma code before the Germans changed their encryption key. Once the key changes, they must start over from scratch. Talk about frustration.
Updating frequently is good because we cause the adversary pain; however, frequent updates do not solve the problem: there is no guarantee the crack an attacker finds will be addressed in an update. In fact, unless you are talking about an extreme example like changing encryption keys, it probably won’t be.
Ultimately, frequent updates in this context are akin to taking aspirin when you have a backache; you mask the pain, but don’t fix the root cause. Not to mention, security products rarely focus on ease-of-use and intuitiveness, often resulting in painful upgrades. Instead of looking for a product that doesn’t upgrade their detection frequently, look for a product where upgrades are frequent, easy, and seamless.
Every human mind is different. Creativity spurs differentiation. Therefore, this definitively solves the identical lock problem. The challenge here is that it is not possible for any organization to find and retain the talent necessary to effectively protect an entire environment. You should still strive to hire as many smart, creative analysts as possible because they are gold in this industry. And like gold, great analysts are difficult to find, expensive, and everybody else wants them too.
People are good at certain things; machines are good at others. People cannot continuously hunt 24x365 even in small environments. Machines can. There is simply too much data, and we are dealing with a big data problem in cyber security. People aren’t good at shifting through vast amounts of data, but we are good at building, managing, and improving machines to handle that data. Machines are exactly the opposite of people in this regard. For this reason, arming analysts with state-of-the art, automated hunting capability is essential.
By writing unique detection rules, no one will have the same lock that you do. The identical lock problem is solved, but at what cost? Misconfiguration and improper tuning are concerns, and so is the ongoing maintenance of continuously adding new rules to stay ahead of adversaries. Writing a good rule is incredibly challenging. A good rule is one that detects malicious behavior (painful for adversaries to adapt to) instead of IOCs and artifacts (easy for adversaries to adapt to) while keeping false positives tolerable. When seen this way, the word ‘rule’ gives the wrong connotation; in order to detect behaviors like this, complex algorithms and data models are better suited for the job.
The options above have downsides, don’t actually solve the problem, or are impossible practically.
The best solution is for vendors to automatically and seamlessly introduce enough differentiation in their product's detection capability so every deployment is unique. If an adversary cannot get an identical copy of the product you use, he must interact with your environment to find a crack. Furthermore, he is less motivated because success doesn’t grant him the keys to the kingdom.
It sounds counter-initiative at first, but having the attacker lurking around your environment isn’t a bad thing when you have the tools to properly detect, monitor, understand, and respond to that attacker. Consider the alternative: the adversary prepares the attack safely in his basement, where you have no visibility into what he’s doing, how he operates, or where he’s going to hit you. The more we understand our adversary, the more we gain the upper hand.
Unfortunately, dealing with the Identical Lock Problem is often not part of vendors’ methodologies. We can fix this, though, by investing in products that can solve this problem (yes, they’re out there) and by asking the right questions to encourage the market to fix it.
Here are a few questions to ask your security vendor (and be sure to drill in):