How machine learning impacts the cybersecurity arms race
During the early days of the Internet, there was little concern about security. Fundamental protocols like TCP and UDP were designed for use between trusting parties. Those days are over.
Cybercrime is now as much a part of the Internet ecosystem as e-commerce and social media, and it’s becoming increasingly threatening and expensive: Last year saw a 27 percent increase in the number of successful breaches in businesses over the prior year.
Meanwhile, the standard ensemble of defensive measures many businesses deploy offers limited protection against emerging threats. The pattern recognition capabilities of machine learning, however, are providing a new range of protective capabilities.
Businesses typically depend on a “block and tackle” approach to cybersecurity. First, prevent malicious code and users from entering your systems by blocking with countermeasures like firewalls and anti-malware. If that fails, disrupt malicious activity by isolating compromised devices, terminating malicious processes, and locking access to accounts.
Until recently, that overall approach was sound. Unfortunately, adversaries are constantly probing systems and evaluating new techniques to compromise systems.
Cybercriminals have responded to the widespread use of signature-based detection methods by developing polymorphic malicious code that changes patterns in the code without changing the code’s overall function.
Advanced persistent threats (APTs) use multiple techniques deployed to multiple systems over extended periods of time. APTs are designed to avoid triggering alarms or generating log entries that might indicate malicious activity.
With attackers investing so much to avoid detection, it is clear that security professionals have to invest in measure to improve detection. Machine learning is an increasingly important part of that investment.
Both supervised and unsupervised machine learning algorithms are increasingly being used in cybersecurity.
Neural networks are being adopted for big data-scale machine learning problems, especially since the development of deep learning architectures. Even simple neural networks can achieve over 90% accuracy when detecting misuse based on a small set of features, such as protocol, source and destination port, and payload length.
Given the large volume of network data available, is is not surprising that association rule-mining techniques, such as those used in the NetMine framework, have been used for anomaly detection.
Probabilistic graphical models are used in problem areas where uncertainty and missing or erroneous data can be a problem. This makes them a good fit for cybersecurity analysis.
Bayesian Networks, a type of probabilistic graphical model, have proven especially adept at working around the inherent uncertainty that comes with limited data. These techniques have proven especially useful when monitoring for malicious insiders.
Clustering is a form of unsupervised learning — i.e. it does not required labeled examples to learn how to classify data. Unsupervised learning techniques are useful when large amounts of data are available. For example, BotMiner uses clustering techniques to identify traffic from botnets.
Machine learning for cybersecurity has moved out of the academic and corporate research labs and is being deployed in enterprises.
Chronicle, a cybersecurity firm that emerged from Alphabet’s research group, applies machine learning techniques to big data generated by networks and devices. Chronicle’s security-as-a-service approach uses scalable services to collect and analyze patterns of activity within a customer’s IT infrastructure.
This approach demonstrates the importance of scalable compute and storage resources when applying machine learning to cybersecurity. Finding the proverbial needle in a haystack is a simple task compared to finding rare but related events in large volumes of network traffic, logs, and infrastructure configuration data.
Machine learning is also changing the way we protect endpoints. Cylance’s CylancePROTECT, for example, employs artificial intelligence and specialized machine learning techniques to analyze and categorize malicious code before it executes.
This is especially useful for preventing zero-day attacks that cannot be detected by signature-based detection methods. It can also enhance spear-phishing countermeasures that use site reputation to prevent a user from browsing to a malicious site.
In addition to analyzing code before it executes, Cyclane provides agents that hook into user-mode application programming interfaces and watch for behavior that indicates malicious activity. The agent can be configured to alert, block an API call, or terminate a process completely.
Cybercriminals have had the upper hand in the balance of power in cybersecurity. Complex, distributed systems are difficult to protect without constraining their functionality.
The “block and tackle” approach to security has not been able to counter advances in malicious techniques and advanced persistent threats. Applying machine learning, however, has the potential to alter the balance of power by improving the ability to detect and counter novel patterns in emerging malicious activity.