How Machine Learning Powers The Zero Trust Revolution

Image Source: PixabayZero trust is taking the security world by storm. It is a new paradigm that has tremendous potential to improve security in modern, distributed IT environments. However, zero trust requires advanced technology that can assess activity on computer networks, dynamically determine risk, and assign policies automatically.

In this article, I’ll explain these complex requirements of zero trust systems, and how machine learning is making them possible.

Zero trust is a security model that aims to protect networks against insider threats. It involves implementing security measures that enforce a “never trust, always verify” approach. Common zero trust security measures include microsegmentation, granular access control policies, and layer seven threat prevention.

John Kindervag at Forrester Research conceived this security model. He realized that traditional security models naively assume that everything within a network is trustworthy. As a result, these networks do not suspect that user identity is compromised, assuming all users act responsibly.

The zero trust model sees trust as a vulnerability because it enables insiders — malicious or otherwise — to move laterally across the network and access any data and actions allowed for the user.

Security leaders are moving from a traditional compliance-focused security approach to a risk-driven approach. The difference is that in a traditional approach the organization reacted to known compliance requirements, while in a risk-based approach it continuously evaluates the threat landscape and takes proactive action to prevent threats.

Organizations can rely on machine learning technology to evaluate user requests in real time, assess the security context such as the device, network, and related behavioral data, and generate a risk score. Access control policies can include this dynamic risk score to allow access, deny access, or require more stringent authentication.

Machines-learning powered risk scores can take into account a huge variety of factors to identify and respond to risks:

* Location from which access was attempted.
* Device characteristics including firmware type, browser type, and operating system.
* Device health including security patches, software updates and anti malware protection.
* User attributes such as role, seniority, and historical access logs.
* Time of day, month, and year.
* Unusual recent changes to privileges or requests for privileges.
* Unusual commands or access to resources.

In a large organization, there can be hundreds of applications and cloud services and thousands of users. There is a need to enforce standardized policies across this complex environment and reliably detect violations.

Machine learning can help by automatically adjusting access policies according to real time analysis of behavioral patterns. Instead of having IT and security teams continuously review access requests and manually grant access, this can be done automatically based on central policies the organization defines.

From the end-user’s perspective, this also provides a better experience because if the user’s request is legitimate, they receive fast access without waiting for manual approvals.

Traditionally, there was a drawback between improved security and a positive user experience. More secure systems were often frustrating and difficult to use. Organizations are now understanding that security should be transparent and non-intrusive for users — this will encourage users to adopt secure authentication mechanisms and will improve security overall.

Early generations of multi-factor authentication (MFA) had a negative impact on user experience, and were frowned upon by users and employees. Many organizations preferred not to adopt MFA, or to make it voluntary, to avoid scaring users away. However, the new generation of zero trust technology is changing that.

Modern authentication based on machine learning techniques, which takes the user’s security context into account, can enable a more streamlined user experience. One example is passwordless authentication systems that can authenticate a user by combining dynamic authentication factors. Another example is users logging in from the office using their corporate workstations might be logged in using SSO authentication tokens without even providing a password. Conversely, if a user tries to log in remotely or at unusual hours, the authentication system can convey that the login attempt is unusual and require additional authentication.

This type of differential authentication makes sense and will be supported by users, especially when their day-to-day login process is positive. Differential authentication, which is based on ML risk scoring, can turn users into partners in an organization’s security efforts.

Several security technologies play a key role in zero trust deployments:

* Next-generation antivirus (NGAV) — used to verify health of end user devices and enable zero trust systems to block access by compromised devices.
* eXtended detection and response (XDR) — used to cut across silos, collecting data from across a hybrid environment to detect and respond to sophisticated threats.
* User and event behavioral analysis (UEBA) — the engine behind behavioral analysis of users and service accounts, which underlies zero trust access methods.

NGAV improves on legacy antivirus by detecting new attacks that do not match a known file signature. It can protect against zero day attacks, fileless attacks, and evasive malware that changes its source code to avoid detection by traditional antivirus.

NGAV leverages several machine learning techniques to detect unknown threats:

* Static features — comparing individual pieces of binary code or assembly files to code segments from known malware.
* String analysis — analyzing printable strings from an executable program, which may appear in URLs, file paths, menus, API command, configuration files, or command line output.
* N-grams analysis — analyzing sequences of bytes extracted from suspected binaries.
* Entropy — analyzing statistical variance of code to identify encrypted or obfuscated code.
* Visualizing binary content — converting binary code of malware into image format (where every byte is a pixel in the image) and analyzing it using computer vision techniques.
* Control flow graph (CFG) — using a control flow diagram to compare suspected malware to known malware.

XDR is a new category of security technologies, which correlate and analyze data from a large variety of sources to identify evasive threats. This enables detection and proactive hunting for advanced threats in a zero trust environment.

XDR solutions leverage machine learning techniques to stitch together data points from network logs, endpoints, and cloud logs, to detect attacks and enable simpler investigation. This data has high dimensionality (typically with over 1,000 dimensions of behavior for one attack). Analysis of combined attack data leverages:

* Automated data pre-processing — XDR solutions aggregate data from many sources, normalize it and clean it automatically to enable its use in machine learning algorithms. In a sense, XDR solutions are a “data scientist in a box”, pulling real-time data from IT systems and turning them into standardized datasets that can be fed to unsupervised and supervised ML algorithms.
* Unsupervised machine learning — XDR solutions use unsupervised algorithms to create a baseline behavior of users and devices, and identify groups of peers, which helps determine normal behavior for a specific entity. It can then compare past behavior, current behavior, and peer behavior, to detect malware, command and control (C&C) communications, lateral movement, and data exfiltration.
* Supervised machine learning — XDR solutions identify broad types of entities on a network, such as Windows computers, email servers, or Android phones. It can then train large-scale supervised algorithms on traffic and events related to a category of entities, using known suspicious events as a training set. The algorithm then performs inference to identify suspicious events at runtime. This approach can dramatically reduce false positives.

UEBA is a foundation of many security systems that attempt to identify unknown threats, by comparing current behavior to a known behavioral baseline.

The input of a UEBA system is log data. This data is processed into events, ingested into machine learning algorithms, and the output is a risk score for each event. Commercial UEBA systems do this at a very large scale, operating on real-time big data with low latency inference.

An important part of the UEBA is its automated data pre-processing. It performs statistical analysis to see which data fields are suitable for analysis, extracts variables or features that can be useful for security analysis, and identifies which of its available algorithms best fits the data.

UEBA uses unsupervised and supervised approaches to analyze security data, dynamically deciding which type of algorithm is most suitable for each type of data:

* Supervised algorithms — UEBA leverages supervised algorithms when it has pre-labelled data. For example, if a security system provides a historical log of confirmed incidents, that is labeled data of malicious activity. The UEBA system can train on this data, and analyze any similar data to detect the same type of malicious activity. Algorithms commonly used in UEBA systems are logistic regression, linear regression, and deep neural networks.
* Unsupervised algorithms — in many cases, UEBA does not have a labeled dataset to work with. In this case, it uses algorithms like K-means clustering, which can learn similarity between data points. For example, a clustering algorithm can look at a large dataset of access logs, identify statistical properties like frequencies, histograms, and time-series structure, and group them into clusters with apparent similarity. Typically, outlier behavior will be grouped into the smallest cluster, and that is where the system can find suspicious or malicious access attempts.

In this article, I explained the basics of zero trust, and how machine learning is impacting the adoption of zero trust in organization. I covered three case studies that show how machine learning is used in zero trust technology:

* Next-Generation Antivirus (NGAV) — leverages a variety of techniques to analyze binaries suspected as unsafe, and identify malicious properties in new and unknown malicious software.
* AI in eXtended Detection and Response (XDR) — performs automated data pre-processing, creates massive datasets correlated from multiple IT systems, and uses supervised and unsupervised approaches to stitch together related events and identify attacks.
* AI in User and Entity Behavioral Analytics (UEBA) — collects data about entities in a network, dynamically decides which machine learning algorithm to apply to them, and computes risk scores based on labeled or unlabeled datasets.

In the years to come, we can expect to see AI and machine learning become even more pervasive in the security industry. Soon, a strong ML foundation will become a critical part of any zero trust security technology.