What is Anomaly Detection?

In many cases, malicious attacks initially manifest as unusual behaviors of user accounts, system activities on the network, or network traffic patterns. Detecting that anomalous behavior is important to further investigating. This is not threat detection because the anomaly may or may not be a threat.

An everyday example of automated anomaly detection is used by credit card companies. The detection system determines the normal patterns of spending behavior for a person. How much is typically spent. The stores and locations where the card has been used. The system uses these patterns to identify transactions outside the normal behavior. The person may have been saving up for a trip to the Canary Islands, or suddenly make a very large purchase, or this abnormal behavior could be evidence of fraud.

Similarly, an employee regularly accesses business applications from the office between 9:00am 5:00pm. One day that employee logs into those applications at 3:00am. That is anomalous behavior. It could mean nothing. But, it could be symptomatic of an external malicious attack or hidden attacker inside the network.

The first step in detecting anomalies is baselining what normal, daily behaviors looks like. This is where machine learning comes in. Machine learning algorithms can rapid sift through massive amounts data to (a) look for patterns of behavior and (b) use those patterns to identify anomalies. However, these algorithms are only the beginning of an effective solution.

Algorithms do not actually understand the data they are analyzing. For example, a machine learning algorithm could identify a Wednesday pizza buying pattern in the credit card transaction data. However, a fraud management expert would understand that a pizza buying pattern isn’t a meaningful indicator of fraud. They would exclude it from an automated fraud detection solution being built. The same is true for automating anomaly detection for a cybersecurity solution. An effective solution would combine the machine learning technology with expert understanding of the data being analyzed.

In addition to machine learning enabling cybersecurity, technology is also changing the workplace. Read more thought leadership on Corvil's blog: Learning Ecosystems: The Real “Deep” Learning.