All EDR/EPP solutions record data, but true protection requires recording absolutely everything from every endpoint.

Cyber attacks and the security products designed to protect against them have both developed by leaps and bounds in recent years. Antivirus (AV) or ‘next-generation antivirus’ (NGAV) tools provide a good level of protection even though they only record partial endpoint data. AV/NGAV can also provide a high level of visibility once an alert is triggered. Collecting, storing and analyzing everything that happens on every endpoint is a massive undertaking that requires strong hardware and advanced Big Data techniques. Storage and every query are expensive. This begs the question, ‘Why should my organization invest in a Big Data powered EDR tool?’ and ‘Is it even necessary to record and store every activity that occurs in the operating system?’

Short answer: Yes

If some data can deliver ‘pretty good’ security, big data can deliver excellent protection. When your organization’s digital assets are at stake, there is no such thing as ‘good enough’. Below are 4 reasons collection of all endpoint OS events and analysis with big data techniques out-perform AV/NGAV and EDR solutions that settle for partial data.

Advantages of Big Data EDR:

1) Long attack timelines
Cyber attacks can span several hours, days, or even weeks. Often the first phases can appear benign and fly under the radar of your security products for some time. Eventually, the attack moves into a second ‘attack’ phase, but without all the historical data security tools are not able to correlate all the information and therefore cannot reliably identify the attack.

2) Empower threat hunting
Successful threat hunting requires access to all the endpoint data from the period leading up to the attack, including everything that happens in memory, file, registry, etc. If a security product is limited to the data collected from existing alerts and basic data elements i.e.; file names and hashes, threat hunting will not likely produce meaningful results. Effective threat hunting requires the ability to quickly analyze all related data from all endpoints, over a long period of time in order to assemble a coherent, complete attack story.

3) Thwart sophisticated evasion attempts
The first rule of crime is ‘don’t get caught’. Thus, the primary goal of malware authors is to discover a new vulnerability in security tools and exploit it to evade detection. Every day we read about their success (New LockPos Malware Detection Technique, ‘Early Bird’ Code Injection Evades Detection.) This is why security systems must collect and analyze absolutely everything to shine the bright light of big data analysis on every resource and action so that malicious activities can be discovered and malware authors have nowhere to hide.

4) Effective remediation
Once an alert is generated, analysts must quickly determine whether the alert represents a real breach and if it does quickly investigate and take steps to remediate. This requires a complete attack story that shows every step of the malware’s activity. Obviously, this requires complete historical data so the analyst can look back at everything the malware has done on the network since initial penetration. In the first moment an attack is discovered, it is impossible to know everything that has been affected, therefore the EDR system must automatically record everything about all processes.

Once you have decided that your organization requires the highest level of endpoint protection, how can you determine if a given EDR product really has full big data capabilities?

Today’s endpoint security market is crowded with Endpoint Protection Platform (EPP) solutions that perform some Endpoint Detection and Response (EDR) functions.

Ask these four questions to determine if your EDR is Big Data empowered:

1) What type of data is stored long term?
Even when no malicious activity is detected, data must be stored in the event it will become relevant in the future.

2) Is data stored in a central repository and optimized for queries and investigation?
EDR should have high-performance querying capabilities. Ask specifically about average query times.

3) Can you ‘turn back the clock’?
Historical investigation should allow you to easily look back and see what happened on a specific endpoint in the past, regardless of whether an alert was triggered.

4) Can benign activities trigger alerts because of other suspicious characteristics?
Sometimes a completely benign activity is an important clue to uncovering malware. For example, if the benign activity has never been seen in your environment, such as a process running on only one endpoint initiates communication with a port that has never been used.

Tomer Hevlin is Product Manager of Cyberbit EDR