How to Evaluate an EDR Vendor: The Questions That Actually Matter
Most advice on choosing an EDR — endpoint detection and response — is a spec-sheet comparison. Detection rates, feature checklists, a tidy table of green ticks. I’ve sat on the other side of those procurements, and I can tell you the spec sheet is the least useful thing in the room. Every serious vendor’s spec sheet looks excellent. They’re designed to.
What actually determines whether an EDR deployment succeeds or quietly fails has almost nothing to do with the feature matrix and almost everything to do with operational reality — who runs it, how much noise it makes, what it can actually do when something goes wrong, and what it costs you in human attention rather than dollars. These are the questions I ask when I’m evaluating a vendor for a real budget with real consequences, and they’re the ones the marketing doesn’t want you dwelling on.
If you take one thing from this article, make it the first section. It’s the question that sinks more EDR projects than any technical shortcoming.
Start with the question almost everyone skips: who is going to run this?
EDR is not antivirus, and the most expensive mistake organisations make is treating it like it is. You don’t install it and walk away. A good EDR is a detection engine — it surfaces suspicious behaviour and waits for a human to decide what it means and what to do about it. That human needs to exist, needs to be competent, and needs to be available when the alert fires, which is rarely during business hours.
So before you evaluate a single product, answer this honestly: do you have a security operations capability that can triage and respond to detections around the clock? If the answer is no — and for most organisations outside the large enterprise, it is — then the product you’re actually shopping for isn’t EDR at all. It’s MDR: managed detection and response, where the vendor or a third party runs the thing for you, with their analysts watching your endpoints 24/7.
I’ve watched organisations buy a best-in-class EDR, deploy it beautifully, and then have its alerts pile up unread in a console nobody was staffed to watch. That’s not a security control. That’s an expensive log generator and a false sense of safety. Decide who operates it before you decide what to buy. Everything else is downstream of this.
Detection: how good is it, really — and how would you even know?
Every vendor claims excellent detection. You cannot take their word for it, and you cannot test it credibly yourself in a two-week trial. So how do you get an honest read?
The closest thing the industry has to an independent, transparent benchmark is the MITRE ATT&CK Evaluations — a standardised exercise where vendors are run against emulated real-world attack techniques, with the results published openly. They are genuinely useful. But they are also routinely misrepresented, including by the vendors quoting their own results, so you have to read them properly.
They are not pass/fail, and they are not a ranking. MITRE deliberately doesn’t crown a winner. Any vendor telling you they “won” the MITRE evaluation is editorialising. Read the actual results, not the vendor’s slide about them.
Understand what was detected versus what was prevented versus what was merely visible. A technique that the tool saw and logged is very different from one it actively blocked, and different again from one it raised a clear analyst-ready alert on. The headline numbers blur these together; the detail doesn’t.
Check the configuration the result was produced under. Some strong results come from settings no one would run in production — maximum sensitivity, every module enabled, tuned for the test. Ask whether the configuration that scored well is the one you’d actually live with.
Used carefully, MITRE evals are the best signal you’ll get. Used as a marketing trophy, they tell you nothing. Learn to read them yourself.
The false-positive question that decides whether you’ll still be using this in a year
Here’s the question that separates people who’ve operated these tools from people who’ve only bought them: how much noise does it make?
An EDR that detects everything and floods your team with false positives is not a good EDR. It’s a liability. Alert fatigue is a real, well-documented failure mode — when analysts are buried under thousands of low-quality alerts, they stop investigating properly, and the one alert that mattered gets closed in the same reflexive sweep as the noise. The most dangerous EDR isn’t the one that misses a threat. It’s the one that buries the threat it caught under a thousand things that didn’t matter.
So ask the vendor, and ask their existing customers: what’s the realistic daily alert volume for an organisation our size? How much of the triage is automated correlation versus a human reading raw detections? How much tuning effort does it take to get the signal-to-noise to a workable place, and how long before it gets there? A tool you have to spend six months taming might be the right one — but you need to know that going in, not discover it after you’ve signed.
Response: can it actually do something, or just tell you the house is on fire?
The “R” in EDR is response, and it’s the half that gets least scrutiny in evaluations even though it’s what you’ll be desperate for during an actual incident.
When you’ve confirmed a compromised machine at 2am, what can the tool actually do? Can it isolate that host from the network instantly, so the attacker can’t move laterally while you investigate? Can it kill a malicious process, quarantine a file, pull forensic detail, remediate remotely — and can it do all that across hundreds of endpoints at once, fast, rather than one machine at a time? How much of that response is automated on a trusted detection versus requiring an analyst to click through manually?
In a live incident, the gap between “the tool told us” and “the tool contained it” is measured in minutes, and minutes are the difference between an isolated incident and an organisation-wide one. Evaluate the response capability as hard as you evaluate the detection. It’s the half that earns its keep on your worst day.
Living with the agent: footprint, coverage, and the double edge of deep access
An EDR runs an agent on every endpoint, and that agent has to earn its place on machines people are trying to work on.
Performance footprint matters more than you’d think. An agent that noticeably slows machines or breaks legitimate software generates a steady stream of complaints and, eventually, pressure to weaken or remove it. The best-protected endpoint is the one the user doesn’t resent.
Coverage has to match your actual estate. Windows, macOS, Linux, servers, virtual desktops, and any awkward legacy systems you’d rather not admit you still run. A gap in coverage is a gap in visibility, and attackers look for exactly those gaps.
And then there’s the double edge of deep access. EDR agents run with profound privileges on your systems — that depth is what lets them see and stop threats. But it also means a faulty update to that agent can do real damage at scale. The widespread 2024 incident in which a flawed update to a kernel-level security sensor crashed Windows machines across the globe is the defining cautionary tale here, and the lesson generalises to every vendor in this space: software with that level of system access is a systemic dependency. So ask the operational questions — how does the vendor test and stage updates, and crucially, how much control do you have over the rollout? The ability to canary an update to a small group before it hits your whole fleet is no longer a nice-to-have. It’s a question you ask in the evaluation.
Visibility, retention, and whether your team can actually hunt
Detection out of the box is table stakes. The organisations that get the most from an EDR are the ones whose teams can go beyond the vendor’s built-in detections.
Ask how much telemetry the platform captures, and how long it retains it — because retention costs money and is often quietly capped, and you cannot investigate an incident that unfolded over a window your data no longer covers. Ask whether your team can query that telemetry to threat-hunt proactively, and whether they can write custom detections tuned to your environment, or whether you’re locked entirely to what the vendor ships. The difference between a platform you can hunt in and one you can only receive alerts from is the difference between a security capability you own and one you merely rent.
Integration with the stack you already have
An EDR that doesn’t talk to the rest of your security tooling becomes another silo, and silos are where signals go to be missed. Look hard at how it integrates with your SIEM, your SOAR or automation tooling, your identity provider, and your ticketing — and at the quality of the API, because sooner or later you’ll want to wire it into something the vendor didn’t anticipate. The tool’s value compounds when it’s part of a connected picture and diminishes when it stands alone.
The cost question — and why the sticker price is the smallest part
Per-endpoint licensing is the number on the quote, and it’s the least important cost in the deal.
The real total cost of ownership includes the data and retention charges that scale with your telemetry, the cost of MDR if you’ve correctly concluded you need someone to run it, the professional services to deploy and tune it properly, and — the biggest and most invisible line item — the human attention required to operate it. An EDR that needs two full-time analysts to run well costs you those two analysts, every year, on top of the licence. Build the honest TCO before you fall for the headline price, because the cheap-on-paper option that demands a SOC you don’t have is the expensive one.
The vendor as a partner, not a logo
Finally, remember that you’re not buying a product so much as entering a relationship you’ll lean on during the worst moments of your security year.
What is the vendor’s support actually like when you’re mid-incident and need help now — not the sales experience, the 2am-during-a-breach experience? How good is their threat intelligence, and how quickly does it translate into protection for you? Is the company stable, and where is the platform heading? And — the question vendors least like — how hard would it be to leave? Understand the lock-in and the data portability before you’re committed, because the easiest time to negotiate your exit is before you’ve signed the entrance.
The Australian context
If you’re evaluating EDR for an Australian organisation, it’s worth placing it correctly in the local picture.
The ASD Essential Eight is the baseline most Australian organisations measure themselves against, and it’s fundamentally about prevention — hardening, patching, controlling what can run. EDR lives in a different and complementary place: it assumes that prevention will sometimes fail, and provides the detection and response layer for when it does. A mature security posture needs both. Treating the Essential Eight as the whole job and skipping the detection-and-response layer leaves you blind to the attacks that get through the front door — and some always do.
The local operational reality reinforces the very first question in this article. A great many Australian organisations, particularly in the mid-market, simply don’t have a 24/7 in-house SOC and aren’t going to build one. For them, the honest answer is almost always managed detection and response rather than a tool they’ll struggle to staff. And for organisations covered by critical-infrastructure obligations, the expectation to actually detect and respond to incidents — not merely to have bought a tool — makes the “who operates it” question a compliance matter, not just an operational one.
The bottom line
The questions that decide an EDR purchase are operational, not technical. Who runs it. How much noise it makes. What it can do when it matters. What it really costs in human attention. How the vendor behaves on your worst day.
Decide who’s going to operate it before you shortlist a single product, read the independent evaluations yourself rather than trusting the marketing of them, and weigh the response capability and the false-positive burden as heavily as the detection rate. The best EDR on the market, badly operated, loses every time to a modest one run well. Spec sheets won’t tell you that. A practitioner will.
This article contains no affiliate links — it’s reference material, not a buying funnel. Where Plain Text Security does earn commissions on product recommendations, it’s disclosed clearly; you can read the full affiliate disclosure any time.
