Running a security PoV that makes sense

When you're evaluating email security tools, you shouldn't have to choose between a fair test and keeping your users safe. Yet somehow, that's become the ask.

Here's a pattern we've seen. Organizations run a PoV comparing API-only tools with SEG solutions. Then someone suggests turning off core protections on the SEG (URL rewriting, attachment controls, advanced threat policies) to "level the playing field."

That's not leveling anything; it's just removing the field entirely.

Let's talk about how to evaluate these architectures honestly without creating risk, and why the quirks of API-only design make this conversation necessary in the first place.

Two architectures, different jobs to do

The core architecture differences aren’t just technical; they shape what each solution can and can't protect.

SEGs sit inline. They inspect every message before delivery, analyze links and attachments in real time, and enforce controls at the perimeter. (If something's dangerous, it never makes it to the inbox.) They inspect every URL or attachment as they are clicked by the user, offering another layer of security against dynamic threats.

API-only tools connect after delivery. They watch what lands in Microsoft 365 or Google Workspace, analyze behavior patterns, and attempt remediation, sometimes minutes or hours later, through API calls.

One prevents. One responds. Both can be valuable. But they're not interchangeable and evaluating them side-by-side requires an understanding of what each architecture can actually do.

Why someone might ask you to weaken your SEG

When an API tool runs behind a fully configured SEG, something predictable happens: most threats never make it to the analysis stage.

Commodity malware gets blocked. Known phishing campaigns stop at the gate. Obvious impersonation attempts don't get through. What remains for the API tool to analyze is a significantly smaller, cleaner dataset.

That's the ideal security outcome: layers working together, with the most dangerous stuff stopped early. But this makes it harder for an API-only vendor to demonstrate value during a short evaluation.

So, the workaround becomes: disable the SEG’s most effective controls, let more through, and create more "detections" to report.

The problem? You've just made your environment demonstrably less secure for the duration of the test.

What you actually lose when you dial things down

This isn't a hypothetical risk. Weakening your SEG removes protection at three stages where attacks succeed or fail:

H3 Pre-delivery: Stopping threats at the gate

With a SEG fully enabled, dangerous emails never reach inboxes. Deep inspection catches malicious URLs and attachments. Impersonation techniques get identified. Threats get blocked before delivery. Relax those controls, and you're flooding Microsoft 365 with content that shouldn't be there. The API tool can only respond after delivery, after the message is already visible, readable, and actionable. Even a 30-second delay is 30 seconds of unnecessary exposure.

Click-time: Protecting users when they engage

URL rewriting is among your most valuable controls. When users click links, a properly configured SEG:

• Routes clicks through real-time inspection
• Follows redirects to final destinations
• Analyzes pages at engagement
• Blocks access to sites that turned malicious after the email arrived

Turn that off during a PoV, and users will click directly through to whatever destination the attacker chose. API-only tools can't intercept clicks. They can only try to pull the email back after the fact, assuming the API connection works, Microsoft's processing isn't delayed, and the user hasn't already given up credentials. That’s backwards.

Internal and outbound: Where API-only tools go dark

API tools typically analyze inbound email from external senders. What they don't control is what happens inside your organization, where compromised accounts and lateral phishing operate. A SEG that protects internal and outbound communications catches:

• Lateral phishing from compromised accounts targeting colleagues
• Data exfiltration to external addresses
• Malicious insider activity using legitimate accounts
• Policy violations on what leaves your environment

These attacks use authorized accounts and trusted domains and never touch external mail flow. API-only tools watching inbound messages will miss them entirely.

How to run a fair PoV without breaking things

You don't need to compromise security to evaluate vendors meaningfully. You need better success criteria.

Keep your SEG fully enabled

Any vendor requiring you to weaken existing defenses isn't demonstrating value; they're manufacturing it. URL rewriting, attachment sandboxing, and threat policies should stay active throughout any evaluation.

Measure exposure, not just detection counts

Ask every vendor:

• How long was the threat visible to users?
• What happens if API connectivity degrades?
• Can you prevent clicks, or only clean up afterward?

A detection that occurs after a user reads, clicks, and enters credentials isn't prevention. It's incident response.

Evaluate complete security outcomes

Compare what each vendor actually stops:

• SEG solutions: Block threats pre-delivery, protect clicks in real time, remediate post-delivery, control outbound and internal email
• API-only tools: Analyze post-delivery inbound messages when APIs permit

One is comprehensive platform protection. The other is a supplementary layer.

What a good evaluation looks like

The goal isn't to make one vendor look bad or create an artificial scenario where everything succeeds. It's to understand what each solution contributes to your security posture, and where gaps exist.

If you're evaluating an API-only tool, test it in a realistic configuration. Run it behind your existing SEG with everything enabled. Measure how it handles the sophisticated threats that make it through your first line of defense. Understand what it can't address such as click-time protection, internal threats, and outbound controls. Then, decide whether that supplementary analysis justifies the investment and operational overhead of managing another vendor relationship.

Or keep your platform-level protection running comprehensively and let it do what it does best: stop threats before, during, and after delivery across your entire email environment.

Run your PoVs smarter

Point solutions solve point problems. Platforms solve the entire problem.

If an evaluation requires weakening the protections you have in place today, that's a signal worth paying attention to. Security tools should prove value on their own merits, and not by handicapping the competition.

Run your PoVs smarter. Keep your users safe. And if anyone suggests turning off URL rewriting to make things "fair," maybe that's a good time to ask what normally happens when unusual requests get made.