Before an AI analyst gets write access

An agent reads telemetry, investigates an alert and proposes a fix. The difficult part starts when it is allowed to apply that fix.

I want AI in the SOC. Enrichment, query preparation, summaries and gathering evidence are useful jobs for it. But a good result on alert classification does not tell me whether an agent can finish an investigation, or whether I should give it permission to change a production system.

Test the whole investigation

The SIABench study separates alert triage from deeper, multi-step incident analysis. That distinction matters when evaluating a product: a correct label is one part of the analyst’s job.

The Cyber Defense Benchmark asks agents to find malicious events in large sets of Windows logs without guided questions. In its April 2026 evaluation, the best tested model correctly flagged an average of 3.8% of malicious events. That is a result for a particular benchmark and set of models, not a permanent verdict on AI. It is a reason to test the workflow you actually intend to delegate.

The log is an input an attacker may control

AIOpsDoom demonstrates how manipulated telemetry can steer an operations agent into harmful actions. The study concerns AIOps rather than SOC products. My concern is the shared trust boundary: an agent reads untrusted operational data and then uses privileged tools.

What I would ask before enabling remediation

• Which actions are available, and which require a person’s approval?
• Can text in a log influence tool arguments, destinations or permissions?
• Can an analyst inspect the evidence behind a decision?
• What happens when a proposed action is wrong, and how is it reversed?

I would start with read-only access and explicitly bounded tools. Changes to IAM, firewall rules and endpoint containment need a separate approval path. I would measure completed investigations, missed attacks, incorrect actions and analyst interventions, rather than choosing a product from one accuracy figure.

If an analyst has to check the AI, repair its investigation and then investigate the original alert, the work has not disappeared. It has changed shape.