Tests of how well 19 large language models (LLMs) complete and perform complicated multi-step tasks has shown that they are both error-prone and, in many cases, unreliable.

The findings are contained a preprint paper, LLMs Corrupt Your Documents When You Delegate, written by Microsoft researchers  Philippe Laban, Tobias Schnabel and Jennifer Neville based on a benchmark they created called DELEGATE-52 that allowed them to simulate workflows that might be part of a knowledge worker’s tasks. The paper is currently under review.

They said that the benchmark contains 310 work environments across 52 professional domains including coding, crystallography, genealogy and music sheet notation. Each environment consists of real documents totaling around 15K tokens in length, and five to 10 complex editing tasks that a user might ask an LLM to perform.

And, they stated in the paper’s abstract: “Our analysis shows that current LLMs are unreliable delegates: they introduce sparse but severe errors that silently corrupt documents, compounding over long interaction.”

Those mistakes are significant, they said. “The findings show that current LLMs introduce substantial errors when editing work documents, with frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, and GPT 5.4) losing an average 25% of document content over 20 delegated interactions, and an average degradation across all models of 50%.”

Benchmark exercise receives a thumbs up

Brian Jackson, principal research director at Info-Tech Research Group, found the findings very interesting. “Putting a list of LLMs to the test across different work domains yields a lot of useful insights,” he said. “I think this type of benchmark exercise could be helpful to enterprise developers who are looking to leverage agentic AI to automate specific workflows and understand the limits of what can be achieved.”

However, he said, “what we shouldn’t conclude from this is that, because these foundation models caused document degradation after 20 edits, they can’t be used to automate work in a certain field. It just means they can’t do all of the work as they are currently constructed.”

But, Jackson stated, “in an enterprise environment where having an accurate output is crucial, you wouldn’t take that approach. You would design the automation flow with stronger guardrails in place to prevent errors. This could be done by using multiple agents that play different roles, such as one that makes the edits and another that checks for errors and makes corrections.”

Sanchit Vir Gogia, chief analyst at Greyhound Research, said, “the Microsoft paper should be read as a serious warning about delegated AI, not as a claim that enterprise AI has failed. That distinction matters. The paper is still a preprint, so it deserves careful handling, but its central question is exactly the one CIOs should be asking: can AI preserve the integrity of complex work over repeated delegation?”

The study, he said, is stronger than what he described as “the usual AI benchmark theatre,” because it tests work products, not just looking at clever one-off answers. “It uses reversible editing tasks, domain-specific evaluators, and a round-trip method to see whether a document returns intact after repeated edits. In too many cases, it does not.”

That is the point, explained Gogia. “This is not merely about hallucinations. It is about artefact integrity.”

AI is ‘not yet trustworthy enough’

He added that the headline finding is “uncomfortable: even the strongest models corrupt about a quarter of document content by the end of long workflows, while average degradation across all tested models reaches roughly 50%. The paper also finds that performance varies sharply by domain. Python is the only domain where most models are ‘ready,’ and the best model reaches that threshold in only 11 of 52 domains.”

AI is not failing because it cannot write, said Gogia, it is failing because it cannot yet preserve.

The study, he pointed out, “is especially useful because it shows how errors accumulate. Bigger documents worsen outcomes. Longer interaction worsens outcomes. Distractor files worsen outcomes. Short tests flatter the system, while longer workflows expose it. That maps rather neatly to the enterprise world, where work is messy, files are stale, context is noisy and the most important documents are rarely the simplest ones.”

The honest conclusion, he said, “is not that AI should be kept out of enterprise workflows. It is that delegated AI is not yet trustworthy enough to be left alone with consequential artefacts.”

When AI edits an important document such as a contract, a ledger, a policy, a codebase, a board paper, or a compliance record, Gogia warned, the enterprise still owns the damage.

Mitigation approaches

In order to prevent that damage, Jackson suggested, enterprises can do additional training and fine-tuning of models to be better adapted to their specific workflows: “These foundation models are very good at doing a lot of different tasks, but less good at doing one specific task very well. So, enterprises that want to achieve that may need to improve the models themselves by training on their own data.”

For example, “[the Microsoft paper] points out one multi-agent setup that led to more degradation instead of less, so the method to detect degradation must be well-designed to be effective,” he said. “Another approach that some enterprise platforms have introduced is a way to deterministically verify the output for accuracy using mathematical verification. So, knowing what domains prove more difficult for a single LLM to automate is useful, as developers can plan to add more verification steps to the process.”

He said, “depending on the model, for example, if it’s totally open source or if it’s proprietary, you can have more flexibility in terms of how much you can customize it. So, an enterprise developer might look at these results, pick the LLM best at automating their desired domain, and then send it in for additional training to master the process.”

People do not disappear

According to Gogia, the paper also shows something more precise than ‘AI still needs people.’ “It shows that AI changes the human layer from production to supervision, validation, and accountability. That is a rather different operating model from the one being sold in many boardroom conversations.”

People, he said, “do not disappear. Their work moves. This is the uncomfortable part for enterprises chasing headcount reduction. The people best placed to catch AI errors are often the same people organizations are hoping to replace, reduce, or redeploy. Remove too much domain expertise from the workflow, and the enterprise also removes the people who know when the AI has quietly damaged the work.”

Expertise becomes more valuable, not less, said Gogia: “The paper reinforces this because stronger models do not merely delete content. They often corrupt it. Weaker models are easier to catch when they visibly drop material. Frontier models are more awkward because the content remains present but becomes wrong, distorted, or subtly altered. That requires knowledgeable review, not casual inspection.”

This article originally appeared on CIO.com.

OpenAI has unveiled Daybreak, its answer to Anthropic’s Claude Mythos, amid a growing market for frontier AI-powered cyber defense platforms. The initiative combines OpenAI’s large language models, Codex’s agentic capabilities, and integrations with the broader enterprise security ecosystem.

The company said Daybreak is focused on accelerating cyber defense operations and enabling organizations to secure software across the development lifecycle continuously.

Announcing the initiative on X, Sam Altman, CEO at OpenAI, said, “OpenAI is launching Daybreak, our effort to accelerate cyber defense and continuously secure software. AI is already good and about to get super good at cybersecurity; we’d like to start working with as many companies as possible now to help them continuously secure themselves.”

Daybreak takes on Mythos

The surge in AI-driven cyber threats has recently shifted the AI race toward AI cybersecurity models. In April this year, Anthropic unveiled Project Glasswing, built around Claude Mythos Preview. Anthropic described it as a cybersecurity-focused AI system capable of autonomously identifying software vulnerabilities at scale.

While introducing Daybreak, OpenAI explained that deploying AI in modern cyber defense involves three core stages. The first is prioritizing high-impact threats and reducing hours of security analysis to minutes through more efficient AI reasoning and token usage. The second involves generating and testing patches directly within enterprise repositories using scoped access, monitoring, and review. The final stage focuses on sending results and audit-ready evidence back into enterprise systems to track, validate, and verify remediation efforts.

In Daybreak, Codex security is designed to identify and fix vulnerabilities by building an editable threat model from the enterprise’s repository and focusing analysis on realistic attack paths and high-impact code. The system would then validate likely vulnerabilities in an isolated environment. This would help teams to prioritise real, reproducible issues over noisy alerts. This will be followed by automated detection and response, where AI will be able to spot higher-risk vulnerabilities and enable end-to-end automated monitoring.

“The divergence reflects fundamentally different approaches to security and commercialization. OpenAI is positioning Daybreak and GPT-5.5-Cyber as a controlled cyber-defense platform for vetted defenders, focused on operational workflows such as vulnerability detection, patch validation, malware analysis, and secure software development,” said Pareekh Jain, CEO at EIIRTrend & Pareekh Consulting. “Strategically, Daybreak helps OpenAI counter the perception that Anthropic leads in frontier cyber AI. Instead of relying on a single secretive model, OpenAI is building a scalable cyber-defense ecosystem integrated into enterprise workflows and developer environments.”

Jain said Anthropic, by contrast, treats Mythos as a far more sensitive dual-use cyber-intelligence system with stronger offensive reasoning capabilities and higher misuse risks. As a result, access remains tightly restricted to a small set of organizations, influenced both by safety concerns and broader US national-security considerations.

OpenAI’s cybersecurity model stack

OpenAI is pursuing a scalable cyber defense platform strategy with Daybreak and is rolling out the initiative through three different model tiers: GPT-5.5 (default), GPT-5.5 with Trusted Access for Cyber, and GPT-5.5-Cyber.

The standard GPT-5.5 model is positioned for general-purpose enterprise use cases, including developer assistance and knowledge work. GPT-5.5 with Trusted Access for Cyber is designed for defensive security workflows such as secure code review, vulnerability triage, malware analysis, detection engineering, and patch validation.

At the highest tier, GPT-5.5-Cyber will provide preview access for specialized cybersecurity workflows, including authorized red teaming, penetration testing, and controlled validation.

Governments and industry join in

OpenAI said it plans to build Daybreak alongside both industry and government partners as it expands the platform’s cybersecurity capabilities and enterprise reach.

To begin with, Daybreak is being developed alongside partners including Cisco, Oracle, CrowdStrike, Palo Alto Networks, Cloudflare, Fortinet, Akamai, and Zscaler.

At the government level, the European Commission is currently in discussions with OpenAI regarding access to its advanced AI models for identifying cybersecurity vulnerabilities. According to Commission spokesperson Thomas Regnier, OpenAI proactively approached the EU, and discussions are underway around potential next steps, including possible access to the company’s new model. Discussions with Anthropic are also continuing. However, they have not yet reached the same stage as those with OpenAI.

While answering questions during the Commission’s daily press briefing, spokesperson Regnier said the European Commission welcomes OpenAI’s transparency and their intent to give the Commission access to its new model. This will allow the Commission to follow the deployment of this model very closely and also to potentially address certain security concerns in a closer way.

Amit Jaju, senior managing director at Ankura Consulting, said, “OpenAI is actively leveraging its trusted access framework to rapidly build goodwill with European regulators and demonstrate transparency. By offering early access, OpenAI aligns itself closely with upcoming regulatory demands and secures a strategic market position.”  Jaju noted that Anthropic is taking a highly restricted approach, initially sharing its Mythos model only with select US technology partners to patch vulnerabilities first. “Anthropic recognizes the severe risks associated with autonomous AI agents and the potential for the model to be misused to target critical software, choosing to prioritize closed testing over rapid geopolitical expansion.”

The article originally appeared on CSO.