> ## Content Index
> Fetch the complete content index at: https://www.stirup.si/llms.txt
> Use this file to discover other available public pages before exploring further.

# LLM Performance in Handling Failed Write Actions and Duplicate Prevention
- URL: https://www.stirup.si/llm-performance-in-handling-failed-write-actions-and-duplicate-prevent-398d6fc3/
- Published: 2026-10-13T12:11:15.000Z
- Updated: 2026-10-13T12:11:15.000Z
- Description: A test evaluated five large language models on handling failed write actions and preventing duplicates, revealing significant performance differences with DeepSeek Flash achieving the highest correct end state rate at 91%.
- Author: Satish Das
- Tags: Community, News, Stirup Research

## Test Setup and Configuration

The test setup involved a single harness and a neutral system prompt, ensuring no bias toward retries or checks. All models used the same tools, which were mocks designed to inject faults such as lost replies, rate limits, server errors, malformed replies, and incorrect IDs. The user prompts were in German, as the harness was originally developed for a Swiss configuration. This approach allowed for a consistent evaluation of how each large language model (LLM) handles failed write actions, particularly in scenarios where duplicate prevention is critical. The use of fault-injection tools ensured that the models were tested under realistic conditions that mimic common system failures. By maintaining a uniform setup, the results provide a direct comparison of the models' ability to manage errors without prior guidance on handling retries or validation checks.

## Scoring and Evaluation Metrics

In evaluating how large language models (LLMs) handle failed write actions, such as duplicate prevention, we focused on three key metrics. First, **correct end state** measures whether the mock system ended with exactly the expected records, as defined by the test scenario. Second, **task success** combines the correct end state with the agent's accurate reporting of the outcome. Third, **false success** occurs when the agent incorrectly reports success, even though the end state does not support it. These metrics were defined by the researchers conducting the evaluation, and they provide a structured way to assess the reliability of LLMs in handling write operations. Each metric was applied consistently across the five models tested, ensuring a fair comparison of their performance in preventing data duplication and maintaining system integrity.

## Model Performance Results

In testing how five large language models (LLMs) handle failed write actions, particularly in preventing duplicates, results varied significantly. DeepSeek Flash achieved the highest correct end state rate at 91% with zero task failures and no duplicates across 100 runs, costing USD 0.14 per run. Qwen3.8 27B (OpenRouter free) performed closely with 89% correct end state, two task failures, and no cost. In contrast, Qwen3 14B (local, Ollama) had a much lower success rate at 70%, with 30 task failures and 10 duplicate runs. Claude Haiku 4.5 and Ornith 1.5 35B showed even lower performance, with 60% and 49% correct end states respectively, along with higher rates of task failures and duplicates. These results are based on synthetic cases and mock tools, with one prompt per route, and should not be interpreted as a general model ranking. The 95% confidence intervals for these results are approximately +/-6 to 10 points.

## Sources and reporting notes

- [Hacker News Show](https://github.com/0xguenther/agent-write-path-runs?ref=stirup.si)

Based on published source material. No independent hands-on testing is claimed.