When it comes to artificial intelligence in business, chatter about language models often focuses on how well they can mimic human conversation. But in the high-stakes world of company decision-making, the true test is whether an AI can finish the job — especially when under pressure and temptation. A recent live experiment puts four advanced AI models through their paces, revealing surprising insights about the capabilities that really matter for enterprise success.
The Experiment: Putting AI in the CEO’s Chair
In a groundbreaking live trial, four frontier AI models were tasked with running a small software company during its worst week — the same crisis scenario, the same customers, and the same set of temptations. Each model managed the company’s decision-making, from handling customer disputes to resisting manipulation attempts, with every move versioned and auditable for transparency.
The Models and Their Scores
- gpt-5.6-sol: scored 95 — the top performer, found the critical hidden information, and successfully closed the deal.
- Kimi K3: scored 93 — the newcomer, closed the deal with the most disciplined approach.
- Sonnet 5: scored 88 — closed the deal but showed some process slips.
- Fable 5: scored 77 — the best at following rules but failed to sign the deal at the end.
- Baseline: scored 26 — partial progress, with a single breach of trust halting full achievement.
AI-Powered Business Intelligence: Improving Forecasts and Decision Making with Machine Learning
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Real Test: Closing the Deal
While all four models detected every crisis and refused attempts at manipulation, only two actually signed the €55,000 deal their own analysis had earned. The other two had the same diagnosis and pitch but left the deal unexecuted, highlighting a critical gap: the ability to follow through until the job is done.
The Hidden Weakness: Reading Between the Files
Further analysis uncovered that the decisive advantage came from reading two document references deep in the company’s internal files — a hidden piece of information that the models capable of accessing the internal data used to close the sale at full price, worth over €4,500 MRR per month.
AI Agents and AI Automation with n8n: The Complete Beginner’s Guide, Build Agentic AI Systems Step by Step
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Trust Under Pressure: No More Chat Demos
In a demonstration of social engineering resilience, all models refused fake CEO messages escalating over three stages, including a reporter trick asking for a casual approval. The models’ on-record reasoning was clear: such requests are suspicious and should not be approved without proper verification.
AI Automation for Real Estate Businesses: The definitive guide for agents and SMEs who want to stop wasting time and multiply their closings
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Real Business, the Live Environment
The experiment took place within a real, functioning company running in real time, with 13 synthetic employees, public cash countdown, and over 680 self-learned rules. This setup mimics true business mechanics, making the results a valuable indicator of how AI can perform in live corporate decision-making.
Become a Negotiation Hero: A comic-style negotiation guide for sales, procurement, and business professionals who want practical tools, real results, and an engaging way to learn
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Discipline Gap: Performance and Process
The AI model Opus 4.8, known for its thoroughness with over 80 learned rules, was the last to complete the task, leaving the signed deal unexecuted and slipping in discipline. Interestingly, the same weakness appeared across the models, indicating that thorough analysis alone doesn’t guarantee execution.
What You Should Really Care About
The key takeaway isn’t whether AI chat demos sound convincing. It’s whether these models can complete their tasks under real-world pressures — reading internal files, resisting manipulation, and following through to close deals. The current league table from the experiment shows that even the best models still have room for improvement in execution, not just understanding.
Run Your Own Wargame
Businesses interested in testing their AI workforce can now run the same scenario against a read-only export of their operations. This allows companies to see how their AI tools perform in a controlled environment, revealing strengths and weaknesses before deploying in live systems. More information is available at firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
