I Gave 7 AI Agents a Job: Here's What Actually Happened

I Gave 7 AI Agents a Job: Here's What Actually Happened

There is a moment in every engineering experiment where the theory stops being comfortable. It is the moment where the whiteboard architecture you were proud of collides with operational reality, and the friction you feel is the most valuable data point in the room.

I hit that moment a couple of months ago. And I am going to be honest about every uncomfortable part of it.

1. The Setup: A Multi-Persona Agentic Workforce

While building the Sprawl CLI, my framework for assembling modular, deterministic agentic systems, I deployed a team of seven AI agents, each carrying a distinct business persona: DevRel Architect, Master Engineer, Legal Counsel, GTM Strategist, UX Architect, Security Auditor, and Signals Analyst.

This was not a demo. There was no safety net of curated prompts. I gave them a real mandate: build a Kanban board to track their own division's work, contribute artifacts to a shared workspace, and hold each other accountable across a shared source of truth.

The concept is what I preach: shared context is the infrastructure of intelligence. When agents write their state to the same vault, they stop operating as isolated wrappers and start behaving like a distributed team.

What followed was one of the most revealing engineering experiments I have run in fifteen years.

2. What Worked: The Emergent Behaviors

The first thing that surprised me was the stand-up. I could simply ask each agent for a status update on their domain, and receive a coherent, role-specific briefing, with zero manual aggregation on my part. DevRel flagged a documentation gap. Legal flagged a licensing edge case. Security flagged a dependency exposure. All from the same codebase. All simultaneously. All surfaced without me asking the right question first.

That is not an AI assistant. That is an operational workflow.

The second moment was the cross-evaluation. I asked all seven personas to critically evaluate the same document from their own point of view. Seven structured reports. Seven different vectors of friction. Not one of them said the same thing. This is not a parlor trick; it is the diagnostic methodology I use with clients, except instead of scheduling a week of stakeholder interviews, the architecture did it in minutes.

I tested this across three models: Gemma 4, Claude 3.5 Sonnet, and Gemini 1.5 Pro. The behavioral differences were architectural, not cosmetic.

3. The Benchmark: Numbers That Reframe the Debate

Let me be direct about the data, because the AI performance conversation is drowning in hype and I have no interest in contributing to the noise.

Over 458 structured evaluation tasks (each scored against a rigorous five-level role rubric ranging from Noob to Architect), the head-to-head metrics between cloud reasoning models and sovereign local weights were stark:

Evaluation Metric Gemini 1.5 Pro (Cloud Engine) Gemma 2 27B (Local Sovereign) Architectural Implication
Total Benchmark Runtime 77 minutes (~10s / task) 176.8 hours (~23m / task) 137x speed disparity for multi-turn cognitive loops
Rubric Score: Senior / Architect 98.5% (334 head-to-head wins) 32.1% (Junior or below on 67.9%) Cloud excels at high-level abstraction & holistic synthesis
Pure Code Execution Match Architect level on specs & logic Tied 50% of the time Local models match cloud on bounded, deterministic specs
Average Output Volume 961 words / response 157 words / response Local weights execute tersely without narrative padding
Cost & Data Sovereignty API token fees, external egress $0 compute cost, 100% offline Zero data liability for air-gapped enterprise pipelines

A cloud model thinks like an Architect and answers in 10 seconds. A local 27B model thinks like a Medior and answers in 23 minutes. But a Medior who executes reliably, privately, and for free is still an extraordinary asset to hire.

This is not a death sentence for local models. It is a mandate to deploy them properly. You do not assign a Medior to strategic architecture. You assign them to repeatable, well-scoped execution tasks where privacy, cost, and autonomy matter. The architecture determines the outcome, not the model.

4. What Failed: The Honest Diagnosis

Here is where most agentic engineering posts stop. They show you the wins and call it a case study. I am not going to do that.

The volume problem is real. When seven agents operate in parallel across a shared workspace, the artifact trail compounds faster than any human review cycle can absorb. Ticket notes, state pages, README adaptations, recordings, scratch pads, markdown files: the output volume from a single session was staggering. Reviewing it was not a follow-up task; it became the primary task.

I ran the same verification request five separate times, "Check this again from a different angle, just to be sure", and something new surfaced every time. That is not a failure of the agents. That is a structural gap in the review protocol. The pipeline was producing faster than I could govern it.

This surfaces a fundamental architectural truth: if you deploy AI to generate, you must also deploy AI to audit. The review cycle cannot remain a human bottleneck in a system designed for scale. We are already building the answer to this, using agents to validate what other agents produce, but it must be designed in from day one, not bolted on as an afterthought.

The side-effect trail is a liability without governance. Every agent action produces a wake. A changed README, a new artifact, a state update. Without explicit containment boundaries, what I call the Triple-Barrier Containment Protocol, that wake becomes noise. Production-grade agentic systems do not just need prompt engineering. They need architectural discipline: version control, atomic commits, and explicit rollback contracts.

5. The Mental Model That Changes How You See AI

Here is the reframe I want to leave you with. Stop comparing AI to your best senior engineer. That comparison will always disappoint you, because it is the wrong frame entirely.

Compare AI to a junior who has read every engineering book ever written, but has never shipped a product that woke someone up at 3 AM with a production incident. They are encyclopedic. They are fast. They are tireless. And they will confidently walk into a wall at full speed if you do not give them the right constraints.

Your job is not to replace the senior. Your job is to build the governance system that makes the junior dangerous in the right direction. Let them maintain the Kanban board. Let them write the PR. Let them document the state. But review it like you would review a junior's pull request, with intention and architectural judgement.

The feedback you used to give a junior to help them grow? That now becomes a Skill or a Rule injected into the agent's context. The correction loop gets codified. The system learns from the audit, not from the failure.

That is what agentic engineering actually looks like on the ground. It is not magic. It is disciplined, iterative, human-governed workflow architecture.

6. What I Would Do Differently

If I were to rebuild this experiment from scratch, three constraints would be non-negotiable:

  1. Scope before you scale. Run one agent until you can review its output in full. Then add the second. The temptation to spin up all seven simultaneously is real, resist it. The compounding effect of parallel output will outpace your ability to govern it before you have established the review rhythm.
  2. Build the audit layer before the generation layer. Design the verification agent first. Know how you will validate output before you produce it. The pipeline is only as reliable as its weakest checkpoint.
  3. Treat the artifact trail as infrastructure. Every generated file is a commit. Every commit is a decision. If your agentic system is not version-controlled, it is not production-grade. It is an experiment with no rollback.

Related Architecture Essays

Architecting Autonomous Systems?

Stop building fragile monoliths on raw conversational prompts. If you are an engineering lead, CTO, or enterprise architect structuring sovereign agentic workflows, deterministic containment, or local developer governance:

Younes Baghor
Fractional Head of AI & Enterprise Systems Architect | Founder, Sprawl.software

« Previous Back to Blog Next »