Can a team of AI agents deliver software? Our tests and their limits

Our tests covered an initial development batch. Here is what they showed, how we evaluated it and what still needed checking before real use.

A booking screen can look finished while accepting two customers for the same time slot. To understand what an agent team has actually delivered, you need to inspect the available features and try them. Our internal measurements taught us to distinguish a completed test run from software ready for use.

What an agent team does

The agents divide up the work: understand the request, write the rules, build part of the software and check the result. In Maestro, you direct that work from a requirements document. This organisation helps track decisions; it does not eliminate AI errors.

What we tested on 6 August 2026

The historical campaign for our method v2.5 covered nine scenarios: shared expenses, working hours, bookings, inventory and others. Each run went from the initial need to a first development batch, with a checking and repair phase. It did not build every feature planned for the product.

We compared those results with nine BMAD-METHOD v6.10 outputs retained from earlier tests on the same scenarios. Three AI judges assessed each anonymised output against shared rubrics. One retained run per method and scenario is not enough to measure how consistent the results are.

The results of that campaign

Our method received a higher score on eight of the nine scenarios. BMAD retained the lead on room bookings. The scores also credited design work: they do not represent a percentage of working features or a reliability score for the current software.

All nine retained runs of our method reached the end of the test, consuming $115.65 in AI usage. The BMAD outputs totalled $238.66, with three runs stopped by their spending cap. These amounts exclude human work and invalidated runs, among other costs; they are not quotes for building an application.

What the scores had missed

In a later campaign, a highly rated time-tracking application failed to compile when a judge tried to launch it. That finding reinforced our requirement to execute the software. It also means the old scores cannot be presented as proof that every feature was tested. Our account of the campaigns explains this limitation.

What you can take from this

Ask for a demonstration using your own cases: book a slot, attempt a duplicate, correct a record and retrieve it after closing the application. Have someone check the parts you cannot assess yourself. An agent team can help build software; its output still needs to prove itself in your work. Our article on measured costs also explains what the published spending covers.