Eighteen assumptions we got wrong
Every one of these was something we believed strongly enough to plan around. This is what each cost when it turned out to be false.
We keep a list of assumptions a project depends on, and how each one could be shown wrong. The point of writing them down is that some of them get crossed out.
In a field moving this fast, the speed at which a team can falsify its own assumptions is what protects its engineering budget. Every assumption below was found to be false in days rather than quarters — which is the difference between a corrected plan and a wasted one.
Below is the full list from one product investigation in August 2026 — eighteen assumptions, all falsified or amended, in the order the damage mattered. Two of them killed a business model. Several saved months. One existed only because a voice transcript misheard a word.
The entries follow the same shape throughout: what we believed, what overturned it, what it cost.
"There is no GUI for this, so that is our opening"
the agent harness we had chosen shipped without a graphical interface, and building a good one was our way into the market.
more than forty free, MIT-licensed GUIs already existed. Four had over five hundred stars. One shipped a signed and notarised macOS installer and a Homebrew formula — further along in distribution than our plan's first year.
the core commercial assumption of the product. Not a feature, the reason for the product. We repositioned to selling what sits above the interface layer rather than the interface itself.
"The teacher edition is the obvious first product, because that is where the market gap is"
the largest underserved segment was the best place to start.
a gap in a market is not the same thing as somebody willing to pay. In education the budget sits with students and parents. Teacher-side purchasing runs through institutions, takes a long time, and lands at a low unit price — the gap is a gap partly because it is hard to sell into.
the flagship product changed audience. The first version now targets parents, and the teacher edition moved to a much later stage.
"These two projects can be combined"
two open-source agent projects could be integrated, taking the best parts of each.
both are complete harnesses, not components. They make the same decisions in incompatible ways. The official integration pull request had already been opened and closed. A second, related belief fell at the same time: we thought one of them had no desktop application, when it had shipped one several versions earlier.
we stopped treating one as a part and started treating it as a competitor. We picked one base and borrowed design ideas from the other instead of code.
"Letting users sign in with their existing AI subscription is a good selling point"
users could authenticate with a subscription they already pay for, and that this would be a differentiator.
it depends entirely on the vendor, and we had generalised from one case. One provider's terms explicitly prohibit it. Another permits it, with paid commercial products already doing so. A third offers no honest route at all. Our follow-on assumption — that no paying commercial product did this anywhere — turned out to be true of exactly one vendor and false of the rest.
the authentication matrix was rewritten per vendor. One integration moved from "backlog" to "viable for version one" as a result, so this one cut both ways.
"We know which model versions are current"
we had the current version numbers for two open-weight model families.
both were behind — in one case by two releases. Worse, one "upcoming version" we had noted was not real. It traced back to content-farm articles that existed to rank in search results, not to any announcement.
little directly, but it changed how we source facts. Model release information now comes from vendor documentation or repositories, never from a search summary. Search results about fast-moving AI releases are now the single least trusted input we have.
"This model provider is the cheap option"
one provider was cheap enough to build a cost argument around.
prices rose between three and roughly five times, announced mid-investigation on 2026-08-16. Separately, its tool-calling proved unreliable, which matters more than price for agent work. A related belief — that its off-peak discount window suited our timezone — survived, but only just: the discount window overlaps the Taiwan working day, and it only helps because the intended users are working in the evening.
we recommended excluding the provider. The near-miss was writing marketing copy about off-peak pricing that would have been wrong for anyone using the product during the day.
"Open weights means we can run it locally"
a mixture-of-experts architecture would let us run larger models on modest hardware, since only some experts activate per token.
all expert weights must be resident in memory. What is sparse is the computation, not the storage. The memory requirement is the full model.
this one is worth stating as a rule, because it is a common misreading: open weights does not mean runs locally. Our local-hardware plans were sized against a number that was never achievable.
"We can fork a large skill collection wholesale"
the big aggregated skill libraries could be adopted as a base and built on.
licensing. One large collection declares no licence at repository level at all, which means no permission is granted rather than permission being broad. Another mixes material under terms that do not permit what we intended.
any adoption now requires a folder-by-folder licence review. Wholesale reuse is off the table.
"The document-generation skills are permissively licensed"
a well-known vendor's published skills were available under a permissive open-source licence.
the repository declares no licence, and the document-format skills specifically — word processing, PDF, presentations, spreadsheets — are proprietary, with derivative works not permitted.
document generation had to be written from scratch rather than adapted. That is real work we had not budgeted, and it is now one of the things we build ourselves.
"There are six thousand skills available to draw on"
published skill counts described a usable supply.
the counts are inflated several times over. Directories duplicate, re-list, and pad. In one collection of several hundred entries, the substantive content amounted to a small fraction of the headline figure.
nothing yet, but it set a policy. We do not count skills. We evaluate specific ones, and a directory's install count is not evidence about any of them.
"Existing skills are low quality, so we will rewrite everything"
published skills were not good enough to build on.
partly wrong, and wrong in a useful direction. The domain knowledge in many of them is written by people who genuinely know the field. What is missing is the engineering — scripts, templates, defined steps, checks.
this one changed our model of the business rather than costing us. Somebody else has already done the knowledge half. The engineering half is the part we are good at, and pairing with people who hold the domain knowledge is a better shape than replacing them.
"We will have to build the sandbox ourselves"
the harness had no sandboxing, so OS-level isolation was ours to build.
the project's own repository ships a sandbox extension that enforces filesystem and network restrictions at the operating-system level, using the platform's native primitives. The README says the tool has no built-in permission system, which we read as a description of what was possible. It is a disclaimer, not a capability statement — the examples directory had what we needed.
nothing, because we found it in time. It unblocked the base-platform decision. The lesson is narrower and more annoying than the others: we read the README and stopped. The answer was one directory further in.
"The hardware we are targeting has that much memory"
a specific machine was available in the memory configurations our local-model plans required.
those configurations had been discontinued during a memory shortage. The models actually on sale topped out lower.
the supported-hardware matrix. Combined with entry 07 — where the memory requirement turned out to be higher than we thought — the two errors compounded in the same direction: we had planned for more memory than exists, to run models that need more memory than we assumed.
"Python is the right place to start"
the intermediate layer should be written in Python.
nobody had ever said Python. The assumption entered our notes through a voice transcript that misheard the name of the harness itself — a two-letter word — as "Python". It was written down, carried forward, and planned around for as long as it took someone to check it against the original.
almost nothing, and that is the only reason it is funny. The layer is TypeScript. We keep this entry on the list because it is the cleanest argument for the practice: an assumption nobody had actually made survived in our plan until it was written down explicitly enough to be checked.
Fourteen entries from eighteen assumptions; the merged ones were halves of the same finding.
Two of these would have been caught by anyone who looked. Several took real work to find. One had already been answered inside a repository we had open.