We Have AGI At Home

The case against open-weight models has always been that they were the cheaper, weaker option, and if you wanted real quality you paid a frontier provider. As of mid 2026 that case has mostly collapsed, with the best open-weight models sitting around 3.4 points behind the best closed one on the Artificial Analysis Intelligence Index, measured across 270 models. Epoch AI cites a roughly four-month lag.

So we find ourselves in a time where the model itself is no longer the bottleneck. In this post we will explore the idea that we don't actually need better models, and instead target the setup a small-to-mid-sized organisation needs to run the ones we already have. And by "running" we don't mean a single dev running the latest TMBBs (Trust Me Bro Benchmarks) - we mean building out a tool organisations can use to do actual work. We covered the deployment mechanics in an earlier post on baremetal AI.

The current state of open models

This landscape changes seemingly every other week, but as of the time of writing (mid September, 2026), here's the gist:

At the top, some open models can now hold their own in real agentic pipelines as frontier substitutes. DeepSeek's V4 line reached the low-80s on SWE-bench Verified under an MIT license, while GLM-5.2, Kimi K3, and Qwen's largest models trade the lead on coding and agentic benchmarks.

In the middle sit the 24-to-32-billion-parameter models such as Qwen's 27B, Gemma 4 31B, and Mistral-Small, which handle reasoning and coding well enough for most production work.

At the small end, models like gpt-oss-20b and Phi-4 can do drafting, extraction, and classification on very little hardware.

Most of the frontier-grade open models use a mixture-of-experts design with million-token context windows, so a model with hundreds of billions of total parameters activates only ten to twenty billion per token. The point is that "open" no longer means "compromise" for models doing real work. The deciding factor is where and how you run it.

The Hardware

The common assumption is that serving a capable model means buying or renting the same GPUs the labs use. But as the models get better and increasingly more compatible with commodity hardware, is it really necessary to spend $25,000 to $33,000 on a H100? Or $350,000 on an eight-GPU cluster? That is hyperscaler and AI-lab territory, and we all know hardware prices are only going up.

What small-to-mid-sized organisations can actually buy, without a procurement committee, are workstation-class cards:

So now we can map what you own to what you can run:

Serving a model is bounded by memory, not compute. You budget VRAM as the weights (parameters times bytes per parameter, set by quantisation) plus the key-value cache for in-flight requests plus a few gigabytes of runtime. Get it right and a 27-to-40-billion-parameter model at 4-bit, on one 24-to-48 GB card, delivers reasoning and coding that competes with commercial offerings. That is real work, on hardware that costs about as much as a nice laptop, or that you already own.

The real juice is in the harness

The view that base models do not need to get much better, and that most of the remaining gains live in the harness around them, has gained traction as the AI space has matured. This is backed by evidence. Pure parameter scaling is showing diminishing returns: knowledge tasks plateau beyond about 30 billion parameters, reasoning tasks around 70 billion and up, and high-quality training data is running short.

Meanwhile, the harness continues to pay off. Tool orchestration, persistent memory, environment sandboxing, and automated feedback loops now separate a good agent from a useless one, and many leading agents reach frontier performance through scaffolding rather than a bigger underlying model.

In any case, the harness dictates not only how effective the model is for the individual user, but also its usability for organisations - Orchestration, autoscaling, retrieval plumbing, access control, and audit are just as crucial as how elegantly or autonomously your agent can write code.

The jump from one card to a platform

So now you have open-weight models running on actually obtainable hardware. How do you build that up from a single dev rambling about benchmark scores into a platform an entire organisation can use for real work? Well, such a shared service needs four things.

• GPU capacity that scales with demand.
• Model weights that are versioned and delivered wherever they run.
• Endpoints that are access-controlled and audited.
• Enough utilisation to justify the hardware.

That last one determines the economics. Self-hosting beats a frontier API only once the GPUs are busy, on the order of tens of millions of tokens a day at high utilisation. Keeping expensive silicon fed means autoscaling and sharing one fleet across teams.

Each one of the aforementioned requirements already has a solution in the AWS toolkit. The only catch is that using it means running on AWS. But you just went to all that effort ditching the cloud-served frontier models in favour of the open ones, and buying your own commodity hardware to run them, only to realise you still need the cloud!?

Until now, this was the "rock" to the proverbial "hard place" of the increasingly concerning token economics of the frontier labs. Luckily, there's a new kid on the block that can help with this.

Spinifex implements the aforementioned AWS toolkit on hardware you own, so the platform you would have built in the cloud maps onto your own GPUs with tooling your team already knows. EC2 provisions and scales the GPU instances. EKS schedules the vLLM or SGLang pods and scales replicas with demand, which is what keeps utilisation and the economics where they need to be. S3, backed by Spinifex's erasure-coded Predastore, becomes the model registry that versions and distributes weights. IAM and STS gate which teams reach which endpoints, with an audit trail. ELBv2 spreads load across replicas.

So now you get the same Terraform and the same AWS CLI, with one provider override pointing at your endpoint. We walked through a concrete build, an EKS cluster on a Supermicro X14 with two RTX Pro 6000 GPUs wired entirely in Terraform, in the earlier baremetal AI post and the reference architectures in our docs. Unlike a premium integrated rack, none of it requires special hardware, and unlike a hyperscaler cluster, none of it requires a cluster. You start on the commodity box you own and grow the fleet.

Is it worth it against a frontier API?

Often, but not always. But the direction of travel is clear: by some counts more than half of enterprise inference now runs on-premise or at the edge, up from about an eighth in 2023. One representative scenario puts break-even at roughly four months at 30 million tokens a day.

So it becomes a question of how much commodity compute is obtainable for your organisation, what kind of work needs to be done and at what rate. Spinifex provides the tools to help make the economics more viable by replicating what you'd usually use AWS for. But for those with the right set of circumstances, a fully functional open weight model, running on on-prem, obtainable hardware, with a harness that makes the platform effective at both the individual and organisational level, is actually within reach.

What could that look like?

• Retrieval over private corpora. Point SGLang at your legal, financial, or internal knowledge base and let prefix caching absorb the repeated system prompt. The documents never leave your control.
• High-volume batch inference. Classification, extraction, and summarisation pipelines run the GPUs hard and steady, which is exactly where owned hardware wins on cost.
• Agents on internal systems. With open models clearing the low-80s on SWE-bench, agents that read and act on private codebases are viable with no token bill and no data-egress review.
• Fine-tuning on data that cannot leave. Train and serve on the same fleet, with the tuned weights landing in self-hosted S3 next to the base models.
• Air-gapped and sovereign inference. The model never phones home and the stack runs disconnected, which is the requirement for defence, healthcare, and finance operators who cannot use a public API at all.

Get started

Our reference architectures cover deploying GPU-backed EKS clusters and AI workloads on your own hardware, including the air-gapped install path. The source is on GitHub, written in Go and AGPL-3.0 licensed. Or sign up for the free sandbox to explore the AWS-compatible API surface before you commit hardware.