Kimi K3.1 Pro promises open weights: how I split tasks between my own model and a cloud API

Moonshot released Kimi K3.1 Pro and promises to open the weights by November 4. Independent benchmarks put it next to the closed first-tier models. The same week, n8n published a table of which open models fit into which GPU, Vercel added a $30 subscription for open models, and Ollama learned to run small decision models. For a business owner the question is the same as before: what to compute in-house, what to send to an API, and by what rule to split them.

What came out

On October 23, Simon Willison wrote about Kimi K3.1 Pro: the model is available through an API, with the weights promised by November 4, 2026. The price is $2 per million input tokens and $10 per million output tokens, the same as GPT-6.1 Sol and half the price of Claude Opus 5.5. The Highspeed version is 2 to 3 times faster at double the price. On the Artificial Analysis index the model scored 55 against 58 for Claude Opus 5.5.

On October 24, Latent Space collected the details and the objections in its roundup. The model has 2.8 trillion parameters, 104 billion of them active, and a context of one million tokens. And right there, the arithmetic: 2.8 trillion parameters is on the order of 5.6 TB in 16-bit form or 1.4 TB in 4-bit. The authors say it plainly: being able to run it yourself and running it on an ordinary computer are two different properties. The license and the small Flash version haven't been announced yet.

On October 23, Vercel added both versions of Kimi K3.1 Pro to AI Gateway: provider prices without markup, a budget per key, and provider failover. A week earlier, on October 15, the same Vercel launched Token Pass: for $30 a month a team gets $80 of credit for GLM 5.3 and Kimi K3, the unused balance expires, and when the credit runs out requests stop with no extra charges. For a small team that's a clear ceiling on spending.

On October 22, n8n posted a breakdown of self-hosted models versus cloud APIs. The most useful part is the table: Qwen3 8B in about 8 GB of VRAM at 8-bit, Mistral Small 3.1 at 24B in 12 to 16 GB with 4-bit quantization and 44 to 55 GB without it, Llama 4 Scout at 109B from 55 to 80 GB. The advice I'd back: first run the task through open models over an API, find the smallest one that handles it reliably, and host only that one. For batch processing they suggest renting a GPU server only for the duration of the job.

And the Ollama v0.40.11 release from October 25, release notes: speech generation through an OpenAI-compatible endpoint arrived on macOS, and the d3 family brought decision models from vLLM Semantic Router at 2B, 4B, 9B and 27B: give it text or an image, get yes or no, an option from a list, or a score with a probability. It's the local counterpart of what OpenAI sells as the Decisions API.

Why this matters to a business

Take a manufacturer where the product structure lives in Excel bills of materials, with contractor prices and engineering drawings alongside. Nobody wants to send those files to an outside service, and the volume is steady: the same document types every day. That's the case for a small model on your own server: sort, pull out item numbers, flag what's missing. A rare hard task, a nonstandard estimate from a new supplier, is cheaper to hand to a cloud model: it happens once a week.

The rule from this example is simple. A steady volume and sensitive data pull a task toward your own server. Rare, hard tasks where accuracy on a single document matters pull it toward the cloud. The rest is arithmetic: requests per month against the hardware and the person who maintains it.

How it works for me

In the document parsers for a PDR company in the US, I use three cloud services, Anthropic, OpenAI and Gemini, each for its own task. Accuracy on every estimate matters there: documents can be 60 to 70 pages long, and a mistake in one line costs money. Since I connected the validators, about 99.9% of documents go through without manual correction, and the employee only does the final check.

Speech, on the other hand, I ran locally. In late 2023 and early 2024, I built a subtitle system for a consulting company in London on a local Whisper model: video from Google Drive, podcasts, YouTube, with timestamped SRT files on the way out. There was nothing to pay per minute of recording.

My studio's server is shared, with 2 GB of memory; it runs the website and the “Coffee Together” service in Docker. No model will fit there, and I don't pretend one will. I haven't run Kimi K3.1 Pro: by the size from the Latent Space roundup it needs a data center, and a small business server won't carry it. I'd look at it through an API, as one more provider next to the others.

What I would do

I'd sort the tasks into three piles: a steady stream with sensitive data, rare hard cases, and everything where a mistake costs nothing.

For the first pile I'd take the smallest model from the n8n table that reliably passes my real examples. I'd test on 50 to 100 real documents, not on demos.

For the second pile I'd put up a gateway like AI Gateway or my own proxy with a budget per key. Then switching models is one line in a config.

I'd calculate the break-even point for a year: renting a GPU or buying a card plus maintenance hours against the token bill. While the volume is small, the cloud is cheaper, and that's fine.

Related reading

On verifying model output: the article AI in data processing: extraction and verification and the page AI in a process where the result can be verified. Cases: document parsers for a PDR company and SRT subtitles on a local Whisper. From the digests on the same topic: why I build on free tools, speech models on your own server, what's changing in a small business AI budget and a model on the device and a fleet of sites. Earlier: the cost of AI per accepted task and knowledge graph or vector RAG. If you're deciding which part of your processing to keep in-house, tell me about the volume and document types. Discuss your task.