DeepInfer's 155 trillion tokens a day, “Black October” and 47 canceled data centers: AI budget

In one week of October several pieces of news came out that separately look like they're about big companies, and together change how a small business should count its AI spending. DeepSeek opened its serving stack and showed that tokens can cost pennies. CNews writes about the cancellation of 47 data center projects in the US. And “Black October” was a reminder that the AI tools in your chain need updating like any other software.

What came out

On October 21, Latent Space took apart the release of DeepInfer, DeepSeek's serving stack that runs the V4.1 Flash model. The numbers from DeepSeek's report: 155 trillion tokens a day from one cluster, a 96% input cache hit rate, 230 tokens per second on generation, and a prompt cache that lives up to 72 hours against 12 hours at OpenAI. At prices of $0.15 and $0.60 per million input and output tokens, the authors came up with $38 of revenue per card per hour, a margin above 80% at a cost of $5 to $8 per card per hour, and payback in 81 days. Two caveats from the same place: these are DeepSeek's own claims without an independent audit, and KV cache management and load balancing didn't make it into the open code. The vLLM team replied that on GB200 in a comparable configuration it trails on decoding by only 6%.

On October 20, CNews, citing experts, writes that the $10 trillion AI infrastructure boom in the US has cracked: delays in data center construction, rising borrowing costs, pushback from local residents, 47 canceled projects and investor doubts about payback.

Two conversations on Latent Space cover how the compute market works. On October 22, Erik Bernhardsson, CEO of Modal, explained why a GPU contract looks like a rectangle, a fixed number of cards for a fixed term, while real load rises and falls over the day. Modal pools demand from thousands of customers: the company passed $1 billion in annual revenue and raised $750 million at a $16 billion valuation. Erik also names a risk: if GPUs get cheaper, whoever locked in capacity at today's prices is left holding an expensive reserve. On October 20, Evan Conrad of SF Compute talked about a compute exchange with physical delivery: the buyer gets working cards, and the price accounts for whether the job can be interrupted and what network it needs.

And “Black October” in the Latent Space digest of October 23: within 48 hours there was an RCE in PHP, a GitLab hole rated CVSS 10 that had been there since 2014 and was closed in 19.5.1, and a fix for an RCE in the Brotli decoder in Node.js. LiteLLM 1.107.0 turned out to contain a password stealer, Railway asked customers to rotate their secrets, and Zeabur is shutting down after a leak of 321,810 accounts. LiteLLM matters most here: it's the proxy companies use to reach models, exactly the layer where your keys live.

Why this matters to a business

Take a manufacturing company that runs incoming specifications through a model. At DeepSeek's price, six million input tokens a day cost less than a dollar. But if the proxy the requests go through hasn't been updated in six months and has a stealer in it, one incident will outweigh a year's model budget. Tokens get cheaper, while the cost of ownership is made of other things: who updates the libraries, where the keys are stored, what a human check of the result costs, and what happens if the model provider raises prices or shuts down.

The second takeaway, from the news about data centers and rectangular contracts: don't lock in capacity you don't need around the clock. A farm doesn't keep seasonal pickers on payroll all year.

How I count the cost on my projects

At the PDR company in the US, parsers read insurance estimates, reports and photos with VINs. The core is written in Python: OpenCV prepares the image, Tesseract recognizes the text, Camelot pulls tables out of PDFs, Pandas cleans the data. Models come in only at the end: AI validators check the data that has already been extracted, and I use three services, Anthropic, OpenAI and Gemini, each for its own task. We pay per document checked. In 2026 I connected these validators and shipped the release version: about 99.9% of documents go through without manual correction. The parsers free up about 60 hours of manual work a week.

Another example. In late 2023 and early 2024 I built a subtitle system for a consulting company in London: a video from a Google Drive folder or a YouTube link goes in, and an SRT file with timestamps comes out. A local Whisper model did the speech recognition, so there was no per-minute fee for the recording. The server cost money, but there was no separate charge for volume. Burning the subtitles into the video was planned, but the client stopped the project. Here the price of an accepted result, a finished subtitle file, matters more than the price of a token.

What I would do

Count the cost of one accepted result: a document that passed its check, or an answer the customer accepted. It includes tokens, the employee's time on the check and a share of the server costs. In that sum, tokens are not my main item.

Keep a list of the AI libraries and proxies in the project with their versions, and update them on a schedule. After “Black October,” check LiteLLM, pip, certifi and the system certificates, and rotate the keys if they went through Railway.

Don't tie yourself to one model provider. My validators in the parsers run on three services, each for its own task, and that leaves room to maneuver.

For tasks that run steadily and in volume, cost out a local model on your own server, as in the subtitles project. For rare and peak tasks, take an API and pay as you go.

Where AI helps parse documents and how to check the result: AI in data processing and the section AI in a process with result verification. Cases: document and photo parsers for a PDR company and subtitles on a local Whisper. Other reviews: how I choose between my own model and an API and how I read the numbers in vendor case studies; earlier I wrote that you should count accepted tasks rather than tokens alone. If you want to know what AI really costs in your process, describe the task: Discuss your task.