The engineering bottleneck was bigger than coding time
6:03:31 ↗An engineer can write code faster and still spend days moving a feature toward deployment. David Abitbol opens Revolut’s AI story with two measures from a sample of about 2,000 engineers: coding time, measured in hours, and lead time from picking up a feature to having it merged and ready to deploy, measured in days.
Early use of Gemini helped with looking up information and reading documentation. Abitbol reports improvements in both measures. The initial rollout of Claude Code was less compelling economically: some engineers consumed substantial token budgets without a similarly clear return, so the company imposed limits.
That created a second problem. When tokens feel scarce, engineers save them for a difficult coding task instead of using them across the surrounding work. Abitbol describes self-hosting GLM models as a way to give engineers much more freedom to use AI without unpredictable bills.
He reports that the move to GLM 5.1 coincided with roughly 60% better coding time and 70% better lead time on Revolut’s chart. The internal comparison does not isolate the model change from other causes. His interpretation is that engineering delays include the inbox, Jira, Confluence, and other work around the code. Cheaper access broadens which tasks people are willing to delegate.
Tune the same GPUs before buying more
6:07:24 ↗Open models became Revolut’s default for this workload, with frontier services kept available for exceptions. Abitbol reports using 56 B200 GPUs to serve GLM 5.3 to roughly 2,000 engineers. His target is a time to first token below three seconds; once it exceeds five, users start complaining. Around 200 tokens per second is adequate for the interactive workload he describes, especially as engineers increasingly launch tasks in parallel and return when they finish.
The economics he reports are a blended cost near $0.30 per million tokens and spending around a tenth of the prior Anthropic/OpenAI invoice. Those figures describe Revolut’s deployment and utilization, not a portable price for any self-hosted model.
Simply loading weights onto GPUs did not deliver the desired performance. The team tuned its serving system against actual demand. The chart below holds the GPU count constant while showing higher request volume and lower latency.
At 6:10:16, the slide reports 76% higher requests per second and an 88% reduction in time to first token, from 14.26 seconds during migration to 1.68 seconds in the stable setup. Abitbol’s advice is to resist adding GPUs until the existing configuration has been tuned.
Context, cache, and the shape of inference
6:10:29 ↗The context window competes with concurrent requests for GPU memory. The KV cache stores attention data from tokens already processed, so the model can reuse it while generating the answer. Reserving room for the maximum advertised context may waste capacity when most coding requests are smaller. Abitbol sizes the window to the work engineers actually send.
Quantization can reduce the GPU memory footprint. He favors FP4 on the Blackwell-generation hardware in this deployment. Tensor parallelism also affects the tradeoff: fitting the weights across four GPUs does not necessarily leave enough KV-cache space for the workload, so an eight-way configuration can perform better.
Cache reuse is especially valuable when coding tasks repeatedly carry similar context. He advises checking the cache-hit rate, then considering techniques such as speculative decoding and separating prefill from decode. Prefill processes the input context; decode generates the answer token by token. Their different workloads create opportunities to tune or separate the serving stages.
The production application still needs to handle capabilities outside the main model. For a text-only coding model, Abitbol suggests routing image understanding to another model rather than sacrificing coding quality just to keep every task on one endpoint. Compatible APIs then let engineers use their preferred coding harness against the hosted models.
Keep control around the model
6:12:47 ↗Abitbol finishes with the operational case for self-hosting. Downloading weights and running them on infrastructure selected by the company is a different data path from sending inference requests to a model developer’s hosted service. The deployment choice matters more than the model’s country of origin alone.
He describes Revolut’s approach to human oversight: configure the sandbox and approval rules so restricted operations require intervention. The rules constrain what a coding agent can actually execute, rather than relying on a promise that a human will review everything later. He also discusses misuse by insiders and prompt injection as concerns for the surrounding controls.
His comments about regulatory classification and acceptable hosting locations describe his view of Revolut’s circumstances. They should not be read as a general determination that a model, region, or sandbox automatically satisfies another organization’s obligations.
The final requirement is observability. If an incident occurs, the team needs enough telemetry to inspect what happened and follow it through the system. That same data supports operational analysis during ordinary use. Lower inference costs let engineers use the model more freely. Sandbox rules limit what it can do, and telemetry lets the team investigate when something goes wrong. Self-hosting leaves Revolut responsible for all three.