# Shipping like a startup, regulated like a bank

David Abitbol · AI Engineer NYC 2026 · October 13

[Watch recording](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21811s)

[06:03:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21811s) Hello everyone, welcome.

[06:03:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21815s) My name is David.

[06:03:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21817s) I lead AI operations at a company called Revolut.

[06:03:42](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21822s) We have 15 minutes today together.

[06:03:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21825s) we'll talk about high performance engineering and self-hosting open source model on dedicated GPU.

[06:03:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21832s) How does that sound?

[06:03:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21834s) Cool.

[06:03:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21835s) So it's going to be a technical one, so bear with me.

[06:03:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21839s) So maybe to start off, because I know this is the US and you may not know Revolut,

[06:04:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21844s) quick words about our company.

[06:04:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21847s) We're a digital bank in Europe.

[06:04:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21849s) We have 80 million customers.

[06:04:11](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21851s) We're live in 45 markets.

[06:04:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21854s) with the most downloaded financial app in Europe for the last year.

[06:04:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21859s) And as of July, we're Europe's largest private company.

[06:04:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21865s) So we're doing okay.

[06:04:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21869s) The emphasis of Revolut is shipping a lot of product,

[06:04:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21872s) and so there's a high focus on engineering culture and excellence on that demand.

[06:04:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21879s) And so we'll talk about that.

[06:04:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21881s) So let's talk about some AI.

[06:04:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21885s) We have a chart here.

[06:04:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21887s) I'm going to see two lines.

[06:04:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21888s) This is a sample of 2,000 engineers' performance over the last year.

[06:04:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21894s) So the green line is going to be lead time.

[06:04:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21896s) So it's going to be when an engineer picks up a feature

[06:04:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21899s) until a feature is merged and ready to be deployed.

[06:05:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21904s) The blue line is going to be coding time.

[06:05:06](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21906s) So the green line is in days.

[06:05:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21907s) The blue line is in hours.

[06:05:11](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21911s) Just to reset the timeline in case people forget,

[06:05:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21913s) because AI moves really fast.

[06:05:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21916s) Claude Code came out around February 2025, right?

[06:05:20](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21920s) And that's the Claude Code, Sonnet 3.7,

[06:05:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21922s) the one that would delete your prod database, right?

[06:05:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21925s) Not the safe cloud and codex that we have today.

[06:05:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21928s) So bear in mind when we talk about adoption.

[06:05:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21932s) So what you see on this chart is January 25,

[06:05:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21935s) first introduced AI in the company, right?

[06:05:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21938s) Gemini.

[06:05:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21939s) First time people were able to use private data within AI.

[06:05:44](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21944s) Around February, we started playing around with Claude Code, and then January is when

[06:05:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21948s) we first launched Claude Code at scale.

[06:05:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21952s) What you see is Gemini had 10% improvement in lead time, 20% improvement in coding time,

[06:05:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21958s) so that's people no longer need to look at Stack Overflow and reading documentation.

[06:06:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21963s) But then Claude Code, not so much impact at the time.

[06:06:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21968s) So when we launched it, we were highly skeptical about the cost of AI.

[06:06:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21972s) we were seeing engineers spend a thousand a month,

[06:06:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21975s) 10,000 a month with not so much ROI.

[06:06:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21978s) And so when we launched it,

[06:06:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21979s) we launched it with strict budget limits.

[06:06:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21983s) And so we're looking at the next thing, right?

[06:06:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21984s) So how can we provide limited tokens to our engineers

[06:06:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21987s) so they can use AI guild-free for a fraction of the cost?

[06:06:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21991s) And so we looked into open source.

[06:06:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21994s) So around April 26 is when we started

[06:06:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21997s) self-hosting open source model.

[06:06:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=21999s) We're a big fan of GLM for coding.

[06:06:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22001s) so around that time what you see

[06:06:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22003s) April 26th, GLM 5.1

[06:06:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22007s) we see 60% improvement

[06:06:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22008s) in coding time

[06:06:50](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22010s) 70% improvement in lead time

[06:06:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22013s) so traumatic impact

[06:06:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22016s) what do we learn from this?

[06:06:57](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22017s) well there's more impact on the lead time

[06:06:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22019s) than on the coding time

[06:06:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22019s) so what we find is the bottleneck for engineering

[06:07:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22022s) is not necessarily coding

[06:07:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22025s) you can save a lot of time

[06:07:06](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22026s) not having to dig through your inbox

[06:07:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22028s) or messing with Jira and Confluence

[06:07:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22030s) and so when you have a limited token you're free to do that. You no longer have to save

[06:07:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22036s) your tokens for the end of the quarter or the most difficult task. So where do we stand?

[06:07:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22044s) Well open source is now a priority, a revolute, and your economics are favorable, the billing

[06:07:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22050s) is flat, we don't have any bill shock any of the month, and the frontier is the exception.

[06:07:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22055s) So obviously we keep using codex, code code.

[06:07:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22058s) The latest model are fantastic.

[06:07:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22060s) Jimmy and I are going very good as well.

[06:07:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22063s) But we keep them as exception.

[06:07:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22067s) So how do we get there?

[06:07:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22071s) Some metrics that matter when you're self-hosting model.

[06:07:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22074s) So to serve 2,000 engineers, we roughly need 56 B200.

[06:08:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22080s) And that's to host GLM 5.3.

[06:08:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22082s) GLM is a big model, right?

[06:08:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22084s) 8.500 billion parameters.

[06:08:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22087s) What you want to aim for is

[06:08:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22089s) sub-3 second time to first token.

[06:08:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22093s) This is empirical evidence.

[06:08:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22095s) As soon as you exceed 5 seconds,

[06:08:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22098s) my Slack inbox starts to blow up,

[06:08:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22099s) and so we know that's the limit.

[06:08:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22102s) In terms of throughput,

[06:08:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22103s) it doesn't really matter as much.

[06:08:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22104s) We serve, in terms of interactivity,

[06:08:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22107s) 200 tokens per second.

[06:08:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22109s) People are fine.

[06:08:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22109s) You don't need to go higher.

[06:08:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22111s) The way people work has changed.

[06:08:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22113s) which we no longer look at the stream of thinking tokens.

[06:08:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22116s) People run tasks in parallel

[06:08:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22117s) and check in when the task is done.

[06:08:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22120s) But if you're in a high throughput

[06:08:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22121s) and you're an engineer at Revolut,

[06:08:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22123s) you just connect after 5 p.m. when one is on GPU

[06:08:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22126s) you can get 800 TPS and then, you know, it's a dream.

[06:08:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22131s) In terms of cost, in terms of impact,

[06:08:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22133s) self-hosting, GLM, or blended cost of token

[06:08:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22136s) is 30 cents per million.

[06:08:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22138s) And so our invoice is one-tenth

[06:09:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22140s) of what we were spending on Anthropic and OpenAI.

[06:09:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22148s) So there's a lot of tuning

[06:09:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22150s) that goes into running open source models.

[06:09:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22154s) If you just get GPUs and upload weights on a GPU

[06:09:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22159s) and you start doing some inference,

[06:09:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22162s) you're going to get terrible performance.

[06:09:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22164s) That's because your workload is going to be very dependent

[06:09:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22168s) on the parameters that you set.

[06:09:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22170s) So if you look all the way to the left of that chart, the blue line is going to be demand,

[06:09:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22174s) so that's request per second.

[06:09:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22175s) That's going to be what's going to be the bottleneck for your latency for your GPUs.

[06:09:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22180s) And the red line is what we're optimizing for, that's the time to first token.

[06:09:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22185s) So when we started back in April, you could see request per second around 3,

[06:09:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22191s) which is high but not so high, and time to first token over 10 seconds, which is very slow.

[06:09:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22198s) All of the data that you see is

[06:10:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22200s) the same number of GPU. We did an increase GPU count with LLF56.

[06:10:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22205s) And by the end, you see in August, requests per second were over 7, and time to first occur, and we're around 1.6.

[06:10:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22212s) So we've had 88% improvement in performance for 76% increased demand.

[06:10:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22219s) So the learning from this is there's a lot of parameters that you can improve on, and you really need to resist scaling.

[06:10:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22225s) if you're looking to self-host a model.

[06:10:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22228s) So let's talk parameters.

[06:10:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22231s) There's no secret recipe,

[06:10:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22233s) but the things you consider are the following.

[06:10:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22237s) First one, long context is a scam.

[06:10:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22240s) You really need to be mindful that

[06:10:42](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22242s) even though you can serve one million token,

[06:10:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22245s) 256 is really what you need.

[06:10:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22249s) Quantization, if you're running Blackwell generation GPU,

[06:10:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22252s) you don't want to run anything higher than FP4.

[06:10:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22255s) And the FP4 is best, roughly going from FP4 to FP8

[06:11:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22260s) is a 40% saving in GPU footprint for zero loss in precision.

[06:11:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22268s) Other parameters to consider, tensor parallelism,

[06:11:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22270s) even though GLM fits on TP4, you don't have enough KV

[06:11:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22274s) cache, and so maybe a TP8 works better,

[06:11:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22276s) depending on how large your workload is.

[06:11:20](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22280s) and then on the more esoteric

[06:11:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22282s) check your cache hit

[06:11:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22283s) coding test needs to be above 90%

[06:11:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22285s) cache hit

[06:11:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22287s) and you can look into optimize

[06:11:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22290s) spec decoding

[06:11:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22291s) sequential pre-fill, PD disaggregation

[06:11:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22293s) all of these

[06:11:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22296s) cool

[06:11:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22297s) so now let's talk about production setup

[06:11:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22301s) GLM obviously

[06:11:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22301s) is a text only model

[06:11:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22303s) which is a limitation for some people

[06:11:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22306s) the trade off I would say

[06:11:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22307s) we would make every time is get the smarter model and figure out how to do

[06:11:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22311s) image recognition another way. You can easily have a local router, open codex is

[06:11:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22316s) one, where you would route any image recognition to Gemini or Luna and keep

[06:12:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22323s) GLM for the main task. If you really need some vision model, GLM 5G flash works

[06:12:11](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22331s) really well. We have our UI designer making great use of it. Turnaround for UI product has gone a

[06:12:20](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22340s) really, really long way. And then the last one is going to be about the harness. So all of these

[06:12:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22349s) endpoints are going to be OpenAI compatible API with response API. And so you can use them

[06:12:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22355s) inside OpenCode Codex, Codex app as well, works very well.

[06:12:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22360s) And so that's what we recommend.

[06:12:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22366s) And then last message, this one is for the CIO in the room.

[06:12:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22372s) There's been a lot of stigma in using Chinese open source model.

[06:12:57](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22377s) So the thing to consider is you're not using

[06:13:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22381s) inference, dedicated endpoint that goes to China.

[06:13:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22383s) you're just getting the weights and putting them on a US-based service.

[06:13:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22388s) So for data sharing side, this is totally fine.

[06:13:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22393s) Revolut in particular, because we're an EU-regulated bank,

[06:13:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22398s) we fall under the most strict regime in terms of compliance.

[06:13:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22403s) so what we find that works is all of these coding tasks are not considered autonomous agents.

[06:13:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22410s) They're always with a human in the loop.

[06:13:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22412s) And so how do you give reassurance

[06:13:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22414s) that you have a human loop?

[06:13:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22416s) You start by creating a sandbox that guarantees that.

[06:13:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22420s) And the sandbox means now if you've used Codex Claude Code,

[06:13:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22423s) it has all of these features of approve for me.

[06:13:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22426s) You just have to set the rules for the operations

[06:13:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22429s) that you don't want to allow.

[06:13:50](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22430s) So you can allow grep but not gradle, for instance.

[06:13:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22433s) And that's gonna give confidence to your regulator

[06:13:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22436s) that it's not an autonomous agent

[06:13:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22438s) that needs different supervision.

[06:14:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22442s) This sandbox also protects you from rogue use.

[06:14:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22445s) You may have bad actors inside your company, and you want to shield for that.

[06:14:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22449s) You'd be surprised what you find in a large company.

[06:14:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22453s) And then the last one is prompt injection.

[06:14:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22455s) Not so much a thing now, given how smart models are, but always a thing.

[06:14:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22461s) In terms of data privacy, we spoke about it.

[06:14:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22464s) As long as the data sits in one of the countries that are in the data equivalency framework,

[06:14:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22469s) so US, EU, UK, Israel, you're fine, you can get your GPUs there. All of your inference provider

[06:14:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22476s) or GPU provider are going to have zero data retention by default. And then the last thing

[06:14:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22483s) you need is a very strong open telemetry so that if you ever have an incident, you need to be able

[06:14:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22491s) to go deep and look at the logs.

[06:14:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22494s) Also gives you some analytics.

[06:15:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22500s) All right, thanks everyone.

[06:15:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=22501s) That was all for my talk.