# Beyond the Model Card: The Jagged Frontier of AI in Finance

Rayan Krishnan · AI Engineer NYC 2026 · October 13

[Watch recording](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20015s)

[05:33:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20015s) Hi everyone, I'm Rayan, I'm the CEO and one of the co-founders of Vals AI.

[05:33:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20019s) I'm going to talk a little bit about going from beyond the model card, how evaluations

[05:33:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20023s) can be used to measure this jagged frontier, a special focus on finance for this audience

[05:33:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20029s) as well. Yeah, so first off, talking about model cards themselves, we've built some of the most

[05:33:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20034s) important benchmarks over the last couple of years that the major foundation model labs have included

[05:33:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20038s) in their model releases. So people, I think, are familiar with artifacts like this. Last week,

[05:34:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20043s) we have Sundar talking about our VALs index on stage. And then even just yesterday, Muse models

[05:34:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20048s) was citing our finance agent benchmark. You know, it keeps our lives interesting to have to keep

[05:34:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20053s) re-updating these slides as new models come out. I had to double check level numbers. You can get

[05:34:17](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20057s) sense of how capable a model is for valuable work, but that's actually driving value for

[05:34:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20063s) a lot of companies.

[05:34:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20064s) I think two interesting stats that jump out at me, the groups that are able to actually

[05:34:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20068s) track AI cost and spend are much, much more likely to have inferior performance.

[05:34:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20073s) We also have refining the X-axis, again, it's not immediately obvious which is number one

[05:34:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20076s) or what will be consistently two or three.

[05:34:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20078s) In the X-axis, you've got cost.

[05:34:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20080s) In particular, it's cost per task, not just tokened Excel models.

[05:34:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20087s) that are core to results may be unintuitive.

[05:34:50](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20090s) When we actually build benchmarks,

[05:34:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20092s) one of the most important things that we need to look at

[05:34:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20093s) is the metric we use to report accuracy.

[05:34:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20096s) And so you have a different takeaway

[05:34:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20098s) if you pick a different metric.

[05:34:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20099s) A lot of benchmarks deal with partial credit,

[05:35:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20101s) so what this is answering is,

[05:35:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20103s) in expectation, if I were to give a question

[05:35:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20105s) or a task to a model,

[05:35:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20107s) what percentage of the points would I expect to see satisfied

[05:35:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20109s) by the work output of that model,

[05:35:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20113s) especially compared to humans?

[05:35:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20115s) And then the fully correct answers metric

[05:35:17](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20117s) is more looking at, of the percentage of questions

[05:35:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20119s) that I give to models, what percentage of them

[05:35:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20121s) will come back perfect?

[05:35:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20122s) And that's a very different metric.

[05:35:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20124s) It may mean that a model is very good at producing

[05:35:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20127s) an incremental response that you can work with

[05:35:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20130s) to build a better response,

[05:35:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20131s) but if you were to pass this directly to a client,

[05:35:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20134s) it would be a coin toss of whether it was

[05:35:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20136s) meeting all the criteria that you have.

[05:35:38](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20138s) And so picking the right metric is very important

[05:35:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20140s) to accurately representing the performance of the model

[05:35:42](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20142s) and actually forming a good intuition of what it can do.

[05:35:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20146s) There's another three components that we think a lot about.

[05:35:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20149s) So validity, are you actually evaluating what's going on in the real world?

[05:35:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20152s) That means are the questions or tasks reflective of actual work going on or are they contrived?

[05:35:57](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20157s) And then are the rubrics actually describing what we're looking for or the standard of

[05:36:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20160s) excellence we want to see?

[05:36:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20162s) Reliability, this is usually a function of the number of tasks or the sample count.

[05:36:06](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20166s) A lot of companies, they have maybe five or six samples and they're kind of vibe checking

[05:36:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20170s) new models or products as they get access.

[05:36:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20172s) But it's actually really important to have a rigorous approach and have enough tasks and samples to make sure your repeated runs actually show consistent results.

[05:36:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20181s) And then last is discriminability.

[05:36:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20182s) So we don't think it's that valuable to have benchmarks where all models score 100% or all models score 0%.

[05:36:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20187s) Ideally, we can actually discriminate between models and see what the frontier looks like.

[05:36:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20192s) In particular, we also want to make sure we're building benchmarks forward-looking.

[05:36:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20195s) So what are the new set of tasks that are even more ambitious than what models can do today

[05:36:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20200s) to make sure that there's room at the frontier for us to continue to see model progress as they get better?

[05:36:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20207s) Okay, and then some of the high-level problems that we've seen and I think others can and do run into.

[05:36:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20214s) The first is that agents solve the test, not the problem.

[05:36:56](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20216s) So this is talked a lot about as reward hacking.

[05:36:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20219s) And we've seen a lot of instances where we'll have a sandbox and models will be run with our model proxy.

[05:37:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20224s) and sometimes they may try and actually call other models

[05:37:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20228s) within that proxy to escape the sandbox

[05:37:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20230s) or make use of the internet.

[05:37:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20232s) So it's really important to build the containers

[05:37:15](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20235s) in a way that you're actually testing for what matters.

[05:37:17](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20237s) Can they do the task at hand

[05:37:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20238s) or are they actually leveraging other tools

[05:37:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20241s) or maybe even other models to get there?

[05:37:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20243s) We've also seen this issue in finance in particular

[05:37:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20245s) where a lot of the times evals are looking at a set of rubrics

[05:37:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20249s) or items that we want to see included in the response

[05:37:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20251s) And so models can hedge and give three possible answers, which has a higher chance of satisfying

[05:37:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20257s) all the criteria, but in practice that's way too much.

[05:37:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20261s) A person does not want to consider three different answers, they just want to get the first answer

[05:37:45](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20265s) and have it be right.

[05:37:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20266s) And so the goal here is to try and build evals that are reflective of the actual work people

[05:37:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20271s) are doing and the work output they want to see, not to build these contrived evals that

[05:37:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20275s) the models can actually gain.

[05:37:57](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20277s) The second is, I think a lot of people are starting to see

[05:38:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20280s) that the cost of software is coming down,

[05:38:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20282s) and it's actually very, very easy to produce code.

[05:38:06](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20286s) But unfortunately, we still see in practice

[05:38:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20288s) that running evals is incredibly challenging,

[05:38:10](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20290s) especially from an infrastructure side of things.

[05:38:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20293s) And in particular, as we get to longer-running agents,

[05:38:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20296s) it means that it's really important to have infrastructure

[05:38:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20298s) that can stay consistent over a long period of time,

[05:38:20](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20300s) and then, if interrupted, can be resumed and respun back up.

[05:38:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20303s) We have, I think, just recently crossed six trillion tokens just in evaluation.

[05:38:28](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20308s) So we've run a considerable set of evals and we run everything in as high-degree parallel as possible.

[05:38:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20315s) So I think this has really been load tested, but I think other people are going to run into this even on a much smaller scale.

[05:38:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20321s) And the last is that I think a lot of the existing benchmarks, they deal with a single prompt, single response,

[05:38:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20327s) but in practice everything is multi-turn.

[05:38:49](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20329s) You have initial spec which you think covers everything, but then you get questions back

[05:38:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20333s) or you have additional feedback once something is built, and then it takes more of this iteration

[05:38:57](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20337s) to actually drive to the end output.

[05:38:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20339s) And so I think benchmarks and evals need to actually cover this multi-turn component.

[05:39:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20343s) That's not going to go away.

[05:39:05](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20345s) So when we look at a case study in particular, when we look at our team, we did a little

[05:39:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20349s) token maxing experiment where we got free access to some tools and just gave it to all

[05:39:13](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20353s) the engineers, and they really made use of these unlimited tokens.

[05:39:18](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20358s) and if we actually look back for that month,

[05:39:20](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20360s) we spent $1.5 million worth of tokens.

[05:39:23](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20363s) This was like a team of I think 10 or 12.

[05:39:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20365s) So it was roughly 10 times the amount

[05:39:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20367s) we were spending on salary for tokens in that month.

[05:39:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20369s) And so that was pretty insane,

[05:39:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20371s) but we saw great output.

[05:39:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20372s) And so our goal was to try and maintain

[05:39:34](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20374s) the same level of output across all these engineers,

[05:39:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20376s) but massively cut down on cost.

[05:39:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20379s) And really our solution to do that

[05:39:40](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20380s) was of course to build an eval.

[05:39:42](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20382s) We built a bunch of tools internally

[05:39:44](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20384s) and then have now exposed this

[05:39:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20386s) as a product called Vals Smith.

[05:39:47](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20387s) This is what we are doing in coding and I think it is a sign of what is going to come

[05:39:51](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20391s) in other parts of knowledge work.

[05:39:53](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20393s) So we were able to look at the GitHub repo as a centralized way of seeing what work has

[05:39:59](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20399s) already been done

[05:40:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20400s) and what work is continuing to happen.

[05:40:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20402s) In particular, GitHub repos are comprised of PRs, which have a GitHub issue and a resolution.

[05:40:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20407s) So what we can do is unwind the clock and give the same tasks to agents to see if they could have done the work of humans.

[05:40:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20414s) In doing that, we found a much clearer understanding of which models were overkill, way too expensive and did not need to be used,

[05:40:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20421s) and then which ones were actually capable of getting the task done.

[05:40:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20424s) And I think, again, this is another place where the intuitions

[05:40:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20427s) that a lot of people have may not be what's actually borne out in practice.

[05:40:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20430s) So I think you may see we're using a lot of closed-source models.

[05:40:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20433s) We should try and cut costs and use our own self-hosted open-source model.

[05:40:36](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20436s) You can see the comparison here we have between GPT-5.6 Sol and Kimi K3,

[05:40:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20441s) and we see that 5.6 Sol is better.

[05:40:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20443s) It's going 2% better on this internal eval.

[05:40:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20446s) It's also considerably cheaper than hosting it

[05:40:48](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20448s) through a very inference-optimized stack.

[05:40:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20452s) So it's $2.50 instead of $11, and it's faster half the time.

[05:40:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20455s) And so there's no real reason why this kind of intuitive sense

[05:40:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20458s) of using open source would be a legitimate way

[05:41:00](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20460s) to drive down costs.

[05:41:02](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20462s) And so this has resulted in two main differences.

[05:41:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20464s) One, we can actually figure out which tools we're

[05:41:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20467s) going to procure, what the usage limits of those things

[05:41:09](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20469s) should be across the organization.

[05:41:11](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20471s) But then also, we can give recommendations for engineers

[05:41:14](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20474s) say, we think that for this GitHub issue,

[05:41:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20476s) you should actually use this particular coding tool with this model.

[05:41:21](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20481s) And this is able to drive the cost down considerably.

[05:41:24](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20484s) I think we're spending 90% less, but still able to get the same output.

[05:41:29](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20489s) I think the question I would have for you all,

[05:41:30](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20490s) we're continuing to expand this approach

[05:41:32](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20492s) and take this model of coding to other domains,

[05:41:35](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20495s) knowledge work, legal, finance, things like that.

[05:41:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20497s) But the key question is, where are you tracking the history of work

[05:41:41](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20501s) that has already been done by that firm?

[05:41:43](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20503s) Some of these law firms have been around for like 100 years.

[05:41:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20506s) And so where is that, I think, quality of the organization stored?

[05:41:50](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20510s) Where are you tracking the standard of excellence?

[05:41:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20512s) And so today may exist in places like email,

[05:41:55](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20515s) from the history of responses that people have had with their clients,

[05:41:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20518s) may exist in SharePoints, may exist in Databricks dumps,

[05:42:01](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20521s) but I think this is the key treasure trove that a lot of companies have

[05:42:04](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20524s) and don't realize will be the major unlock for evals

[05:42:07](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20527s) and then the unlock for AI adoption more systematically.

[05:42:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20536s) Once these evals are constructed, they can enable adoption to happen, but we have also

[05:42:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20542s) seen that there are more advantageous ways that they can be used.

[05:42:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20545s) First is right sizing.

[05:42:27](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20547s) Rather than just token maximizing, how do you figure out what the right usage limit should

[05:42:31](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20551s) be across the organization?

[05:42:33](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20553s) Negotiation, once you realize you have actual leverage, you can take that to the conversation

[05:42:37](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20557s) you have in bulk pricing.

[05:42:39](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20559s) Then routing, this is becoming a core primitive in a lot of AI products, but can also be done

[05:42:46](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20566s) done by users directly, and so they can build intelligent routers that actually figure out

[05:42:50](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20570s) the right model to use.

[05:42:52](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20572s) And then fine tuning or post training.

[05:42:54](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20574s) This can be the most reliable way for you to actually mint the tasks and then have an

[05:42:58](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20578s) external eval set to make sure that this approach is working.

[05:43:03](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20583s) Last thought I'd leave you with is I think for a lot of companies, they have a competitive

[05:43:08](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20588s) advantage that has existed in the heads of employees through things like institutional

[05:43:11](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20591s) knowledge.

[05:43:12](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20592s) and what evals allow you to do is actually codify the competitive advantage

[05:43:16](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20596s) and put that down in words or in scripts, in language

[05:43:19](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20599s) and that will actually be the biggest unlock

[05:43:22](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20602s) for making sure that a company succeeds the agentic era.

[05:43:25](https://www.youtube.com/watch?v=akEqiXFo3GM&t=20605s) Okay, that's all. Thanks, everyone.