1
00:00:07,161 --> 00:00:08,641
My name is Freddy Lecue.

2
00:00:08,821 --> 00:00:10,301
I'm part of the Wells Fargo team.

3
00:00:10,581 --> 00:00:13,861
I'm leading the Frontier AI modeling organization here.

4
00:00:14,221 --> 00:00:19,521
And I'm going to show you the work we've been doing lastly on building our first foundation

5
00:00:19,521 --> 00:00:22,261
model, Wells Fargo Foundation model.

6
00:00:22,521 --> 00:00:26,481
So I explain why we did that, how we did it, and what we accomplished.

7
00:00:26,861 --> 00:00:33,501
in doing so so so first from financial model it doesn't necessarily need to be language based i

8
00:00:33,501 --> 00:00:40,021
mean we're all playing with uh claude and chat gpt and others they're all language-based

9
00:00:40,021 --> 00:00:45,741
from the show models in financial services uh we also have language of course uh from a

10
00:00:45,741 --> 00:00:50,601
structure unstructured representation data but we also have a lot of structure representation

11
00:00:50,601 --> 00:00:56,041
a lot of these data is tabular data we have a lot of information from our customers we have

12
00:00:56,041 --> 00:01:02,181
information in investment banking, consumer banking, and all of this data is data that we

13
00:01:02,181 --> 00:01:08,581
have in Wells Fargo, and all big financial organizations have too. And we want to take

14
00:01:08,581 --> 00:01:15,221
benefit of all of this huge amount of data that we have. And it's not like you just take it and

15
00:01:15,221 --> 00:01:22,061
put it in a language model and get the best of it. Of course, you can get a lot of benefit out of it,

16
00:01:22,061 --> 00:01:25,061
but it's not necessarily the best way to do it.

17
00:01:25,241 --> 00:01:28,621
So what we decided to do is to look at the data we have,

18
00:01:28,781 --> 00:01:31,621
all the transactions, all the customer transactions

19
00:01:31,621 --> 00:01:34,861
that they're doing with their debit card, credit card, digital, check,

20
00:01:35,141 --> 00:01:38,001
all of those coming from many different channels

21
00:01:38,001 --> 00:01:41,601
and see if we can build a foundational model

22
00:01:41,601 --> 00:01:44,921
that we can use and fine-tune for downstream tasks

23
00:01:44,921 --> 00:01:48,481
like credit scoring or that you can use it for fraud detection,

24
00:01:48,661 --> 00:01:50,161
anti-money laundering, do marketing.

25
00:01:50,161 --> 00:01:56,881
So how we can build one backbone that you can use and streamline across downstream tasks.

26
00:01:57,121 --> 00:02:02,061
So now you're moving from many models very siloed across different organizations.

27
00:02:02,441 --> 00:02:12,261
In really one backbone that every organization can just reuse because you learn the representations of customers' transactions at scale.

28
00:02:12,501 --> 00:02:13,441
So that's the intent.

29
00:02:13,601 --> 00:02:16,521
And just spoiler alert, you may say, hey, that's tabular data.

30
00:02:16,521 --> 00:02:19,921
why do you just don't use XGBoost and the usual suspect?

31
00:02:20,241 --> 00:02:20,941
We did that.

32
00:02:21,081 --> 00:02:22,481
This is the model we had in production.

33
00:02:22,961 --> 00:02:27,321
But the model I'm going to show you is something that beats XGBoost,

34
00:02:27,481 --> 00:02:32,521
even from a latency perspective, but also from a fault detection,

35
00:02:32,881 --> 00:02:38,821
we get much better reduction of false positive and better detection rate.

36
00:02:39,061 --> 00:02:44,701
So it was a really good lesson learned in terms of having better model

37
00:02:44,701 --> 00:02:46,281
using foundation model.

38
00:02:46,281 --> 00:02:51,281
And to foundation, I mean transformer blocks, and that's what we build.

39
00:02:51,461 --> 00:02:57,361
So we have, as I mentioned, billions of transactions on a monthly basis.

40
00:02:58,101 --> 00:03:03,461
Our customers are interacting with their debit credit card, and they do checks, and they do different sort of transactions.

41
00:03:04,141 --> 00:03:11,821
And the question is really how we can use this information at scale, and not for one and every single downstream task.

42
00:03:11,821 --> 00:03:17,881
So I'm going to cover a bit of the data, the representations, the architecture that we use,

43
00:03:19,341 --> 00:03:23,501
what the result we get, and the lesson we learned in building this.

44
00:03:25,461 --> 00:03:28,901
So left-hand side, language foundational model.

45
00:03:29,181 --> 00:03:36,281
They're processing language tokens, and then the goal is to learn the representations that's going to predict the next token.

46
00:03:36,281 --> 00:03:39,821
and then you have supervised fine-tuning

47
00:03:39,821 --> 00:03:42,201
and then you do LHF, LRVR

48
00:03:42,201 --> 00:03:45,501
and all of the great post-training process

49
00:03:45,501 --> 00:03:48,641
to get better at the task that you want to, right?

50
00:03:48,741 --> 00:03:50,301
And that's for languages, that's great.

51
00:03:50,761 --> 00:03:54,661
But what if your data is very much more specialized

52
00:03:54,661 --> 00:03:56,121
in terms of representation?

53
00:03:56,821 --> 00:04:00,261
So that's where we build this transaction foundation model.

54
00:04:00,501 --> 00:04:03,761
So instead of having languages as the base,

55
00:04:04,101 --> 00:04:05,621
the base is now transactions.

56
00:04:06,381 --> 00:04:07,661
transaction from our customers.

57
00:04:08,441 --> 00:04:11,001
And you still try to predict the next thing,

58
00:04:11,181 --> 00:04:12,901
and here the next thing is the next transaction,

59
00:04:13,381 --> 00:04:15,201
or it's the next token in the transaction,

60
00:04:15,601 --> 00:04:17,381
using all of the channels that you have.

61
00:04:17,761 --> 00:04:19,301
But that's not the end goal.

62
00:04:19,641 --> 00:04:21,241
I mean, similarly to language model,

63
00:04:21,421 --> 00:04:22,921
where you try to predict the next token,

64
00:04:23,261 --> 00:04:25,061
that's not the end of the story, right?

65
00:04:25,141 --> 00:04:26,221
You learn the representation,

66
00:04:26,501 --> 00:04:28,001
and then you use that representation,

67
00:04:28,281 --> 00:04:30,621
and then you fine-tune it,

68
00:04:30,761 --> 00:04:33,381
and you post-tune it for any other task, right?

69
00:04:33,501 --> 00:04:35,481
So it could be classification tasks for languages,

70
00:04:35,481 --> 00:04:39,141
but in our context for transaction foundation model it could be used for

71
00:04:39,721 --> 00:04:43,881
credit line increase could be used for doing fraud detections and flagging

72
00:04:43,881 --> 00:04:45,021
bad actors from

73
00:04:45,161 --> 00:04:46,661
an anti-money-laundering perspective.

74
00:04:46,961 --> 00:04:55,581
And the goal is to learn this relationship between transactions over time and transactions across customers. That's really the intent.

75
00:04:55,841 --> 00:05:02,561
So the intent number one is you do pre-train from scratch. I'm saying it again because people are sometimes confused.

76
00:05:02,561 --> 00:05:10,801
we pre-train transformer blocks at scale from scratch using our transaction data.

77
00:05:11,061 --> 00:05:15,441
So the zero rate that we use from open rates, we just use the architecture,

78
00:05:15,781 --> 00:05:18,941
and then we pre-train it from scratch using our GPU cluster,

79
00:05:19,321 --> 00:05:23,081
and then we train it for a couple of days and weeks,

80
00:05:23,321 --> 00:05:27,481
and then you do the downstream task of post-training.

81
00:05:27,481 --> 00:05:30,161
but the goal here and you pre-train it

82
00:05:30,161 --> 00:05:32,041
for predicting the next

83
00:05:32,041 --> 00:05:33,521
token or the next transactions

84
00:05:33,521 --> 00:05:35,421
and you try to do this

85
00:05:35,421 --> 00:05:37,541
you try to predict the next pattern

86
00:05:37,541 --> 00:05:39,821
of the customer but that's not the intent

87
00:05:39,821 --> 00:05:41,861
the intent is to do the downstream

88
00:05:41,861 --> 00:05:43,901
task later on and in terms of scale

89
00:05:45,221 --> 00:05:46,341
we operate

90
00:05:46,341 --> 00:05:47,721
at a large scale

91
00:05:47,721 --> 00:05:49,781
so we have billions of transactions

92
00:05:49,781 --> 00:05:51,481
a million of accounts

93
00:05:52,201 --> 00:05:53,721
hundreds of transactions

94
00:05:53,721 --> 00:05:56,021
per client on a monthly

95
00:05:56,021 --> 00:05:57,321
basis interacting with their

96
00:05:57,321 --> 00:05:59,961
debit card, credit card, or doing check.

97
00:06:00,581 --> 00:06:03,601
And the five main dimensions we look at

98
00:06:03,601 --> 00:06:05,581
is really who is doing,

99
00:06:06,001 --> 00:06:08,361
I mean, who is processing and who is asking

100
00:06:08,361 --> 00:06:10,721
and who is providing a service.

101
00:06:10,921 --> 00:06:12,881
So, will it the customer or the merchant?

102
00:06:13,781 --> 00:06:14,581
The what?

103
00:06:14,761 --> 00:06:17,621
So, I mean, what do we buy?

104
00:06:17,781 --> 00:06:18,681
What do we transact?

105
00:06:19,361 --> 00:06:19,901
And how?

106
00:06:20,141 --> 00:06:20,641
So, which channel?

107
00:06:20,781 --> 00:06:21,401
Is it credit?

108
00:06:21,521 --> 00:06:22,081
Is it debit?

109
00:06:22,241 --> 00:06:22,881
Is it digital?

110
00:06:23,301 --> 00:06:24,441
When is it happening?

111
00:06:24,781 --> 00:06:25,941
And where is it happening?

112
00:06:25,941 --> 00:06:29,641
And all of those, they're basically breaking down in hundreds of features.

113
00:06:29,981 --> 00:06:33,141
And if you look at the existing model we have in production,

114
00:06:33,581 --> 00:06:39,341
there's a lot of people that build those models in similar organizations

115
00:06:39,341 --> 00:06:41,681
may understand that.

116
00:06:41,781 --> 00:06:45,461
They build a lot of very hard-crafted, uncrafted engineering,

117
00:06:46,101 --> 00:06:48,181
feature engineering efforts on that.

118
00:06:48,261 --> 00:06:49,941
And that's required a lot and a lot of time.

119
00:06:50,121 --> 00:06:53,741
So we step back and look at the raw features we need,

120
00:06:53,741 --> 00:06:59,641
not the one that's been pre-built and post-built for doing the downstream tasks,

121
00:06:59,721 --> 00:07:01,701
but try to understand which raw features.

122
00:07:01,861 --> 00:07:04,541
And we ended up with basically Android features.

123
00:07:04,681 --> 00:07:07,661
Even at the end of the day, we had thousands of them,

124
00:07:07,781 --> 00:07:12,561
but we tried to find which ones are the raw information unit.

125
00:07:12,781 --> 00:07:16,161
And then we led the architecture, the transformer blocks,

126
00:07:16,321 --> 00:07:19,281
to figure out the relationship between those core features.

127
00:07:19,281 --> 00:07:21,501
and as you can see

128
00:07:21,501 --> 00:07:24,981
we do have a bunch of features

129
00:07:24,981 --> 00:07:26,581
on those five dimensions

130
00:07:26,581 --> 00:07:29,121
and the vocabulary is very unique

131
00:07:29,121 --> 00:07:32,221
to the financial services setup

132
00:07:32,221 --> 00:07:34,901
and that's the reason why

133
00:07:34,901 --> 00:07:36,721
we didn't really use

134
00:07:36,721 --> 00:07:38,781
existing pre-trained model

135
00:07:38,781 --> 00:07:41,541
because the vocabulary that we use

136
00:07:41,541 --> 00:07:43,541
is very specific to the work

137
00:07:43,541 --> 00:07:44,321
that you want to do

138
00:07:44,321 --> 00:07:46,621
and even if you merge

139
00:07:47,501 --> 00:07:51,781
while all digital and checking and saving

140
00:07:51,781 --> 00:07:55,261
and all of those sort of transactions,

141
00:07:55,541 --> 00:07:57,001
even if you merge all of those,

142
00:07:57,141 --> 00:08:00,261
you still end up with a very limited number of features and characteristics.

143
00:08:00,881 --> 00:08:03,401
So at that scale, then the question is,

144
00:08:03,521 --> 00:08:05,581
well, how you work with that?

145
00:08:05,741 --> 00:08:08,061
So we started, everything is tabular data.

146
00:08:08,221 --> 00:08:09,861
I mean, there's no secret, it's tabular data.

147
00:08:10,201 --> 00:08:11,901
And then the goal is to understand

148
00:08:11,901 --> 00:08:14,401
how you're going to capture the vocabulary

149
00:08:14,401 --> 00:08:16,381
because you do have numerical values,

150
00:08:16,381 --> 00:08:21,961
you have categorical values, you have information like zip code,

151
00:08:22,241 --> 00:08:24,541
do you encode each and every zip code in a token?

152
00:08:24,761 --> 00:08:25,821
Does it make sense?

153
00:08:26,061 --> 00:08:29,521
So there's a lot of efforts you need to push in understanding

154
00:08:29,521 --> 00:08:34,741
the vocabulary that you need and the vocabulary that you want also to scale

155
00:08:34,741 --> 00:08:36,861
because at the training, if you have too huge vocabulary,

156
00:08:37,221 --> 00:08:41,341
it would be kind of messy to train such a model with a vocabulary

157
00:08:41,341 --> 00:08:42,721
which is far, far too big.

158
00:08:42,721 --> 00:08:46,941
So there's a lot of effort in understanding this value to tokens.

159
00:08:47,241 --> 00:08:49,601
So which information do you encode in a token?

160
00:08:49,821 --> 00:08:52,161
And then you have the token to ID, the tokenizations,

161
00:08:52,461 --> 00:08:53,861
and then the embedding lookup.

162
00:08:54,021 --> 00:08:56,361
So then when we have a transaction coming up,

163
00:08:56,541 --> 00:08:57,761
you do have the vocabulary.

164
00:08:58,161 --> 00:09:01,441
So basically it's a lookup in aligning a transaction

165
00:09:01,441 --> 00:09:04,241
and all the characteristics of that transaction

166
00:09:04,241 --> 00:09:07,601
to a token and an ID and then an embedding.

167
00:09:10,101 --> 00:09:11,881
So just to give you a number,

168
00:09:11,881 --> 00:09:17,881
well some numbers so we have quantitative qualitative categorical of different kind

169
00:09:17,881 --> 00:09:23,941
right so categories with only eight types some with much bigger types you have transaction amount

170
00:09:23,941 --> 00:09:29,441
which is a big deal you have the delta time between transaction so you have anything related

171
00:09:29,441 --> 00:09:36,501
to ip address postal code so just to give you example of variety and heterogeneity of each and

172
00:09:36,501 --> 00:09:40,481
every characteristic could be encoded in a binary token

173
00:09:40,481 --> 00:09:43,901
or could be encoded in a much larger number of tokens.

174
00:09:44,381 --> 00:09:46,341
And even if you look at the zip code,

175
00:09:46,561 --> 00:09:48,821
I mean, you have a much larger number of zip code

176
00:09:48,821 --> 00:09:52,121
in the US than those 2,800,

177
00:09:52,561 --> 00:09:56,621
but you have much better way to structure

178
00:09:56,621 --> 00:10:00,101
how you're going to regroup zip code

179
00:10:00,101 --> 00:10:02,101
based on distributions, for instance.

180
00:10:02,101 --> 00:10:04,901
so architecture

181
00:10:04,901 --> 00:10:06,041
so here just

182
00:10:06,041 --> 00:10:08,461
so we use the

183
00:10:08,461 --> 00:10:09,161
chinchilla

184
00:10:10,461 --> 00:10:11,941
mathematics behind

185
00:10:11,941 --> 00:10:14,661
to define how we build it

186
00:10:14,661 --> 00:10:16,761
initially because we had

187
00:10:16,761 --> 00:10:18,661
no clue how it's going to work in the tabular

188
00:10:18,661 --> 00:10:20,721
space even though there are

189
00:10:20,721 --> 00:10:21,241
existing

190
00:10:22,481 --> 00:10:24,901
open ways and open solutions

191
00:10:24,901 --> 00:10:26,921
for doing tabular

192
00:10:26,921 --> 00:10:27,981
data foundation models

193
00:10:27,981 --> 00:10:30,481
so we started from

194
00:10:30,481 --> 00:10:34,441
what they learn and adapt it to what we needed to do and that gives you an

195
00:10:34,441 --> 00:10:39,621
example of invading dimensions number of decoder blocks we use and all of the

196
00:10:39,621 --> 00:10:45,621
recipe from a from an architecture perspective so we have and as you can

197
00:10:45,621 --> 00:10:50,381
see we're not trying to build a trillion parameters model we don't try to compete

198
00:10:50,381 --> 00:10:56,301
with any of the the leaders in the language models because we working with

199
00:10:56,301 --> 00:10:59,501
and focusing on transactions, right?

200
00:10:59,641 --> 00:11:02,481
Transactions, foundation models at billion level.

201
00:11:03,061 --> 00:11:06,401
And one billion is because that's the model that gave us,

202
00:11:06,561 --> 00:11:10,541
that was kind of the smallest and gave us the highest boost.

203
00:11:10,941 --> 00:11:13,921
And now we're working on the next versions at 11 billion.

204
00:11:14,161 --> 00:11:18,161
But that's good enough to give a lot of ROI

205
00:11:18,161 --> 00:11:22,201
from a false positive reduction for downstream tasks

206
00:11:22,201 --> 00:11:24,541
like for detections and the detection,

207
00:11:24,861 --> 00:11:26,261
unit detection rate.

208
00:11:27,001 --> 00:11:30,121
So, and the type of architecture,

209
00:11:30,301 --> 00:11:31,741
so the decoder-only architecture,

210
00:11:32,021 --> 00:11:35,121
that's basically for predicting the next token

211
00:11:35,121 --> 00:11:36,021
or the next transactions.

212
00:11:36,481 --> 00:11:38,201
And that's really the pre-training.

213
00:11:39,161 --> 00:11:42,141
So here we initially used 20 times the number of parameters

214
00:11:42,141 --> 00:11:44,441
to define the number of tokens we might need.

215
00:11:44,621 --> 00:11:47,521
We ended up actually 10 times fractured than that.

216
00:11:47,641 --> 00:11:49,521
So by doing a few experimentations,

217
00:11:49,521 --> 00:11:53,581
we needed much more tokens per parameters.

218
00:11:53,821 --> 00:11:57,201
I mean, this is kind of the initial experimentation we did.

219
00:11:58,461 --> 00:12:01,861
So I'll pass on this because this is the usual suspect

220
00:12:01,861 --> 00:12:04,121
from an objective function,

221
00:12:04,301 --> 00:12:07,341
so causal language modeling and mass language modeling

222
00:12:07,341 --> 00:12:12,601
type of optimization work.

223
00:12:12,821 --> 00:12:14,661
And from an architecture perspective,

224
00:12:15,361 --> 00:12:17,781
nothing really fancy here.

225
00:12:18,401 --> 00:12:21,341
the most complex part was really the tokenization

226
00:12:21,341 --> 00:12:22,861
and then

227
00:12:22,861 --> 00:12:25,701
doing the pre-training and then how you do the fine-tuning.

228
00:12:26,201 --> 00:12:28,821
So I would say the recipe or the secret sauce

229
00:12:28,821 --> 00:12:30,741
is really how you do the tokenization

230
00:12:30,741 --> 00:12:33,441
and how you're going to do the post-training.

231
00:12:33,581 --> 00:12:35,581
But anything from a training perspective,

232
00:12:37,201 --> 00:12:38,501
the question is just

233
00:12:38,501 --> 00:12:41,101
how many transactions

234
00:12:41,101 --> 00:12:43,241
you're going to put in your context

235
00:12:43,241 --> 00:12:45,521
or in your history for doing the training.

236
00:12:45,521 --> 00:12:49,521
and evaluation perplexity-based accuracy dashboard

237
00:12:50,281 --> 00:12:51,781
on downstream tasks.

238
00:12:51,961 --> 00:12:53,261
That's the ULs respect.

239
00:12:54,261 --> 00:12:58,001
And so, yeah, so use mixed precision,

240
00:12:58,221 --> 00:12:59,601
distributed data parallel,

241
00:13:00,501 --> 00:13:02,421
maximize macro batch size,

242
00:13:03,441 --> 00:13:04,721
get faster convergence,

243
00:13:05,841 --> 00:13:07,561
yeah, using Muran,

244
00:13:08,441 --> 00:13:09,761
Rope, usual,

245
00:13:10,221 --> 00:13:12,001
and memory efficiency, flash attention,

246
00:13:12,461 --> 00:13:13,421
activation chain pointing.

247
00:13:13,421 --> 00:13:15,681
I mean, here there's nothing really new.

248
00:13:15,981 --> 00:13:20,621
It's just we use the advices from the best

249
00:13:20,621 --> 00:13:23,801
and from the one that built even much bigger models.

250
00:13:24,901 --> 00:13:26,741
So I'll pass on this one.

251
00:13:27,021 --> 00:13:27,641
Time is short.

252
00:13:27,901 --> 00:13:32,401
But I wanted to spend some time on the results.

253
00:13:32,841 --> 00:13:37,801
So, yeah, so when you build this foundation,

254
00:13:37,801 --> 00:13:43,521
That foundation is really, is only a backbone.

255
00:13:43,701 --> 00:13:45,821
It's something you can use for downstream tasks.

256
00:13:46,181 --> 00:13:49,461
And the downstream task was to do fraud classification.

257
00:13:49,821 --> 00:13:54,221
So you have a transaction, and then you try to predict if that transaction is going to be fraudulent.

258
00:13:54,681 --> 00:13:58,741
So in the model we have in production, it's an XGBoost model,

259
00:13:58,801 --> 00:14:04,241
and it's trained on large amount of transactions, which is fraudulent or non-fraudulent.

260
00:14:04,241 --> 00:14:06,001
and the missing pieces

261
00:14:06,001 --> 00:14:08,741
was the relationship

262
00:14:08,741 --> 00:14:10,961
between whatever transaction

263
00:14:10,961 --> 00:14:13,101
you try to predict, try to classify

264
00:14:13,101 --> 00:14:14,921
and the relationship with this

265
00:14:14,921 --> 00:14:17,141
transaction with all of the past transactions

266
00:14:17,141 --> 00:14:19,081
and that information is kind of

267
00:14:19,081 --> 00:14:21,241
gone when you use an XGBoost model

268
00:14:21,241 --> 00:14:23,201
unless you encode it

269
00:14:23,201 --> 00:14:25,001
in a very hard way which is not

270
00:14:25,001 --> 00:14:26,921
so easy but if you use

271
00:14:26,921 --> 00:14:28,921
transformer architectures you do have

272
00:14:28,921 --> 00:14:31,041
these nice attentions

273
00:14:31,041 --> 00:14:32,961
from transactions to

274
00:14:32,961 --> 00:14:34,001
past transactions

275
00:14:34,001 --> 00:14:41,321
And that's basically what helps, and this is what we think, this is what really helps pushing the fraud detection rate much higher.

276
00:14:41,481 --> 00:14:44,941
But at the end of the day, the downstream task was to say, here's the

277
00:14:45,021 --> 00:15:00,721
sequence of transactions. I have someone just using the debit card. Is that transaction fraudulent with respect of all historical information? And that's exactly what you put in at inference time when you try to do the prediction.

278
00:15:00,721 --> 00:15:03,821
so that's the

279
00:15:04,541 --> 00:15:06,241
architecture so nothing

280
00:15:06,241 --> 00:15:08,161
really fancier you just need to have the

281
00:15:08,161 --> 00:15:09,941
good engineering

282
00:15:09,941 --> 00:15:11,161
in place to do

283
00:15:11,161 --> 00:15:13,861
less than 250

284
00:15:13,861 --> 00:15:15,581
milliseconds for doing the processing

285
00:15:16,521 --> 00:15:17,761
because the

286
00:15:17,761 --> 00:15:20,121
decline needed to happen within that time frame

287
00:15:20,121 --> 00:15:21,841
but it's

288
00:15:21,841 --> 00:15:22,121
we were

289
00:15:23,121 --> 00:15:25,801
10 times slower than done from a latency perspective

290
00:15:25,801 --> 00:15:27,221
so it was all good

291
00:15:27,221 --> 00:15:29,441
and just in terms of

292
00:15:29,441 --> 00:15:31,821
final result we get. So the main

293
00:15:31,821 --> 00:15:33,901
metrics and those metrics

294
00:15:33,901 --> 00:15:35,541
relate right away to

295
00:15:38,341 --> 00:15:39,941
savings for the firm.

296
00:15:40,121 --> 00:15:41,701
So it's detection rate

297
00:15:41,701 --> 00:15:43,781
and false positive rate. Of course

298
00:15:43,781 --> 00:15:45,781
you can detect much higher, but

299
00:15:45,781 --> 00:15:47,461
in many cases

300
00:15:47,461 --> 00:15:49,481
sometimes you get much more false positive.

301
00:15:49,721 --> 00:15:51,761
And in this case, we managed to get

302
00:15:51,761 --> 00:15:53,421
more fraudulent

303
00:15:53,421 --> 00:15:55,661
activities and we even

304
00:15:55,661 --> 00:15:57,101
reduced the false positive.

305
00:15:57,101 --> 00:15:58,781
so that was a big win

306
00:15:58,781 --> 00:16:00,481
because at the end of the day

307
00:16:00,481 --> 00:16:03,081
if you decline a transaction

308
00:16:03,081 --> 00:16:04,181
which is not fraudulent

309
00:16:04,181 --> 00:16:06,441
then that person is going to call

310
00:16:06,441 --> 00:16:08,961
there is an operation cost associated to it

311
00:16:08,961 --> 00:16:11,021
you lose the transaction related to

312
00:16:11,021 --> 00:16:13,101
that non-fraudulent action

313
00:16:13,101 --> 00:16:14,701
so it's a very

314
00:16:16,281 --> 00:16:16,841
well

315
00:16:16,841 --> 00:16:19,261
something that you don't really want to happen

316
00:16:19,261 --> 00:16:20,961
so again

317
00:16:20,961 --> 00:16:23,361
that's compared to the state of the art

318
00:16:23,361 --> 00:16:24,721
of the XGBoost model

319
00:16:24,721 --> 00:16:26,441
using fraud detection

320
00:16:26,441 --> 00:16:29,181
So it's basically huge.

321
00:16:29,401 --> 00:16:31,841
It's millions of dollars savings.

322
00:16:32,501 --> 00:16:34,801
So it's a very big boost.

323
00:16:35,081 --> 00:16:38,301
So just to finish up on that slide,

324
00:16:38,541 --> 00:16:45,721
so that foundation model has been built on billions of transactions

325
00:16:45,721 --> 00:16:48,581
from Wells Fargo customers.

326
00:16:49,221 --> 00:16:50,641
That's number one.

327
00:16:51,621 --> 00:16:53,681
It's pre-trained from scratch.

328
00:16:53,681 --> 00:16:56,781
So we don't free train from any open-rate existing model.

329
00:16:56,961 --> 00:17:02,081
We use existing architecture, but the way the zero rate, we pre-training on a couple

330
00:17:02,081 --> 00:17:02,781
of days.

331
00:17:03,081 --> 00:17:06,341
I mean, we did a couple of weeks initially, but then that's pre-trained.

332
00:17:06,461 --> 00:17:10,601
And then when it's pre-trained, it's pretty good in predicting the next transaction.

333
00:17:11,081 --> 00:17:14,501
And when you're done with that, then you can use it for many downstream tasks.

334
00:17:14,621 --> 00:17:15,781
We did it for fraud detection.

335
00:17:16,261 --> 00:17:20,541
Now we're investigating for credit line increase, investigating for anti-money laundering,

336
00:17:21,161 --> 00:17:22,241
investigating for marketing.

337
00:17:22,241 --> 00:17:24,801
so now the same foundation is used

338
00:17:24,801 --> 00:17:26,101
across different organizations

339
00:17:26,941 --> 00:17:28,481
even Wells Fargo

340
00:17:28,481 --> 00:17:30,041
and yeah

341
00:17:30,041 --> 00:17:32,661
thank you for having me and thank you