Your GPU Neighbor Can Hear Your Model

Hello everyone, this talk is about the danger of your LLMZ formation leaking through shared

GPU hardware.

If you're running a model in the cloud on a GPU you share with other tenants, someone

next to you can learn things about it.

They can tell what model you're running, how fast it's responding, and even what kind

work is doing right now, all without touching your data, just by watching timing. That's what

this talk covers, what I quote tenets on the same hardware can learn about your LLM from timing alone

and what you can do about it. To start, I'm Branislav Brzak, a cyber security researcher employed

at Stealthium. At Stealthium we specialize in securing and providing observability into the environments

AI systems run on. My area of focus is low-level GPU-related security research. That's something

these stock covers. When you share a GPU between tenants, you would expect full isolation, but

the isolation provided may not prove to be sufficient. In the case of this attack, the

attacker doesn't even need any special permissions like root access. It doesn't need any traditional

exploit either. The only thing it does is time the parts of the hardware that are still shared among

tenants. This is a demonstration video of what the victims and the attackers dashboards look like.

On the left, we have Qwen3 model doing inference.

On the right, the attacker uses a side channel that Qwen3 leaks into as it's running.

From that, it determines exactly what model it is and what kind of text it's producing right now.

These are just the headline claims, but not the whole story.

We can tell which model it is, architecture and depth, with the 97% accuracy.

We can tell how fast is the coding, with the 99.3% accuracy.

We can tell when and how much, the busy periods, token counts, and the answer length.

What we can tell what kind of work it's doing, code, pros, JSON, or retrieval.

What we cannot tell though are the actual prompt, answers, or contents, just timing.

And all of this on real H100s serving real LLMs.

If you're using NVIDIA hardware, GPU isolation is achieved using multi-instance GPU or MIG

for short.

Each isolated MIG slice has its own physical compute units and dedicated VRAM that provides

hardware isolation on those parts.

In our example, the LLM we are serving runs on a slice with 3 compute units and 40 GB

of VRAM.

The attack runs on just one compute unit and 10 GB of VRAM.

What's isolated are compute units, memory capacity, bandwidth quotas, and faults and errors.

The L2 cache is partitioned per instance too.

There is no cross-instance eviction.

So it's not shared in that sense.

The actual leaks are memory barriers or members, which are GPU-wide.

One instance's barrier still touches every L2 partition on the card.

Timing is the one thing that isn't isolated.

So when one tenant is busy, the other tenant's own reads get slightly slower.

The key point is that a GPU-wide memory barrier takes effect in every L2 partition on the card.

The next slide builds the whole attack on that one fact.

The vector is that the attacker only ever reads its own memory.

It only times its own reads, it never touches the victim's memory.

There are three steps. Load from its own small buffer, time the load, and repeat every 5 microseconds.

This primitive and the MIG member leak itself isn't our discovery.

That goes to the researchers behind behind bars at using security.

A big thanks for their work.

Now try looking at the timing differences.

With the naked eye comparing your tenant idle versus serving an LLM, you wouldn't even be able to discern them.

The median latency is only 3 microseconds higher.

To the attacker though, those 3 microseconds are all it needs.

enough to gather all the information we already talked about.

Qwen3 with 8 billion parameters is running on the victim's slides throughout this one.

All captured data was taken from a live model.

At the top is GPU activity over time, 1 burst per token.

Below is the frequency spectrum of the attacker's own latency trace.

There is a fundamental 881Hz which is just tokens per second plus harmonics, since each

token is a mix of distinct frequencies, not one clean one.

This is a real trace, one token in the time domain.

The token shows a total of 39 bumps.

bumps with green peaks are the layer invocations, while orange ones are additional between layer

processing.

In a later slide we will demonstrate how we determine the actual layer counts even though

those extra bumps are present.

we look at the decode speed itself.

Just by timing we found the token rate completely bind.

So speed alone is something an attacker can recover accurately.

But speed by itself isn't a clean fingerprint.

Qwen3 8 billion and Llama 3 8 billion decode only 1.6% apart in their token rate.

So from speed alone you'd think they're the same model.

It narrows things down but it doesn't nail it.

We need something sharper, layer counts.

That's what the next few slides get into.

We use the same fold and average tick from before.

This time we run it once per model instead of once per token.

What comes out is a bit count that lands almost exactly on a whole number.

The mean distance from an integer across 22 traces is just 0.028.

That's the tell.

If this were just a fixed hardware artifact divided by different token rates, it'd expect

fractions, not whole numbers.

Each count works out to the model's layer count plus 1 to 3 additional peaks.

model keeps its own signal fingerprint.

A mix of tempo and harmonic structure.

To be clear, this is not a trained machine learning classifier.

There's no training set, no labels, no configuration.

It's under 30 lines of code.

The method checks each candidate's depth from 16 to 48.

It measures how strongly that depth's expected signature shows up in the real trace.

Then it picks which are of its best.

For Qwen3, 8 billion 36 layers wins Cleary by 4.0 times over the next candidate.

Because this is a best fit test, not a lookup table,

it also works on models it never seen before.

That includes proprietary on unreleased ones,

since there's nothing it needs to have seen in the first place.

And this works surprisingly well.

The layer fit classifier also outputs the confidence level.

Across R testing, only two least confident answers

ended up being wrong ones.

This same method also picks up fine tunes and distillations that share the same base

model.

That's because the architecture and layer count carry over unchanged.

Take three related models as an example.

Llama 3 8 billion from meta has 32 layers.

Llama 3.1 8 billion also from meta uses that same 32 layer backbone with minor upgrades.

A R1 distilled Llama 8 billion from DeepSeek is an improvement on top of that.

The per-layer timing comes out to only 1.6 nanosecond median latency shifts of each other.

The rest of the signal looks the same.

Regardless of this, our classifier is still able to discern between them.

Now we ask a deeper question on the same channel.

Not which model, but what is doing right now.

We look at four workflow types.

Prose, code, JSON, and retrieval.

We give a workload on every half second based on the last five seconds of timing.

Out of 40 half-second workloads 31 were right.

And four full responses were correctly identified by majority votes.

This is the same classifier from the video earlier misses included.

One condition is worth stating.

This was measured with speculative decoding turn-on.

That's an opt-in in vLLM,

not the default. Here's the mechanism. How predictable the text is changes how many drafted tokens get accepted at each step. That changes step time and that's the signal we're picking up. We tested 11 configurations, dense and mixture of experts, BF16 and FP8. The same figure page showed up every time, just at very different strengths.

17.2% spread on GPT OSS 20 billion vs 0.9% on GLM-4-9B.

10 out of 11 round pros fastest when the chance alone would only give you 1 in 4.

So say the attacker already knows the model from the earlier slides and specular the recording

happens to be on.

It can also tell what kind of text is coming out.

Now we take the Shadeship out of the picture entirely.

MIG needs the attacker on the same physical die.

This attack doesn't. The setup here is an 8x H100 server connected over an NVSwitch fabric.

The victim runs tensor parallel across GPU 0 and 1. The attacker just needs any other GPU in that

same server, GPU 2 in this case. Peer access over NVSwitch is a real intended feature. It's

how multi-GPU jobs normally work, not a bug being abused. MIG tensor parallelism also can't coexist

on the same device. So this is really a different population of deployments, not just the same

attack repeated. The probe runs on GPU 2. It pointer traces memory that actually is on GPU 0

over NVLink, timing every hop along the way. An idle hop takes about 890 nanoseconds. Once the

victim starts working that climbs up to 300 microseconds.

These are three half-second captures and you can actually count the tokens by eye.

What we lose compared to MIG is the layer bit is below the noise at this

distance. So over-enviling the fingerprints we

were left with is mostly tempo, not layer count.

Here we test the five models on identical serving setup. Only the model itself gives away which one it is.

This method gives 79.2% accuracy. Now it fails exactly where you'd expect the tempo to fail.

LAMA 3.58 billion and R1D Steel are both 8 billion models that they code 3Hz apart.

So they blur together, where MIG's layer counts would have separated them easily.

The takeaway is that the NVLink signal-to-noise ratio is worse than MIGs, since it loses the

layer bit. But worse signal-to-noise ratio isn't the same as safety. 79% against the 20% baseline

is still a real fingerprint, not noise. First, some good news. Our own product at work

caught the attack straight out of the box. Stealthium monitors every single kernel launch

across every GPU, analyzes suspicious workloads and runs them through a full alert pipeline,

which is something not every vendor can do. Here is what made this one stand out. A single

kernel relaunched about 3000 times, hitting the exact same 40 kilobytes of memory over and over

instead of moving to new data. Only one kilowatt of data was sent back to the CPU the entire time

because the timing itself was the payload, not the output.

In real terms, that is under 600 milliseconds start to finish with fewer than 50 tokens over reply.

We caught it before the response was ever half typed.

So, three takeaways.

First, placement is something operators actually control.

So do not put an untrusted tenant's MIG slice right next to sensitive model.

Second, know the trade-off.

MIG rules out peer-to-peer communication, meaning there is no tensor parallelism on

MIG.

Sharding over NVLink instead limits what leaks to timing alone rather than the layer accounts.

Third, watch the GPU itself.

Make sure the host infrastructure is actively verifying that no malicious workloads are running

on the hardware.

That's exactly what Stealthium does by catching these probes straight from the Chrome behavior.

Sharing a GPU means timing alone can give away the model, the speed, and the workload.

That's true even across NVLink, without needing any export at all.

Stealthium sees this happen in real time. Most organizations still don't have that kind of visibility.

Thanks again to the researchers of Behind Bars, this whole talk builds upon their work.

Thank you all for watching.