Protecting weights in use - TEEs & attestation

Confidential computing closes exactly one gap - plaintext weights and prompts sitting in GPU memory where a privileged host could read them - and everything it appears to promise beyond that is either attestation doing the work or marketing.

Start here

• The gap is narrow and real. Encryption at rest and TLS in transit leave the in-use state uncovered: while a model serves, its weights and the user’s prompt sit decrypted in GPU memory.
• Attestation is the control, not the encryption. Build a key service that refuses to release the weight-decryption key unless a hardware-signed quote proves both the CPU trust domain and the GPU are genuine and in the expected state. No valid quote, no key, no plaintext weights.
• Three sub-$1,000 memory interposers have now broken attestation. Two of them forged quotes that passed the vendors’ own verification libraries; the third bypassed AMD SEV-SNP attestation by replaying launch measurements. Intel and AMD both classify interposers as outside their threat models. That does not make confidential computing useless. It means what you bought is a strong control against software and remote-operator access, resting on the provider’s physical security.

This is the deep dive on the mechanism II.5 · Guardrails - what holds, and how to prove it names in passing and I.7 · Frontier capability & the if-then frameworks assumes when it says “secure the weights.”

The gap it closes

State	Typical control	Covered?
At rest (disk, registry, object store)	Encryption at rest; signed, provenance-verified artifacts (I.5)	Yes
In transit (network, PCIe)	TLS; encrypted and signed PCIe transfers in confidential-computing mode	Yes
In use (loaded in GPU VRAM)	Host isolation, IAM, trust in the operator	The gap

The in-use gap only matters when someone else controls the host. On your own metal it is a rounding error next to the twenty other ways in. On rented GPUs, a neocloud, or a shared-tenancy inference platform, it is the difference between “our provider promises not to look” and “our provider cannot look.”

The hardware, as of mid-2026

A trusted execution environment (TEE) is a hardware-isolated, attested region the privileged host cannot read into. AI serving needs two, stitched together.

Layer	What it covers	The catch
NVIDIA GPU Confidential Computing	The only piece that closes the in-use VRAM gap. NVIDIA lists support on Rubin, Blackwell, and Hopper GPUs	Generation-dependent. The rack-scale claim is for Vera Rubin NVL72 (72 Rubin GPUs, 36 Vera CPUs) across NVLink
Intel TDX	The CPU-side host for the GPU workload	Quote forgery demonstrated by the DDR5 interposer below
AMD SEV-SNP	Same role; the CPU TEE under Azure’s confidential GPU SKU	Attestation replay (Battering RAM); Ciphertext Hiding defeated (TEE.fail)
AWS Nitro Enclaves	Key handling and decryption; attested via NitroTPM	CPU and memory only. AWS asserts no operator can access instance data or data sent to a GPU, but that is an architectural assertion, not a customer-verifiable GPU quote
Composite CPU + GPU attestation	The thing that actually gates the weights	Never release weights on GPU attestation alone

What you can buy, with dates: Azure’s Standard_NCC40ads_H100_v5 puts the TEE across an AMD EPYC Genoa confidential VM and one attached H100 NVL, and Microsoft’s size page notes it does not support live migration. Google announced on 24 June 2026 that Confidential Space with H100 is GA, that Intel Trust Authority is GA as an independent verifier “decoupling attestation verification from the cloud service provider,” and that Confidential G4 with RTX PRO 6000 Blackwell is in preview. On 9 June 2026 NVIDIA announced Blackwell GPUs with Confidential Computing on Google Cloud serving Apple Foundation Models for Private Cloud Compute - the most consequential production reference the technology has.

Attestation is the whole control

Encryption without attestation buys nothing: the host that hands you a fake TEE hands you a fake guarantee.

flowchart TB
  GPU["GPU root of trust<br/>signing key fused at manufacture,<br/>never exposed to software"] --> EV["Evidence bundle<br/>GPU hardware report +<br/>CPU TEE measurement<br/>(SEV-SNP or TDX)"]
  EV --> VER["Verifier<br/>checks evidence against a<br/>reference integrity manifest"]
  VER --> KMS["KMS / HSM<br/>release policy evaluates<br/>the attestation claims"]
  KMS --> WK["Weight-decryption key<br/>released into the TEE only"]
  WK --> INF["Inference<br/>weights never plaintext<br/>outside the TEE"]
  classDef d fill:#0f1a18,stroke:#5bd1c5,color:#bdeee2;
  classDef l fill:#1a160f,stroke:#e4a23f,color:#f0d9ad;
  class GPU,EV,KMS,WK,INF d;
  class VER l;

The verifier is amber because the whole design rests on it. Break it and every other box is decoration - which is exactly what the interposers below do.

Per NVIDIA’s 2 July 2026 engineering write-up: each GPU carries a private signing key fused at manufacture and never exposed to software, firmware or the host; the NVIDIA Remote Attestation Service verifies a signed evidence bundle - the GPU hardware report plus CPU TEE measurements from SEV-SNP or TDX - against a known-good reference integrity manifest. Attestation happens once at startup, adding no per-request latency.

The artifact that turns that into a control is a release policy on the key. Azure’s Secure Key Release is the clearest documented example: an exportable HSM-backed key in Key Vault Premium or Managed HSM is released only to a TEE whose Microsoft Azure Attestation token satisfies a policy attached at key-creation time.

KMS-backed weight decryption: the key never leaves the HSM unless the TEE attests

# 1) Write a release policy naming the attestation authority and required claims.
#    An empty or over-broad policy is the whole control failing open.
cat > skr-policy.json <<'JSON'
{
  "version": "1.0.0",
  "anyOf": [{
    "authority": "https://<region>.attest.azure.net",
    "allOf": [
      { "claim": "x-ms-isolation-tee.x-ms-attestation-type", "equals": "sevsnpvm" },
      { "claim": "x-ms-isolation-tee.x-ms-compliance-status", "equals": "azure-compliant-cvm" }
    ]
  }]
}
JSON

# 2) Create the wrapping key as exportable, bound to that policy. Non-exportable keys
#    cannot be released to a TEE at all; exportable-with-no-policy is the failure mode.
az keyvault key create --vault-name <hsm-vault> --name model-wrap-key \
  --kty RSA-HSM --exportable true --policy @skr-policy.json

# 3) Encrypt the weights ONCE under a data key wrapped by that KMS key, and publish only
#    the ciphertext. Use a real AEAD. Note that `openssl enc` CANNOT do this: its own
#    manual says it "does not support authenticated encryption modes like CCM and GCM,
#    and will not support such modes in the future". Use age, or a few lines of Python.
python - <<'PY'
import os
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
dek = AESGCM.generate_key(bit_length=256)          # raw 256-bit key, not a passphrase
nonce = os.urandom(12)
pt = open("model.safetensors", "rb").read()
open("model.safetensors.enc", "wb").write(nonce + AESGCM(dek).encrypt(nonce, pt, None))
open("dek.bin", "wb").write(dek)                    # wrap this with the KMS key, then destroy
PY

# 4) Inside the confidential VM the workload attests, then calls key RELEASE (AKV 7.3+).
#    No valid attestation token -> no key -> serving cannot start. Fail closed.

Two things to notice. Microsoft’s documentation states that “AKV only understands and integrates with MAA today,” so the verifier is not portable across clouds. Google’s answer is the opposite bet: Intel Trust Authority is GA precisely so attestation verification is decoupled from the cloud provider. If your reason for doing any of this is “we do not want to trust the cloud provider,” a verifier operated by that provider partially defeats the purpose.

Second, the same release-policy pattern gates more than weights: it is how you keep a retired model from being reloaded (VII.5 · Retirement & decommissioning) and how you crypto-shred one at end of life.

What the operator can still observe

A TEE protects contents, not the shape of the workload.

• Metadata and traffic shape. Request timing, request and response sizes, streaming cadence, GPU utilization curves, billing records. Prompt and output length are visible even when contents are not.
• Availability. The academic analysis of GPU confidential computing (Gu et al., arXiv:2507.02770, MLSys 2026) states the threat model excludes DoS, “as adversaries can control compute resources and launch Denial of Service (DoS) attacks from the host.” Anyone who can run your workload can stop it.
• The control plane. Who launches the workload, which image gets attested, who publishes the reference integrity manifest, who can change the release policy. Attestation proves what is running, not who decided it should run.
• Everything your own code hands out. A TEE is not a guardrail. A prompt injection that makes an in-enclave agent exfiltrate data (II.2 · Prompt injection & the LLM attack surface) succeeds exactly as well inside the enclave. Nor does it stop model extraction through the API (I.4 · Adversarial machine learning) - the attacker is querying the model, not reading its memory.
• Side channels. The same paper identifies timing and plaintext RPC metadata (physical address tables, queue headers) as live leakage concerns, and confirms that decapsulation and on-package HBM probing are out of scope “as such attacks are considered highly impractical.”

What breaks it: three interposers, all under $1,000

This is the part vendor material does not lead with, and the part a Big-4 client needs in the risk paper.

• WireTap (Purdue and Georgia Tech, CCS ‘25) built a DDR4 interposer for “under $1000, including the logic analyzer” and recovered an ECDSA signing key from Intel’s Quoting Enclave on 3rd Gen Xeon Scalable. The forged SGX quote passed Intel’s own verification, letting an attacker “masquerade as a real enclave while in fact running outside of SGX protections.”
• Battering RAM (KU Leuven, Birmingham, Durham; IEEE S&P May 2026) used a “less than $50” aliasing interposer for arbitrary plaintext access under Intel Scalable SGX and bypassed AMD SEV-SNP attestation by replaying launch measurements. No vendor verification library is claimed for this one. Disclosed Feb 2025, advisories Sept 2025; “physical attacks on DRAM are out of scope.”
• TEE.fail (Georgia Tech and Purdue, IEEE S&P ‘26) moved to DDR5 for under $1,000, extracting ECDSA attestation keys “in a fully automated manner from a single signing operation.” The forged TDX quote “can be successfully verified using Intel’s DCAP Quote Verification Library at the highest trust level of UpToDate.” It also extracted keys under SEV-SNP with Ciphertext Hiding. On NVIDIA, the claim is narrower than it is often reported: the researchers can “borrow” NVIDIA attestations from devices that are not theirs, convincing a user that a workload is in a GPU TEE when it is running unprotected. GPU memory was not decrypted. Both CPU vendors again treat interposers as out of scope and tell customers to “separately ensure the physical security of their servers.”

Say this out loud to the risk committee

The selling proposition is that you no longer have to trust the infrastructure operator. The 2025-2026 interposer results say you still do - just about physical access to the DIMM slots rather than root on the hypervisor. That is a genuine and large reduction in trust surface, and a different claim from the one on the marketing page. Write it as: strong control against remote and software-level operator access; unproven against an adversary with physical access to the rack, by explicit vendor design.

The operational tax

• Performance is no longer the objection. NVIDIA’s July 2026 benchmark on an HGX B300 serving a 397B-parameter FP8 model reports a band across concurrency levels 4 to 256: throughput and time-per-output-token from -2.0% / -1.6% at the light end to -7.5% / -8.1% at the worst, and under 4% with longer inputs. Treat it as a vendor benchmark and re-measure on your own shape, but “it is too slow” is stale.
• Capacity and topology are the objection. Azure’s confidential GPU SKU gives one H100 NVL per VM and does not support live migration, so host maintenance is an outage.
• You inherit a firmware and driver treadmill. Attestation compares measurements against a reference integrity manifest, so every driver, firmware and image update is a policy change. Plan that process before the first deployment, not after the first failed key release.
• Debugging gets harder on purpose. Performance counters are disabled to close side channels. Build observability inside the boundary (VII.3).

Worth it, or theater

Situation	Verdict (this book’s judgment)
Frontier or high-value proprietary weights on rented GPUs	Worth it. The case the technology was built for
Multi-party computation where parties will not show each other data	Worth it. Confidential Space style deployments exist for exactly this
Regulated inference through a provider you cannot fully audit	Worth it, with the caveat above. Pair with an independent verifier
Your own hardware in your own facility	Usually theater. Spend the budget on egress control and least privilege instead
Commodity open-weight model anyone can download	Theater. Nothing is protected. Protect the prompts if that is the real concern
Sold as a defense against prompt injection or model extraction	Theater, and worth correcting. Wrong layer entirely (II.5)

The regulatory hook

EU AI Act Article 55(1)(d) requires providers of GPAI models with systemic risk to “ensure an adequate level of cybersecurity protection for the general-purpose AI model with systemic risk and the physical infrastructure of the model.” The GPAI Code of Practice (published 10 July 2025) is the compliance vehicle and has a Safety and Security chapter. UNVERIFIED: this book has not read that chapter’s text, so it does not assert what the chapter says about weights specifically, nor the widely repeated account that an earlier draft set RAND SL3 as a floor before the final made it a reference point. Verify against the published chapter before citing a level to a client. See VIII.5 · Jurisdictions.

For the organization

• Decide the question first: are you protecting the weights or the prompts and outputs? They lead to different designs.
• Make the key the control. Weights ship encrypted; the key is released only against a policy naming both the CPU TEE type and the GPU state. Fail closed.
• Use a verifier you do not have to trust your provider about, and treat the reference integrity manifest and release policy as change-controlled artifacts with named owners.
• Write the residual risk down honestly: physical access to the rack is out of scope by vendor design, and two published interposers have forged quotes that passed vendor verification libraries while a third bypassed SEV-SNP attestation by replay.
• Do not let a TEE displace the controls that stop the incidents you are likely to have - artifact provenance (I.5 · The model artifact & its supply chain), least privilege on the weight store, egress control, tamper-evident logging.

Maps to

EU AI Act Article 55(1)(d) and the GPAI Code of Practice Safety and Security chapter; RAND “Securing AI Model Weights” SL1-SL5 as the ladder this sits on (cited from the framework’s common usage; this book has not re-read the report); Google SAIF infrastructure risks; MITRE ATLAS exfiltration of ML artifacts. Cross-refs: I.5 · The model artifact & its supply chain, I.7 · Frontier capability & the if-then frameworks, II.5 · Guardrails - what holds, and how to prove it, VII.5 · Retirement & decommissioning.