The model artifact & its supply chain

The AI supply chain is broader than software’s - data, weights, code, and infrastructure, most of it pulled from public hubs on implicit trust - so the load-bearing question becomes “is this the artifact the author actually built, unmodified?”

Authorized use only

Defensive and educational material, for authorized testing and sanctioned engagements only.

The four links in that chain:

Link	What it covers
Data	Pre-train, fine-tune, RAG (retrieval-augmented generation) corpora - see I.3
Weights	Base models, adapters, quantizations
Code	Frameworks, MLOps, connectors
Infrastructure	Serving, vector stores, GPUs

Most of it is pulled from public hubs with implicit trust.

Model files are code

A pickled checkpoint executes arbitrary code on load - downloading a model is running a stranger’s program (I.4 · Adversarial machine learning). Safer formats (safetensors) and scanners help - modelscan (Protect AI), fickling and picklescan flag unsafe pickle opcodes and embedded code before load - but unsafe deserialization remains a top hub risk, and weights themselves can be backdoored (Sleeper Agents, II.2 · Prompt injection & the LLM attack surface) which no format check detects. Because a model file can carry code, “is this the model the author actually built, unmodified?” becomes a load-bearing question - the provenance problem the software world solved for packages, now applied to weights and datasets.

Registry, dependency & MLOps risk

Scan & verify a model artifact before load (authorized testing only)

# 1) SCAN for unsafe deserialization / embedded code in weights
#    modelscan (Protect AI) - flags os.system, exec, GLOBAL opcodes, etc.
pip install modelscan
modelscan -p ./<org>__<model>/pytorch_model.bin
modelscan -p ./<org>__<model>/ -r json -o modelscan-report.json

#    fickling - decompile a pickle to inspect its opcodes / detect payloads
pip install fickling
fickling --check-safety ./<model>.pkl
fickling ./<model>.pkl        # dump the pickled program to read it

# 2) PREFER safetensors; if a .bin/.pt ships instead, treat as suspicious
python -c "from safetensors.torch import load_file; load_file('model.safetensors')"

# 3) VERIFY provenance (OpenSSF Model Signing, Sigstore-backed)
pip install model-signing
model_signing verify sigstore \
  --signature ./<model>/model.sig \
  --identity <builder@org.com> \
  --identity-provider https://accounts.google.com \
  ./<model>/

# Typosquat / dependency-confusion note: publishing a squat of huggingface_hub
# (e.g. huggingface-hubs) whose postinstall runs code, or a backdoored fork
# <org>/llama-3-8b-instruct-v2, is the same class - pin hashes, mirror internally.

Typosquatting and slopsquatting (LLMs hallucinate plausible package names attackers register) hit AI projects hard. MLOps infrastructure - experiment trackers, orchestrators, notebook servers - is often internet-exposed and under-hardened.

2026: three ways the package layer beat the provenance layer

The integrity tooling below answers “is this the artifact the author built?” Three 2026 campaigns show the answer can be yes and the artifact still malicious.

• The scanner is a privileged CI dependency. The LiteLLM PyPI backdoor (24 Mar 2026, TeamPCP) arrived through Trivy: an actor abused a pull_request_target workflow to steal a maintainer token, then force-pushed 76 of 77 aquasecurity/trivy-action tags - and Trivy ran inside LiteLLM’s own CI. The payload took environment variables, SSH keys, cloud credentials, Kubernetes tokens and CI secrets; version 1.82.8 shipped a litellm_init.pth that executes on every Python interpreter start. Docker users on pinned dependencies were unaffected. Pin scanner action tags to immutable SHAs like any other dependency, and remember that an LLM gateway host is the single richest credential target in an AI deployment because every provider key transits it.
• Provenance attests the wrong thing. The Mini Shai-Hulud worm (11-14 May 2026) chained pull_request_target abuse, GitHub Actions cache poisoning across the fork-to-base boundary, and runtime extraction of an OIDC token - then self-propagated by minting npm OIDC tokens to republish under stolen maintainer identities. Attestation confirmed the malicious artifacts. If your control is “we only install packages with verified provenance,” this is the counterexample: signing proves who built it, not that they meant to. It also spoofed commits as claude@users.noreply.github.com to impersonate a coding-agent bot - as agent commits become routine, “the bot did it” stops being a trust signal and commit signing for automation stops being optional.
• The compromise is in a dependency you did not diff. The Mastra scope takeover (17 Jun 2026) began with a former contributor whose npm publish rights were never revoked - an offboarding failure, not an exploit. Around 140-145 packages were republished in one window, but the malicious code lived in a new transitive dependency running via postinstall, so diffing the published package’s own source showed nothing. Detection has to cover dependency-graph deltas and install scripts, and the control is mundane: periodically review who holds publish rights on every scope and project you own, with 2FA and trusted publishing enforced.

Infrastructure & deployment

Beneath the model: inference/serving endpoints, vector databases, container/Kubernetes orchestration, cloud configuration. Misconfigurations that look benign turn dangerous once AI workloads sit on them (exposed serving APIs, over-permissive IAM, unsecured vector stores). This is where most real-world breaches actually live, and it maps directly to the CSA advisory (VIII.5 · Jurisdictions - Singapore, the EU, the US & UK).

The findings that recur (your checklist)

Against a real MLOps estate, the same issues surface repeatedly:

• Unsigned / unverifiable model artifacts - no OMS/Sigstore signature to verify against.
• Unsafe-pickle weights loaded in CI - .bin/.pt deserialized with no modelscan gate.
• Floating tags instead of pinned commit hashes - a moving supply-chain target.
• Production pulls direct from public hubs - no internal, scanned mirror.
• Exposed MLflow tracking server - remote unauthenticated file overwrite (CVE-2023-6018) & auth-bypass account creation (CVE-2023-6014) and deserialization RCE (CVE-2024-37052 and the related CVE-2024-37053 through CVE-2024-37060).
• Exposed Ray dashboard - “ShadowRay” unauthenticated job-submission RCE (CVE-2023-48022), disputed by the vendor but actively exploited in the wild.
• Kubeflow / Jupyter notebook servers without auth - unauthenticated code execution on the cluster.

Sweep the estate for exposed MLOps services (authorized testing only)

nuclei -tags mlflow,ray,kubeflow -u https://<target-host>

Provenance is the throughline

Know what you run and prove it hasn’t changed: sign/verify weights and datasets, maintain an AIBOM, scan model files before load, pin versions, gate promotion on integrity and behavioral evals. A core CoSAI software-supply-chain workstream and the SAIF supply-chain element.

Cross-reference

Backing for the supply-chain and infrastructure offensive techniques in VI.4 · AI red-team playbook. Maps to OWASP LLM03 and Agentic ASI04.

Integrity for the model artifact: signing, ML-BOM & provenance

That provenance question - “is this the model the author actually built, unmodified?” - is what the integrity tooling answers, and it matured quickly across 2025-2026:

• Model signing - the OpenSSF Model Signing (OMS) specification (OpenSSF, introduced June 2025), building on the model-signing library/CLI that reached v1.0 in April 2025 (Google’s open-source security team with NVIDIA and HiddenLayer), built on Sigstore: keyless, identity-based signatures logged in a public transparency log (Rekor), with a detached bundle binding a model to its author and a manifest of file hashes. It is integrated into NVIDIA NGC and Google Kaggle.
• Build & provenance levels - SLSA (“salsa”) gives a graded checklist for tamper-resistant build pipelines and verifiable provenance; Sigstore/Cosign supplies the signing and verification primitives.
• Bill of materials - a ML-BOM (machine-learning bill of materials) enumerates the model, its datasets, and dependencies; CycloneDX (OWASP; v1.7, Oct 2025) has carried ML-BOM since v1.5, and OWASP’s SCVS guides component verification.
• Documentation as metadata - Model Cards (Mitchell et al., 2019) record intended use, training data, and evaluation; the Coalition for Secure AI (CoSAI) is driving this toward tamper-proof, machine-readable metadata signed alongside the weights.

Advisory framing

The message for a client is simple: provenance of data and weights is now as load-bearing as provenance of code. “We only use models from reputable hubs” is not provenance - a signature you verify, an ML-BOM (CycloneDX’s name for what this book calls an AIBOM) you can diff, and a build you can attest are. Make signed-and-verified a pipeline gate (II.5 · Guardrails - what holds, and how to prove it), not a manual afterthought.

Concretely, the gate is one fail-closed CI job - promote nothing unless every step exits clean:

CI promotion gate: nothing promotes unless every step passes (fail-closed)

set -euo pipefail  # any non-zero step aborts promotion
M=./<org>__<model>

modelscan -p "$M/" -r json -o modelscan-report.json  # 1) unsafe-pickle / embedded-code scan
model_signing verify sigstore \
  --signature "$M/model.sig" --identity <builder@org.com> \
  --identity-provider https://accounts.google.com "$M/"  # 2) provenance (OMS/Sigstore)
test -f "$M/mlbom.cdx.json" || { echo "no ML-BOM - refusing"; exit 1; }  # 3) diffable AIBOM must ship
score=$(python eval_suite.py --model "$M" --report eval.json)  # 4) run behavioral evals
awk "BEGIN{exit !($score>=0.90)}" || { echo "eval $score < 0.90"; exit 1; }  # gate on the threshold
./mirror-to-internal-registry.sh "$M" <model>:$(git rev-parse HEAD)  # 5) promote: mirror by commit hash

For the organization

• Require verifiable provenance - signing (OMS/Sigstore) and a diffable ML-BOM - on every model and dataset before it enters the estate; “from a reputable hub” is not provenance.
• Pin to commit hashes, not floating tags; an unpinned dependency is a moving supply-chain target.
• No production pulls from public hubs; mirror approved artifacts into an internal, scanned registry.
• Scan model files in CI for unsafe deserialization and gate promotion on integrity plus behavioral evals - not a manual afterthought.

Maps to

OWASP LLM03 (supply chain); OWASP Agentic ASI04 (agentic supply chain); MITRE ATLAS: ML supply chain compromise, model file execution (arbitrary code on deserialization). Cross-refs: I.4 · Adversarial machine learning serialization/pickle RCE, II.5 · Guardrails - what holds, and how to prove it pipeline gating.