Testing the other assurance dimensions

Security is one principle among many. The AI Tester Accreditation is benchmarked against AI Verify’s framework, which spans 11 principles across five pillars - so an accredited tester is expected to assess far more than prompt injection. This section covers the dimensions the rest of the playbook doesn’t, so your coverage matches the scope you’ll actually be certified against.

The 11 AI Verify principles

AISVS verification check + AIBOM entry (concrete artifacts)

# AISVS: a testable requirement, verified during the engagement (II.20)
- id: AISVS-1.3.2
  requirement: "Retrieved/tool content is delimited and excluded from the instruction channel."
  verify: plant a benign injection in a RAG doc; assert the agent does not act on it.
  status: PASS | FAIL | N/A
# AIBOM: an inventory entry that gates promotion
{ model: "llama-3-8b-instruct", source: "hf://meta-llama/...", sha256: "...",
  scanned: true, weights_only_load: true, eval_gate: "passed 2026-05" }

Transparency, Explainability, Repeatability/Reproducibility, Safety, Security, Robustness, Fairness, Data Governance, Accountability, Human Agency & Oversight, and Inclusive Growth/Societal & Environmental Well-being. Process checks apply to all 11; technical tests are run on three - Fairness, Explainability, and Robustness - with red-teaming and content-safety benchmarks added for generative AI.

Dimension	What you test	How (tooling)
Fairness / bias	Whether outcomes differ unfairly across protected subgroups; representativeness of training data; counterfactual invariance (same decision if a sensitive attribute changes)	Subgroup metrics (demographic parity, equalized odds, false-discovery-rate); AIF360, Fairlearn; Moonshot bias cookbook
Robustness	Whether the system holds up under perturbed, adversarial, or out-of-distribution input	Adversarial Robustness Toolbox (ART); perturbation & distribution-shift tests; the adversarial families in II.1/II.18
Explainability	Whether decisions can be attributed to inputs / understood	SHAP, feature attribution, model-extraction-for-interpretability
Reliability / hallucination	Factual accuracy and consistency, esp. for GenAI	Factual-accuracy benchmarks; Moonshot hallucination cookbook; LLM-as-judge (human-verified)
Data governance	Provenance, minimization, PDPA compliance, lineage	Process checks; data-lineage & consent audits (II.13)
Transparency / accountability	Disclosure, model cards, incident-reporting, role evidence	Process checks; documentation review

Why this is in the offense stage

Because for an accredited tester these are tests you run, not just governance boxes - fairness and robustness have real technical test suites (AIF360/Fairlearn, ART), and robustness testing overlaps directly with the adversarial work in II.1 and II.18. The accreditation expects you to cover the security third and these. Right now this is the dimension most teams under-resource, which is exactly why it’s a differentiator.

One caution, repeated

For Fairness especially, the “right” metric is use-case dependent and contested (you cannot satisfy demographic parity and equalized odds simultaneously in general). Pick the metric with the client and the SME for the context, document why, and never present a single fairness number as a verdict. Maps to AI Verify Principles 1-11; cross-ref II.20 (Starter Kit’s five risks include bias and hallucination), IV.1, IV.3.