This article explains why data provenance is the highest-stakes and lowest-coverage area of AI supply-chain governance, defines the provenance dimensions that must be captured, anchors the practice to current standards, and addresses the structural reality that for most foundation models, full provenance simply is not available.
Why Provenance Has Become the Central Question
Three forces have pushed data provenance from an academic concern to a board-level question.
The first is copyright litigation. Multiple suits across jurisdictions allege that foundation models were trained on copyrighted works without authorisation. Outcomes will turn on what data was used, what licensing was secured, and whether technical mitigations (training-data summaries, opt-out honouring, output-similarity controls) were in place. The European Union (EU) AI Act, accessible at https://artificialintelligenceact.eu/, codifies parts of this expectation: Articles 53(1)(c) and 53(1)(d) require General-Purpose AI (GPAI) providers to maintain copyright opt-out compliance and publish a “sufficiently detailed summary” of training content.
The second is privacy and special-category data. The General Data Protection Regulation (GDPR), the California Consumer Privacy Act (CCPA), the Health Insurance Portability and Accountability Act (HIPAA), and sector-specific data laws all impose obligations on data processors. AI training is data processing. Provenance evidence is the only defensible answer to “where did this personal data go?”
The third is bias and fairness analysis. Bias mitigation requires understanding the populations represented in training data. The U.S. National Institute of Standards and Technology (NIST) AI Risk Management Framework (AI RMF) at https://www.nist.gov/itl/ai-risk-management-framework MEASURE-2.11 explicitly requires fairness analysis grounded in training-data composition. Without provenance, bias claims and counterclaims are equally unfalsifiable.
The Provenance Dimensions
A defensible data-provenance record captures the following dimensions for every dataset that materially influences a model.
1. Source Identity
Who originated the data? Is the source known with sufficient specificity (publisher, organization, individual, sensor)? For aggregated sources (web crawls, social media, licensed corpora), identity is at the dataset-publisher level rather than the per-record level — but this distinction is itself part of the provenance record.
2. Collection Basis
Under what consent, contract, license, scraping policy, or statutory permission was the data collected? For personal data, the GDPR Article 6 lawful basis must be specified. For copyrighted works, the license terms or applicable exception (text-and-data-mining exemption, fair use claim) must be recorded. For employee or customer data, the original collection notice and purpose limitation must be referenced.
3. Processing History
What transformations have been applied? Cleaning, filtering, deduplication, labelling, balancing, augmentation? Each step changes what the dataset represents and therefore what the trained model will exhibit. The Software Package Data Exchange (SPDX) standard at https://spdx.dev/, originally a software-component standard, is increasingly extended to capture dataset transformation chains.
4. Distribution and Custody
Where has the dataset been? Which contractors, sub-processors, fine-tuners, or hosting platforms have held copies? Each custody transfer is an opportunity for re-disclosure obligations and for unintended onward use.
5. Restrictions and Obligations
What use restrictions attach? Field-of-use limits, no-redistribution clauses, attribution requirements, deletion-on-request commitments, and revenue-share obligations all flow downstream and must be honoured by the deployer even when the deployer was not the licensor.
6. Output Linkage
For training data that materially shapes specific outputs, what controls exist to prevent regurgitation, copyright contamination, or personal-data disclosure? This is the operational link between data provenance and output-handling controls.
The Asymmetry Problem for Foundation Models
For most major proprietary foundation models, the deployer cannot obtain full training-data provenance. The provider treats it as trade secret. The Stanford Foundation Model Transparency Index at https://crfm.stanford.edu/fmti/ documents the magnitude of this gap: leading providers score weakly across the data-related indicators even after multiple disclosure cycles.
The mature response to this asymmetry has three components. First, demand the regulatorily required summary: under EU AI Act Article 53(1)(d), GPAI providers must publish training-data summaries; obtain and file these. Second, document the gap explicitly: the AI-BOM entry for the model records “training-data composition undisclosed by provider, summary obtained per Article 53(1)(d)” rather than leaving the field empty. Third, insulate downstream where possible: prefer providers that offer copyright indemnification, that honour opt-out registries, and that provide output-similarity tooling.
For data the deploying organization controls — its own training, fine-tuning, evaluation, and Retrieval-Augmented Generation (RAG) data — there is no asymmetry. Provenance for that data is fully achievable and is the deployer’s responsibility.
Provenance for Retrieval-Augmented Generation
A growing share of enterprise AI systems do not retrain models — they use Retrieval-Augmented Generation, in which models are augmented at inference time with retrieved enterprise content. Provenance for the retrieval corpus is just as important as provenance for training data. Confidential customer records that should never reach the model layer routinely leak into RAG indexes when ingestion pipelines lack provenance controls. The AI-BOM described in Article 6 of this module should treat RAG indexes as first-class components with full provenance attached.
How Cybersecurity Supply-Chain Practice Applies
The U.S. National Institute of Standards and Technology (NIST) Special Publication (SP) 800-161 Revision 1 at https://csrc.nist.gov/pubs/sp/800/161/r1/final treats data feeds as a supply-chain category. The U.S. Cybersecurity and Infrastructure Security Agency (CISA) Software Bill of Materials programme at https://www.cisa.gov/sbom is extending to dataset components. Supply-chain Levels for Software Artifacts (SLSA) at https://slsa.dev/ provides build-attestation patterns that map naturally to dataset-pipeline integrity. The Cloud Security Alliance at https://cloudsecurityalliance.org/ addresses the cloud-storage and cross-region implications. Together these references define the operational disciplines that turn data-provenance commitments into verifiable practice.
The International Organization for Standardization / International Electrotechnical Commission (ISO/IEC) 42001:2023 standard at https://www.iso.org/standard/81230.html includes management-system requirements that assume the organization can answer “where did the training data for this AI system come from?” An organization that cannot answer that question is not in conformance regardless of the rest of its program.
Maturity Indicators
| Maturity | What data provenance looks like |
|---|---|
| Foundational (1) | The organization cannot describe the training-data sources for the AI systems it operates; RAG corpora are populated without provenance controls. |
| Developing (2) | Provenance is captured for selected high-profile systems; gaps are acknowledged but not formally tracked. |
| Defined (3) | All six provenance dimensions are populated for systems above the standard tier; foundation-model provider summaries are filed; unknowns are explicitly recorded. |
| Advanced (4) | Provenance flows automatically from data-pipeline tooling into the AI-BOM; opt-out honouring is verified; output-similarity controls operate continuously. |
| Transformational (5) | The organization contributes to provenance standards (dataset cards, data-sheets, FMTI extensions) and influences supplier disclosure practice. |
Practical Application
A consumer-products company building a customer-service generative-AI assistant should treat data provenance as a precondition for production. For the foundation model layer, it obtains the provider’s EU AI Act Article 53(1)(d) training-data summary, files it in the AI-BOM, and accepts the residual undisclosed-composition risk in writing. For its own RAG corpus, it captures the source system for each ingested document, the consent or legal basis for processing the underlying customer data, the deduplication and redaction pipeline applied, and the access controls on the resulting vector index. For any fine-tuning data, it captures the same dimensions and adds the contractual licensing chain. The output is a provenance pack that a regulator, an auditor, or a litigant could read and follow. That pack is the artefact that converts vague claims of governance into evidence.
The next article (Article 8) examines a particular high-risk provenance failure mode: hidden third-party Application Programming Interface (API) dependencies that introduce upstream AI services into systems the deployer believed were entirely internal.