Model Inventory

A regulator asks how many AI models your institution has in production. The CRO says twelve. The CTO says forty. The data science team says somewhere between twenty and sixty, depending on how you define "model." If your institution cannot answer this question with confidence, you do not have a model inventory, and you cannot govern what you cannot see.

What is a model inventory?

A model inventory is a centralised register of every model deployed within an institution, including its purpose, owner, validation status, data sources, performance metrics, and risk classification. In financial services, models include statistical models, machine learning models, actuarial models, pricing models, and increasingly, large language model applications. The inventory is the foundation of model risk management: without it, governance, monitoring, and validation cannot function systematically.

The PRA's SS1/23 requires banks to maintain a comprehensive inventory of all models in use. "In use" includes models in production, models under development, and models recently retired. Each entry must record the model's purpose, its materiality tier, its validation status, its last review date, and its owner. The inventory is not a static document. It is a living register that must be updated as models are deployed, modified, or retired.

The practical challenge is that models proliferate faster than governance can track them. A data scientist builds a prototype in a notebook. It works well enough that someone deploys it to a production server. Nobody registers it in the inventory. Over time, the institution accumulates "shadow models" that operate in production without governance, validation, or monitoring. Every institution that has conducted a model discovery exercise has found more models than it expected.

The landscape

SS1/23 is explicit: the model inventory must be comprehensive. It must include all models, not just the ones the model risk management team knows about. Supervisory reviews increasingly test this comprehensiveness by asking about specific use cases and checking whether the supporting model appears in the inventory. Gaps attract scrutiny.

The EU AI Act's high-risk classification framework adds a new dimension. Institutions must identify which AI systems fall within the high-risk category and ensure they meet enhanced requirements. The model inventory must capture not just what the model does, but its risk classification under both SS1/23 materiality tiers and EU AI Act risk categories.

LLM applications complicate the inventory. A traditional ML model is a defined artefact: a trained model file with known inputs and outputs. An LLM application may be a prompt template, a retrieval-augmented generation pipeline, or a fine-tuned model. Defining what constitutes a "model" for inventory purposes when the boundary between configuration and model is blurred requires clear policy decisions that most institutions have not yet made.

How AI changes this

Automated model discovery scans production environments to identify deployed models that may not be registered in the inventory. By monitoring API endpoints, container registries, and inference logs, discovery tools identify applications that exhibit model-like behaviour (receiving inputs and producing predictions or classifications) and flag them for governance review. This catches shadow models that manual processes miss.

Metadata extraction automates the population of inventory fields. Rather than relying on model developers to manually document their model's characteristics, automated tools extract information from code repositories, experiment tracking systems, and deployment configurations. Training data sources, feature lists, hyperparameters, and deployment infrastructure are captured automatically, reducing the documentation burden and improving completeness.

Risk-based tiering uses the inventory metadata to classify models by materiality. A model that makes credit decisions affecting millions of customers is higher risk than a model that optimises internal email routing. Automated tiering ensures that governance effort is proportional to risk, directing intensive validation and monitoring to the models that matter most while applying lighter-touch governance to low-risk applications.

What to know before you start

Define what counts as a "model" before building the inventory. If you include every spreadsheet with a formula, the inventory will be unmanageable. If you exclude LLM applications because they are "just prompts," you will miss a growing category of AI risk. A practical definition covers any system that produces outputs used to inform or automate decisions, where those outputs depend on learned patterns or statistical relationships. Publish the definition and apply it consistently.

Run a discovery exercise before declaring the inventory complete. Ask each business unit, technology team, and data science team to declare any models they use. Cross-reference against deployment infrastructure. The discovery will surface models that nobody thought to include: a pricing spreadsheet that uses regression, a vendor's API that includes an ML model, a prototype that became production without anyone noticing.

Assign an owner to every model in the inventory. The owner is accountable for the model's performance, governance, and lifecycle decisions. Without ownership, the inventory is a list. With ownership, it is a governance structure. The owner should be a business stakeholder who understands the model's purpose, not a technologist who understands its architecture. Both perspectives are needed, but business accountability drives action.

Start with the inventory itself, not with the tooling. A well-maintained spreadsheet is better than an empty model registry platform. Capture the essential fields: model name, purpose, owner, risk tier, validation status, last review date, and data sources. Build the MLOps automation around the inventory once the inventory exists and the governance process is established.