Can India govern AI it does not own? RBI's model risk framework and its limits
The recently released Guidance on Regulatory Principles for Model Risk Management by RBI is a concrete step towards AI governance in the banking & finance sector. The guidelines set out how entities should govern not only AI models, but any predictive system used for decision making. The guidelines put accountability squarely on businesses through an entity specific model risk management framework (MRMF) that would hold businesses accountable even for 3rd party models.
While such guidance was overdue, lack of sovereign models could limit the control that India exerts on its tech stack, despite the stringency that RBI is pushing for. The release of the guidelines therefore poses a bigger question: Can India truly govern the AI models that it does not own?
RBI’s guidelines are centred around MRMF, which are board-approved framework applicable to all models, whether built in house or not. The MRMF has a wide scope, including model taxonomy, governance, usage scope, risk tiering and critical lifecycle activities.
Also read | Beijing’s warning reveals real stakes of Modi–Takaichi summit
Responsibility is divided across three levels: The entity’s board, a specially created risk management committee and the senior management. Furthermore, the core machinery of the governance attaches obligations at each stage of the model lifecycle, starting from deployment and ending at model decommissioning.
What stands out in the guidance is the accountability laid on users relying on third party models. Financial entities must not only independently validate model accuracy, bias, suitability and risks, but also obtain technical documentation on the model's design, along with audit rights and exit arrangements.
The motivations behind these requirements are straightforward. Generative AI, or more generally neural networks, are notorious for lacking explainability, making technical documentation necessary to understand the model's functioning and limitations.
Similarly, audit rights allow users to assess whether the model was developed within a sound governance framework, while exit arrangements safeguard against sudden loss of model access.
Despite being principally ideal, these requirements could be too ambitious for India, not because they are impractical but because of the broader ecosystem context.
Also read | Ethanol 'experiment': Govt moves to contain fallout
Consider, for instance, the requirement for minimum technical documentation, which expects vendors to disclose model performance benchmarks, known use cases, limitations and other relevant information.
Yet, the model's design will never be fully within the user's control. Model weights, training data and other proprietary specifications remain with the vendor due to their intellectual and commercial value.
Even if some of this information is disclosed, model opacity cannot be entirely eliminated because contemporary AI models remain fundamentally black boxes.
Regulators, including RBI, are aware of these limitations, making empirical validation a practical alternative. Rather than understanding every internal mechanism of the model, its usability, accuracy and safety are assessed across a sufficiently large number of instances.
If the model performs reliably, it can be deployed despite its limited explainability. Such trade-offs are necessary because if only fully interpretable models were permitted, every GenAI model would fail to qualify.
While empirical validation may reasonably overcome the lack of explainability, the problems associated with such opacity reemerge in another way. India is yet to train its own sovereign models and remains entirely dependent on foreign sourced models.
Trained on foreign data and for foreign contexts, these models inevitably carry many of the concerns that the guidance seeks to eliminate.
Biases in data, the absence of India specific context, unknown cybersecurity risks and limited applicability to Indian settings are challenges that persist and remain difficult to identify because of model opacity.
Although frontier models achieve state of the art accuracy, sectors such as banking, finance and medicine have very little tolerance for even minor errors. If a model makes an edge case prediction without sufficient Indian context, its suitability for deployment becomes questionable.
While one may argue that the lack of context can be overcome through model fine tuning, such solutions must be viewed in the right context. RBI’s guidelines will sweep across the entire financial sector, meaning every entity must comply irrespective of capacity and scale.
While compute shortage makes fine tuning difficult in India, it is even tougher for smaller entities. Not only will these players face rising costs of fine-tuning models, but they might also receive limited technical support from AI vendors who could prioritise larger clients.
Other requirements, such as audit rights are similarly constrained by the context of our AI ecosystem. Negotiating audit rights with foreign vendors is difficult, and even if such audits are possible, they are limited to the vendor's governance environment and do not reveal the internal mechanics of the models.
Moreover, audits of foreign entities have an inherent limitation: auditors must ultimately accept some claims at face value, as direct inspection is not always possible.
As far as exit plans are concerned, the situation is equally challenging. USA's ban on Anthropic Fable 5 and OpenAI’s “limited preview of” next generation frontier model GPT 5.6 have renewed concerns about sudden supply chain disruptions, and RBI's requirement for exit plans appears to reflect this risk.
But for an exit plan to be effective, the availability of substitutes is non-negotiable. In the absence of domestic alternatives, companies could be left paralysed if vendors begin to leverage their market power. The concentration of frontier AI models in the USA and China only worsens this dependence.
If required by its strategic interests, the USA can single handedly restrict access to the most widely used AI models in India.
While such guidance was overdue, lack of sovereign models could limit the control that India exerts on its tech stack, despite the stringency that RBI is pushing for. The release of the guidelines therefore poses a bigger question: Can India truly govern the AI models that it does not own?
RBI’s guidelines are centred around MRMF, which are board-approved framework applicable to all models, whether built in house or not. The MRMF has a wide scope, including model taxonomy, governance, usage scope, risk tiering and critical lifecycle activities.
Also read | Beijing’s warning reveals real stakes of Modi–Takaichi summit
Responsibility is divided across three levels: The entity’s board, a specially created risk management committee and the senior management. Furthermore, the core machinery of the governance attaches obligations at each stage of the model lifecycle, starting from deployment and ending at model decommissioning.
What stands out in the guidance is the accountability laid on users relying on third party models. Financial entities must not only independently validate model accuracy, bias, suitability and risks, but also obtain technical documentation on the model's design, along with audit rights and exit arrangements.
The motivations behind these requirements are straightforward. Generative AI, or more generally neural networks, are notorious for lacking explainability, making technical documentation necessary to understand the model's functioning and limitations.
Similarly, audit rights allow users to assess whether the model was developed within a sound governance framework, while exit arrangements safeguard against sudden loss of model access.
Despite being principally ideal, these requirements could be too ambitious for India, not because they are impractical but because of the broader ecosystem context.
Also read | Ethanol 'experiment': Govt moves to contain fallout
Consider, for instance, the requirement for minimum technical documentation, which expects vendors to disclose model performance benchmarks, known use cases, limitations and other relevant information.
Yet, the model's design will never be fully within the user's control. Model weights, training data and other proprietary specifications remain with the vendor due to their intellectual and commercial value.
Even if some of this information is disclosed, model opacity cannot be entirely eliminated because contemporary AI models remain fundamentally black boxes.
Regulators, including RBI, are aware of these limitations, making empirical validation a practical alternative. Rather than understanding every internal mechanism of the model, its usability, accuracy and safety are assessed across a sufficiently large number of instances.
If the model performs reliably, it can be deployed despite its limited explainability. Such trade-offs are necessary because if only fully interpretable models were permitted, every GenAI model would fail to qualify.
While empirical validation may reasonably overcome the lack of explainability, the problems associated with such opacity reemerge in another way. India is yet to train its own sovereign models and remains entirely dependent on foreign sourced models.
Trained on foreign data and for foreign contexts, these models inevitably carry many of the concerns that the guidance seeks to eliminate.
Biases in data, the absence of India specific context, unknown cybersecurity risks and limited applicability to Indian settings are challenges that persist and remain difficult to identify because of model opacity.
Although frontier models achieve state of the art accuracy, sectors such as banking, finance and medicine have very little tolerance for even minor errors. If a model makes an edge case prediction without sufficient Indian context, its suitability for deployment becomes questionable.
While one may argue that the lack of context can be overcome through model fine tuning, such solutions must be viewed in the right context. RBI’s guidelines will sweep across the entire financial sector, meaning every entity must comply irrespective of capacity and scale.
While compute shortage makes fine tuning difficult in India, it is even tougher for smaller entities. Not only will these players face rising costs of fine-tuning models, but they might also receive limited technical support from AI vendors who could prioritise larger clients.
Other requirements, such as audit rights are similarly constrained by the context of our AI ecosystem. Negotiating audit rights with foreign vendors is difficult, and even if such audits are possible, they are limited to the vendor's governance environment and do not reveal the internal mechanics of the models.
Moreover, audits of foreign entities have an inherent limitation: auditors must ultimately accept some claims at face value, as direct inspection is not always possible.
As far as exit plans are concerned, the situation is equally challenging. USA's ban on Anthropic Fable 5 and OpenAI’s “limited preview of” next generation frontier model GPT 5.6 have renewed concerns about sudden supply chain disruptions, and RBI's requirement for exit plans appears to reflect this risk.
But for an exit plan to be effective, the availability of substitutes is non-negotiable. In the absence of domestic alternatives, companies could be left paralysed if vendors begin to leverage their market power. The concentration of frontier AI models in the USA and China only worsens this dependence.
If required by its strategic interests, the USA can single handedly restrict access to the most widely used AI models in India.
Next Story