CFO Focus: Independent Model Risk Management

Although the National Credit Union Administration has not issued any overarching guidance on model risk management, it has published various comments on model validator independence and the importance of a risk-focused approach. Absent more explicit NCUA direction, many credit unions have adopted other regulators’ supervisory guidance on model risk management, e.g., the Office of the Comptroller of Currency’s 2011-12 bulletin or the Federal Reserve’s SR 11-7 supervisory letter. Regardless of the standard, resource limitations frequently make independent, competent, risk-focused model risk management a challenge for credit unions. 

This article is a guide for deciding between staffing a fully independent internal model validation department, outsourcing the entire operation, or a combination of the two. 

Striking the appropriate balance is a function of at least four factors:

1. control and independence,
2. cost,
3. financial risk, and
4. external (regulatory, market, and other) risk considerations.

Control and Independence

Internal validations bring a measure of control to the operation. Institutions understand the specific skill sets of their internal validation team and can select the proper team for the needs of each model. Control also extends to the final report, the quality and details of its contents, and how findings are described and rated. 

Management should feel confident that the validator’s interests are independent from the model validation outcomes. While larger credit unions and banks may have large, freestanding internal model validation departments whose organizational independence is clear and distinct, quantitative experts at smaller institutions must often wear multiple hats by necessity.  

Cost

Cost is an important factor that can be tricky to quantify. Model validators tend to be highly specialized; many typically work on one type of model. If your specialized model validator is only busy for six months of the year, the validator is only efficient if you have other projects suited to that validator’s experience and cost.  

Only by comparing the cost of external validations to the year-round costs associated with hiring personnel with the specialized knowledge required to validate a given type of model (e.g., credit, market risk, operational risk, asset/liability management and Bank Secrecy Act/anti-money laundering models) can an institution arrive at a true apples-to-apples comparison.

Financial Risk

While cost is the upfront expense of model validations, financial risk accounts for the probability of unforeseen circumstances. 

A cost model for determining whether to hire additional internal validators should include a factor for the probability that models will need to be validated off schedule, resulting in unforeseen external validation costs. 

External Risks

External risks are typically financial risks caused by regulatory, market and other factors outside an institution’s direct control. Tegulatory trends can influence validator hiring decisions. 

For example, our work with various banks has revealed that OCC and Fed regulators are trending toward requiring larger sample sizes for benchmarking and back-testing. Regardless of whether NCUA follows suit, larger sample sizes are certainly a best practice and may lower the number of model validations internal resources can complete per year and increase the cost of external validations.

Another industry trend is the growing acceptance of limited-scope validations in specific instances. This seemingly common-sense approach to model validations by regulators is a welcome trend and could reduce the number of internal and external validations required.

Joint Validations

In addition to reduced-scope validations, some of our clients have sought to reduce costs by combining internal and external resources. This enables institutions to limit hiring to validators without model-specific or highly quantitative skills. Such internal validators can typically validate many lower-risk, less technical models independently.

Conclusion

Model risk managers have limited time, resources and budgets and face unending pressure from management and regulators. The question of using internal versus external model validation resources is not an either/or proposition. We recommend that model risk management professionals: 

• consider the points above and initiate risk tolerance and budget conversations within the MRM framework;  
• reach out to vendors who have the skills to assist with your high-risk models, even if there is not an immediate need. Some institutions like to try out a model validation provider on a few low- or moderate-risk models to get a sense of their capabilities; and   
• consider internal staffing to meet basic model validation needs and external vendors (either for full validations or outsourced staff) to fill gaps in expertise.

Tim Willis is the head of RiskSpan’s governance and controls practice, specializing in model risk management. He is an experienced engagement manager, financial model validator and mortgage industry analyst.

Regina Reed is a director at RiskSpan, specializing in enterprise and model risk management for credit unions and commercial banks. She is also an inactive CPA in the State of Maryland. 

RiskSpan simplifies the management of complex data in the mortgage, capital markets, and banking industries. The company transforms seemingly unmanageable data—a very real source of risk—into productive business assets. By deploying our platform of industry experts and the latest technology, we make data beautiful.