How to evaluate a consultant's NCUA and examiner depth

Every AI consultant who wants your business will tell you they understand regulated financial institutions. Most of them mean they once did a project for a bank. That is not the same thing as knowing what an NCUA examiner will ask about the system they are proposing to build for you. The difference tends to surface at your next examination, when it is expensive to fix.

You can test regulatory depth before signing, mostly by asking for documents. The request list is below, along with a worked example scored against the rubric on the scorecard page, where regulatory working knowledge carries a 20% weight, tied with credit union depth for the heaviest criterion. A consultant's regulatory blind spot tends to surface during your next examination cycle. By then the project team is gone and your compliance staff owns the cleanup.

Why banking compliance experience does not transfer cleanly

NCUA supervision differs from OCC and Federal Reserve supervision in ways that matter for AI projects. Examiners arrive with different priorities, credit unions carry a cooperative governance structure that changes how model decisions get approved, and NCUA's guidance on third-party relationships puts specific documentation burdens on you, not the vendor. A firm that learned compliance at a national bank will often over-engineer in some places (building model risk apparatus sized for a $50 billion institution) and under-deliver in others (missing the third-party due diligence file your examiner actually requests).

In the interview, ask the consultant to walk through the file they expect your examiner to request first. A firm with real NCUA exposure talks naturally about supervisory priorities, the third-party risk expectations in NCUA guidance, fair lending review under Regulation B for anything touching credit decisions, and the difference between a model your team can explain and one nobody in the building understands. A firm without that exposure retreats to generalities within two follow-up questions.

Documents to request before you sign

Ask every finalist for these four items. Redacted versions are fine. Treat a refusal to produce any of them the way you would treat a bad reference.

A redacted examiner-readiness or governance deliverable from a prior credit union engagement. You want to see the actual work product: a model inventory entry, a documented risk assessment, a policy the client's board adopted. Judge whether the document would survive your own examiner's review. A thin deliverable reads like a reformatted vendor whitepaper. You want to see the client's specific systems, data flows, and approval chain in there.

A sample third-party risk documentation package. If the firm recommends or implements any vendor tool, you carry the third-party risk obligation. Ask what documentation package they hand you for it: due diligence summary, data flow description, contract terms review, ongoing monitoring plan. A firm that says "your compliance team handles that" is planning to leave you with a liability and no file.

Their fair lending position for any decisioning use case. If the proposal touches lending, collections, or membership decisions in any way, ask for a written description of how they test for disparate impact and what they document. Accept no hand-waving here. "The model doesn't use protected attributes" is not a fair lending analysis; proxy variables are the entire problem.

References from credit unions that have been examined since the engagement ended. This is the single most valuable request on the list. A reference who can say "the examiner reviewed the AI system and the documentation held up" is worth more than any credential. Ask the reference what the examiner requested and whether anything was missing.

Red flags

• The firm's regulatory expertise lives in a partner who "reviews all deliverables" but will not be staffed on your project. Ask who attends your working sessions.
• Compliance appears in the proposal as a phase at the end, after the build. Documentation written retroactively reads that way to examiners.
• Regulatory questions get answered with frameworks ("we follow NIST AI RMF") rather than anything NCUA-specific. Frameworks are useful scaffolding, but they do not answer what your examiner requests during a targeted review, and a firm that cannot name a single NCUA letter or supervisory priority without checking has probably never sat with a credit union compliance officer.
• The proposal promises the system will be "fully compliant." No serious firm promises that; whether you operate the system compliantly is outside their control.

Green flags

• They ask about your last exam early, unprompted, including any DOR items or findings, because it changes what they build first.
• The statement of work names a model inventory entry, a decision log, and documentation mapped to what an examiner requests, instead of a single line reading "governance framework."
• They distinguish between use cases by risk tier and say plainly that a back-office document automation tool and a credit decisioning model deserve different levels of scrutiny.
• They can describe a time an examiner or a client's compliance team pushed back on their work, and what changed as a result.
• They volunteer the limits of their role: they prepare the documentation and train your team to defend it, but your people answer the examiner's questions.

A worked example

What follows is a composite, assembled from real evaluations with identifying details changed. A $600 million credit union in the Midwest evaluated two firms for a lending back-office automation project. Firm A, a national technology consultancy, presented a polished AI governance framework and a compliance workstream priced at $40,000 inside a larger proposal. Firm B, a smaller specialist, presented a plainer deck but opened the second meeting by asking about the credit union's 2024 exam and the documented findings on vendor management.

The evaluation team ran the document test. Firm A produced a governance whitepaper with the client name removed; it contained no reference to any specific system and could have applied to a hospital. Firm B produced a redacted model inventory entry and a third-party due diligence file, then connected the team with a reference client examined eight months after go-live. The reference reported the examiner requested the model documentation, reviewed it in an afternoon, and moved on.

On the rubric, Firm A scored 2 on regulatory working knowledge (generic framework, no examiner-tested artifacts) and Firm B scored 5 (NCUA-specific deliverables, referenceable exam outcome). At a 20% weight, that three-point spread moved the weighted total by more than half a point on a five-point scale, enough to flip the decision despite Firm A's stronger brand. The decision was not unanimous. Two evaluators initially scored Firm A higher on delivery capacity, and the argument only settled when the documents were laid side by side. The credit union hired Firm B and recorded the comparison in board minutes, which matters when someone later asks why you passed on the bigger name.

Where this fits in the full evaluation

Regulatory depth is one criterion of seven. Firms occasionally ace it and still fail on capability transfer or vendor independence. Run the full rubric and the twelve questions on the interview questions page, especially question three, which asks the firm to predict what an examiner would request about their proposed system. Strong firms answer it in under two minutes with specifics.

If you want to see what examiner-ready scoping looks like in a bounded first engagement, review the structure of an AI readiness assessment and hold every firm you evaluate, including its publisher, to the same documentary standard: a redacted inventory entry, a risk assessment, and a reference from a client examined after go-live.