How to evaluate a consultant's NCUA and examiner depth

Every AI consultant who wants your business will tell you they understand regulated financial institutions. Most of them mean they once did a project for a bank. That is not the same thing as knowing what an NCUA examiner will ask about the system they are proposing to build for you, and the gap between those two positions is where credit unions get hurt.

This deep-dive gives you a concrete way to test regulatory depth before you sign: the documents to request, the questions that separate real experience from a slide about "responsible AI," and a worked example scored against the rubric on the scorecard page. Regulatory working knowledge carries a suggested weight of 20% in that rubric, tied with credit union depth for the heaviest criterion, and for good reason. A consultant's regulatory blind spot becomes your examination finding roughly twelve to eighteen months after they cash the final invoice.

Why banking compliance experience does not transfer cleanly

NCUA supervision differs from OCC and Federal Reserve supervision in ways that matter for AI projects. Examiners arrive with different priorities, credit unions carry a cooperative governance structure that changes how model decisions get approved, and NCUA's guidance on third-party relationships puts specific documentation burdens on you, not the vendor. A firm that learned compliance at a national bank will often over-engineer in some places (building model risk apparatus sized for a $50 billion institution) and under-deliver in others (missing the third-party due diligence file your examiner actually requests).

Test vocabulary under pressure. A firm with real NCUA exposure talks naturally about supervisory priorities, the third-party risk expectations in NCUA guidance, fair lending review under Regulation B for anything touching credit decisions, and the difference between a model your team can explain and one nobody in the building understands. A firm without that exposure retreats to generalities within two follow-up questions.

Documents to request before you sign

Ask every finalist for these four items. Redacted versions are fine; refusal to produce any of them is an answer in itself.

A redacted examiner-readiness or governance deliverable from a prior credit union engagement. You want to see the actual work product: a model inventory entry, a documented risk assessment, a policy the client's board adopted. Judge whether the document would survive your own examiner's review. Thin deliverables look like reformatted vendor whitepapers. Strong ones reference the client's specific systems, data flows, and approval chain.

A sample third-party risk documentation package. If the firm recommends or implements any vendor tool, you carry the third-party risk obligation. Ask what documentation package they hand you for it: due diligence summary, data flow description, contract terms review, ongoing monitoring plan. A firm that says "your compliance team handles that" is planning to leave you with a liability and no file.

Their fair lending position for any decisioning use case. If the proposal touches lending, collections, or membership decisions in any way, ask for a written description of how they test for disparate impact and what they document. Accept no hand-waving here. "The model doesn't use protected attributes" is not a fair lending analysis; proxy variables are the entire problem.

References from credit unions that have been examined since the engagement ended. This is the single most valuable request on the list. A reference who can say "the examiner reviewed the AI system and the documentation held up" is worth more than any credential. Ask the reference what the examiner requested and whether anything was missing.

Red flags

• The firm's regulatory expertise lives in a partner who "reviews all deliverables" but will not be staffed on your project. Ask who attends your working sessions.
• Compliance appears in the proposal as a phase at the end, after the build. Documentation written retroactively reads that way to examiners.
• They cannot name a single NCUA letter to credit unions, supervisory priority, or guidance document without checking. You are not hiring them to recite citations, but total blankness signals they have never sat with a credit union compliance officer.
• Every regulatory question gets answered with "we follow NIST AI RMF." Frameworks are useful scaffolding, but NIST does not answer what your examiner requests during a targeted review.
• The proposal promises the system will be "fully compliant." No serious firm promises that, because compliance is a property of your program, not their code.

Green flags

• They ask about your last exam early, unprompted, including any DOR items or findings, because it changes what they build first.
• The statement of work names a model inventory entry, a decision log, and documentation mapped to what an examiner requests, instead of a single line reading "governance framework."
• They distinguish between use cases by risk tier and say plainly that a back-office document automation tool and a credit decisioning model deserve different levels of scrutiny.
• They can describe a time an examiner or a client's compliance team pushed back on their work, and what changed as a result.
• They volunteer the limits of their role: they prepare the documentation and train your team to defend it, but your people answer the examiner's questions.

A worked example

A $600 million credit union in the Midwest evaluated two firms for a lending back-office automation project. Firm A, a national technology consultancy, presented a polished AI governance framework and a compliance workstream priced at $40,000 inside a larger proposal. Firm B, a smaller specialist, presented a plainer deck but opened the second meeting by asking about the credit union's 2024 exam and the documented findings on vendor management.

The evaluation team ran the document test. Firm A produced a governance whitepaper with the client name removed; it contained no reference to any specific system and could have applied to a hospital. Firm B produced a redacted model inventory entry and a third-party due diligence file, then connected the team with a reference client examined eight months after go-live. The reference reported the examiner requested the model documentation, reviewed it in an afternoon, and moved on.

On the rubric, Firm A scored 2 on regulatory working knowledge (generic framework, no examiner-tested artifacts) and Firm B scored 5 (NCUA-specific deliverables, referenceable exam outcome). At a 20% weight, that three-point spread moved the weighted total by more than half a point on a five-point scale, enough to flip the decision despite Firm A's stronger brand. The credit union hired Firm B, and the board minutes recorded the document comparison as the deciding evidence. That paper trail matters when someone later asks why you passed on the bigger name.

Where this fits in the full evaluation

Regulatory depth is one criterion of seven. A firm can know NCUA guidance cold and still fail you on capability transfer or vendor independence, so run the full rubric and the twelve questions on the interview questions page, especially question three, which asks the firm to predict what an examiner would request about their proposed system. Strong firms answer it in under two minutes with specifics.

If you want to see what examiner-ready scoping looks like in a bounded first engagement, review the structure of an AI readiness assessment and hold every firm you evaluate, including its publisher, to the same documentary standard. The test is always the same: ask for the artifact, not the assurance.