How to judge a pilot proposal: scope, price, success criteria, and exit terms

Skip the sales deck and read the pilot proposal closely. Scope boundaries, measurement rights, and exit language reveal how the firm expects the engagement to make money. Loose scope in particular is rarely an accident; it is where change orders come from, and enough change orders make a six-week engagement permanent.

This deep-dive covers the four sections that decide whether a pilot proposal deserves a signature: scope, price, success criteria, and exit terms. It assumes you have already run the firm through the seven criteria on the main guide and the twelve questions on the interview page. The proposal is where their answers become contractual, or quietly fail to.

Scope

Four to eight weeks is the right default for a credit union AI pilot, and one problem is the right size. Integration-heavy work can justify ten or twelve weeks; a proposal that needs six months has stopped being a pilot. Under four weeks, there is rarely time to produce evidence anyone would defend to a board.

The scope section should name three things precisely. First, the process being changed, with its current baseline stated at this level of precision (an illustrative example): "indirect lending stipulation review, currently averaging 26 minutes per file across 340 files per week." Second, the system boundary: which data sources the pilot touches, which it explicitly does not, and whether anything member-facing changes (for a first pilot, the right answer is almost always no). Third, a single named owner on your side with real decision authority, because pilots without an internal owner drift into week twelve with nothing shipped.

Watch the scope language for load-bearing vagueness. "Explore opportunities across the lending lifecycle" is not a scope; it is a fishing license. So is any deliverable described as a "roadmap" inside a pilot that was supposed to produce working evidence. A roadmap is a fine deliverable for an assessment engagement. Inside a pilot, it usually means the firm hedged against its own build failing.

Price: sanity checks by engagement type

Pricing tells you whether the firm is right-sized for you, which is criterion six on the scorecard rubric. Reference points from the current market:

• Audit-style and assessment engagements (readiness reviews, use case prioritization, governance gap analysis) commonly run $10,000 - $50,000. These are diagnostic, fixed-scope, and short. Anything above that range for pure assessment work needs a persuasive explanation, usually institution size or unusual data complexity.
• Working pilots that ship something (a document automation workflow, a back-office copilot with your data, a fraud triage assist) typically land between $30,000 and $150,000 depending on integration depth. The number should map visibly to named people and weeks, not to "program management."
• Larger builds and platform work start around $100,000 and climb fast. If a firm proposes this as your first engagement, ask why the evidence step is being skipped. Global firms often cannot price below this floor; that is a fit problem, not a moral failing.

Two sanity checks catch most bad pricing. Divide the price by the stated team weeks and compare the implied weekly rate to what the staffing section actually promises: a $90,000 pilot delivering six person-weeks of a senior engineer plus oversight implies roughly $15,000 per person-week, which is defensible for specialist work and indefensible if the roster turns out to be one junior analyst. Then check fixed versus variable: a pilot should be fixed price. Firms that insist on time and materials for a six-week pilot are pricing their own scoping failure into your invoice.

One more test. Ask what happens to the pilot fee if you proceed to a larger engagement. Firms confident in their pipeline often credit some portion forward. Firms that refuse to discuss it at all are telling you the pilot is the product.

Success criteria: who measures, and how

Many proposals go noncommittal the moment they reach success criteria. "Demonstrate the value of AI for member service" is unfalsifiable, so it will be declared a success no matter what happens. Insist on a number, a way to measure it, and a named source of truth for the measurement.

Examples at the right level of specificity (composites drawn from real engagements, rounded): reduce stipulation review time from 26 minutes to under 10 on at least 80% of files, measured over the final two weeks against LOS timestamps. Achieve extraction accuracy above 95% on a 500 document holdout set your team selects, not the vendor's. Cut manual GL reconciliation exceptions by 40% month over month, per your finance team's existing exception report.

Note who controls measurement in each example: you do. A pilot where the consultant grades their own homework will pass. Also insist on one explicit failure condition in writing, such as "if accuracy on the holdout set is below 90%, the parties agree the approach did not validate." Firms with real confidence accept failure conditions readily. A firm that fights hard against defining failure usually intends to declare victory regardless of results.

Baseline measurement belongs in week one of the pilot, not in a prior paid engagement. If a firm says it cannot commit to success metrics without a $40,000 discovery phase first, that is sometimes legitimate for genuinely messy data, but ask them to name the specific unknown that blocks commitment. "We need to see your data" is fair. "We never commit to outcomes" means you are being sold discovery hours, and you should price and judge the proposal that way.

Exit terms: IP, data, and the right to walk away clean

Exit terms get the least attention during the honeymoon and cause the most damage later. Four items to verify before signing:

Work product ownership. Everything built with your money and your data (prompts, configurations, workflow definitions, evaluation sets, documentation) should be yours or perpetually licensed to you at no additional cost. Firms may retain their pre-existing tools and general methods. The line between those two categories should be written down, not assumed.

Data handling on exit. The proposal should state where your data lives during the pilot, whether any of it touches model training, and certify deletion or return within a stated window after the engagement ends, thirty days is typical. This is also third-party risk documentation your examiner may ask about, so get it in the contract, not in an email.

No hostage dependencies. If the pilot runs on the firm's proprietary platform, ask what happens at exit. Where the value only exists while you rent that platform, count the ongoing license fee as part of the pilot's cost before calling it a success. Portable builds on your tenant or on mainstream cloud services exit clean.

Continuation is optional and re-scoped. The pilot contract should end at the pilot. Auto-renewal clauses, pre-committed phase two language, or discounts contingent on signing the larger engagement before pilot results exist all convert your evidence-gathering exercise into a sales funnel you already paid to enter.

The short version

If you only have time to check six things, check these: who prices it (fixed beats time and materials on a short pilot), who measures success, whether failure is defined in writing, who owns the work product, how your data exits, and whether continuing is genuinely optional. A proposal that also quotes your baseline data back to you correctly and names the actual staff is one you can take seriously.

Scoring it on the rubric

A pilot proposal gives you fresh evidence for at least three scorecard criteria: sequencing philosophy (did they pick a back-office problem with real numbers), right-sized delivery (does the price fit your budget with room to grow), and evidence (are the success criteria measurable by you). Score the written document rather than the meeting where it was presented. The proposal is what you can hold the firm to later.

For a reference point on what a bounded, fixed-scope first engagement looks like when it is designed to be judged this way, see the structure of an AI readiness assessment, then apply every test in this article to it and to every competing proposal with equal force.