Running an Agentic Accounts-Payable Proof of Concept: What to Test, What to Measure and When to Stop
Most AI proofs of concept answer the wrong question. They show that a model can read an invoice, and then everyone in the room asks whether to "roll it out". Nobody wrote down, in advance, what result would mean yes, what would mean no, and what would mean "not yet". The pilot drifts, the enthusiasm fades, and the work joins the long list of initiatives that stalled between pilot and production.
Accounts payable gives you a chance to do this properly. It is where reported agent payments are starting to look like operations rather than demonstrations, and it already has the controls (approvals, supplier records, audit trails) that make a scoped proof of concept measurable. This guide sets out what to test, what to measure and, most importantly, how to agree the stop-or-scale rule before the first invoice is processed.
Why accounts payable is where agent proofs of concept are starting
The evidence for this is recent, and it is worth being precise about what it does and does not show.
In October 2026, Visa, BILL and Billtrust were reported to be running a live pilot of AI agents paying business-to-business invoices through Visa Intelligent Commerce. The reporting frames this as among the first production-style agent payments; that "first" is Visa's own claim, and we are passing it on as such rather than confirming it. Separately, Mastercard opened Early Access to Agent Pay for Machines, with buyer-side and seller-side APIs and a sandbox, explicitly aimed at agents paying for services rather than only consumer shopping.
Finance operations more broadly is also reported as an early home for deployed agents. Nestlé's use of SAP agents is one example in the reporting, and the same coverage points to accounts payable, collections and banking operations as the places large organisations put agents to work first. Ramp, meanwhile, has shipped a card designed for AI agents, which suggests vendors are productising the spending controls that agent payments will need.
Our reading of this is analysis, not reported fact. Accounts payable suits agents because the things that make delegated payments risky elsewhere already exist there: approval chains, a known supplier list, invoices that can be matched to something, and a record of who authorised what. An open-web checkout asks an agent to act among strangers. Accounts payable asks it to act inside a controlled process. That is a much better place to learn.
Two caveats matter. First, the evidence is led by US companies and US pilots; a UK finance team should treat it as a signal of direction, not a template for its own environment. Second, none of it tells you how well an agent will perform on your invoices, with your suppliers, under your approval rules. That is precisely what a proof of concept is for.
Start with the decision, not the demo
A proof of concept is a decision instrument. Its job is to produce enough evidence, quickly enough, for a named person to decide whether to fund the next stage. If it cannot do that, it is a demonstration, and demonstrations are easy to approve and hard to learn from.
Before any build work, write down three things:
- The decision. Who will decide, by when, and what are the options? Typically these are scale, stop, or extend once with a changed hypothesis.
- The evidence they will want. Not "it works", but the specific measures that would change their mind.
- The cost of being wrong. What happens if the agent releases a payment it should not have, and who carries that risk during the test?
This is the discipline that separates a useful proof of concept from an expensive one. It also protects you from the most common failure: a pilot whose success criteria were invented after the results came in.
Write the stop-or-scale rule first
The rule does not need to be elaborate. It needs to exist before the data does. A workable structure has three outcomes.
Scale if the agent meets your own thresholds on accuracy, control and exception handling for long enough, across enough variety of invoices, that the result is unlikely to be a lucky fortnight. Stop if it fails a control threshold, because a control failure in a payments process is not something to tune around. Extend once if the results are promising but a specific hypothesis was untested or the sample was too narrow, with the change written down.
We are deliberately not proposing numerical thresholds here. We have no benchmarks for agent performance in accounts payable that we can evidence, and inventing them would be worse than leaving the cells blank. Your thresholds should come from your own baseline: what your team achieves today, at what cost and with what error rate. If you do not have that baseline, building it is the first piece of proof-of-concept work.
What to test: five hypotheses
A proof of concept should test hypotheses, not features. Each one should be something that could turn out to be false. For an accounts-payable agent, five are a good starting set.
- The agent can read and match real invoices. Not a curated sample, but the messy mix your team actually receives: different formats, missing references, partial deliveries, amended invoices and duplicates. The question is how often it matches correctly and how it behaves when it cannot.
- The agent stays inside its authority. Given a payment limit, an approval route and a supplier list, does it ever propose or release something outside them? One breach is more informative than a hundred clean runs.
- Human reviewers work better with the agent, not just faster. If approvers speed up by trusting the agent blindly, you have created a risk rather than a benefit. The hypothesis worth testing is that reviewers catch the same errors, or more, while spending less time on routine items.
- Exceptions reach the right person. An agent that handles the easy cases but quietly mishandles the hard ones is dangerous. Test whether unusual invoices, supplier changes and disputed amounts are escalated properly and promptly.
- Every decision can be reconstructed. For each invoice, can someone later see what the agent saw, what it decided, which rule applied and who approved? If the audit trail is thin, the process will not survive a real audit, however good the accuracy.
Notice what is missing: any claim that the agent will save a specific amount. Savings belong in the business case, and they should be calculated from what the test actually showed, not promised in advance.
Test with real users, and in parallel
Accounts payable is a team activity, and a proof of concept run only by the technical team will miss what matters. The people who process invoices, the approvers who sign them off, the finance controller who owns the risk and, where relevant, the suppliers who chase payment all have views that a test environment will not surface by itself.
Put real users in front of the agent early, and watch where they hesitate. Hesitation usually points at a missing control, an unclear escalation or an output that cannot be trusted without checking everything again.
It also pays to test more than one approach side by side. One variant might have the agent prepare payments for human release. Another might allow it to release payments up to a low limit, with everything above that routed for approval. Running variants in parallel shows you which operating model your organisation can actually support, and stops the test from collapsing into a yes-or-no verdict on a single design.
This is the way Innovify AI Labs is set up: enterprise teams run several proofs of concept in parallel, test with real users and finish with a business case ready for funding. The structure is the point. A single proof of concept invites a binary outcome and a lot of sunk cost. Several, tested against the same decision rule, give you comparison.
What to measure: control metrics, not vanity metrics
The most seductive metric in an agent proof of concept is speed. It is easy to measure and easy to present. It is also the one most likely to mislead, because an agent that is fast and wrong is worse than a person who is slow and right.
Measure in the order that reflects risk.
Control measures come first
Track breaches of authority limits, payments proposed to unverified suppliers, duplicates not caught and changes to supplier details that were not escalated. In a payments process these are not just performance measures; they are your stop conditions. Agree in advance which of them ends the test.
Accuracy next, against a human baseline
Run the agent in shadow mode alongside your existing process for a period, so you can compare its proposals with what your team actually did. Record agreement, disagreement and, crucially, who was right when they differed. Without the comparison, accuracy figures have no meaning.
Then effort and flow
Only once control and accuracy are understood does it make sense to look at cycle time, the share of invoices handled without human intervention and the effort per invoice. Record the override rate too: how often did a reviewer change what the agent proposed, and why? A high override rate on a particular type of invoice tells you where the agent needs different handling, or none.
And the audit trail itself
Treat audit completeness as a measure, not an assumption. Sample decisions and check whether each can be reconstructed from the record alone. If an auditor could not follow the trail, the agent has not passed, whatever its accuracy.
Resist the temptation to add a dozen more measures. A handful that map directly to the stop-or-scale rule is more useful than a dashboard nobody reads.
When to stop
Stopping is the outcome most proofs of concept are not designed for, and it is often the most valuable. A test that ends early on a clear control failure has done its job and saved the cost of a larger programme. Build the expectation of that outcome into the project from the start, so that stopping does not feel like failure.
Three habits help.
Agree stop conditions as part of scope. A breach of authority, a pattern of undetected duplicates or an audit trail that cannot be reconstructed should each have a named consequence, written before the test.
Watch for sunk-cost reasoning. The longer a proof of concept runs, the harder it becomes to call it. A fixed end date and a fixed decision meeting make it easier to follow the rule you set rather than the mood in the room.
Separate "not yet" from "no". If the agent fails because the invoices were poorly captured upstream, or because an integration was incomplete, that is a different finding from the agent making unsafe decisions. Record the difference. It determines whether you extend once or stop.
What the proof of concept should hand over
A good proof of concept ends with a business case, not a slide deck. That means evidence the person funding the next stage can read and challenge:
- What was tested, with what data, over what period and by which users.
- What the results were against each hypothesis and against the pre-agreed stop-or-scale rule.
- The operating model that worked: where the agent acts, where a person approves and where escalation applies.
- The controls required in production, including limits, supplier verification, approval routing and audit trail.
- The integration and ownership questions that remain, such as who is accountable for the agent's decisions.
- A cost-and-benefit view built from the observed results rather than from vendor claims.
If any of these cannot be filled in, the test did not produce enough to decide, and that in itself is a finding.
What a UK finance team should add
For UK organisations, two additions are worth making. The first is accountability: decide who owns the agent's decisions in the same way you would for any delegated authority. On 19 October the FCA indicated that it is applying existing frameworks, including Consumer Duty and senior-manager accountability, to AI rather than writing new rules for agents. That statement is about AI in financial services generally, not accounts payable specifically, and firms should read the FCA's own text rather than rely on our summary. But the direction is consistent: existing accountability does not disappear because a task has been delegated to software.
The second is evidence discipline. Because much of the reporting on agent payments is US-led, avoid importing assumptions about suppliers, payment rails and approval practice. Test against your own.
Where Innovify fits
Innovify is a technology partner with deep expertise in banking, financial services, fintech and insurance, and we design compliance in from day one. For teams considering an accounts-payable agent, the useful starting point is a scoped proof of concept with the decision rule written first.
Innovify AI Labs lets enterprise teams run several proofs of concept in parallel, test with real users and finish with a business case ready for funding. If the question is how agents should be authorised to release money, our Agentic Commerce & Payments work covers the payments side. Explore Innovify AI Labs to talk through what a scoped accounts-payable test could look like for your team.
FAQ
What should an AI proof of concept in accounts payable test?
It should test hypotheses that could turn out to be false: that the agent matches real invoices accurately, stays inside its authority limits, works well alongside human reviewers, escalates exceptions correctly and leaves an audit trail that can be reconstructed. Speed and savings come later, once control and accuracy are understood.
How do you decide when to stop an AI proof of concept?
Agree the stop conditions before the test begins. A breach of authority limits, undetected duplicate payments or an audit trail that cannot be reconstructed should each have a pre-agreed consequence. A fixed end date and a decision meeting with a named owner stop the project drifting on sunk cost.
Why is accounts payable a good place to start with AI agents?
It already has approval chains, supplier records and audit trails, so delegated payments are easier to authorise and measure than in open-web checkout. Recent reporting, including a live Visa, BILL and Billtrust pilot, points to accounts payable and finance operations as an early home for agent payments, although that evidence is led by US companies.
What metrics matter most in an agentic accounts-payable pilot?
Control measures come first: authority breaches, payments to unverified suppliers and missed duplicates. Next is accuracy against a human baseline run in shadow mode. Cycle time, the share handled without intervention and reviewer override rates matter after that, because speed means little if control fails.
Can a proof of concept prove a return on investment?
Not on its own. It can produce observed results that feed a business case, including costs, controls and operating model, but the return should be calculated from what the test showed in your environment, not promised in advance. We would be cautious of any claim of a fixed saving made before the evidence exists.
Conclusion
The value of an accounts-payable proof of concept lies less in the technology than in the discipline around it. Agree the decision and the stop-or-scale rule first, test hypotheses that could fail, measure control before speed, put real users in the loop and be prepared to stop. Done that way, the test either earns the case for the next stage or saves you from paying for it.
