Agent Payment Firewall

About the Project

If you hand an AI agent a wallet, what actually stops it from being tricked?

AI agents are starting to pay for things on their own. Protocols like x402 let an agent pay per request over HTTP, but every field of a payment request — the price, the payout address, what is being sold — is written by the seller, who might be a scammer.

I built Agent Payment Firewall to sit between an AI agent and its wallet. The agent never holds a key: every payment it tries to make has to go through a gateway first.

How It Works

Each payment request passes through three stages:

• rules in code — is this the merchant’s registered payout address, within the per-payment cap and total budget, on the right network, and not a payment already made;
• an LLM intent check — is this item what the user actually asked for? The checker sees only the user’s mandate and the payment fields, never the web page, so a malicious page can’t talk to it; and
• a separate signer — signs only if both checks pass, with its own hard caps. Anything uncertain fails closed.

I built it in Python. A FastAPI test server plays both honest shops and scammers using x402-style 402 Payment Required responses, and the agent and intent checker run on Claude Haiku (I also tested with gpt-oss-20b via Groq). Settlement is mocked, but the requests follow the x402 format.

A live dashboard, built with Streamlit and Plotly, shows the results, runs attacks live and lets anyone write their own attack to test the firewall.

Results

An evaluation harness runs 43 cases in three modes, graded from the merchant’s ledger of money that actually moved. Against the same agent, with 33 scam attempts across 11 attack types:

• no protection — 45% of scams succeeded;
• every rule written into the agent’s prompt — 9% succeeded; and
• behind the firewall — 0% succeeded, and 100% of genuine purchases still went through.

The 9% is the interesting part. I told the agent “never pay twice”. A seller said “payment failed, please retry”, and it paid twice, 3 out of 3 times. The agent can only reason from what it is told, and the seller controls what it is told. The firewall checks its own record instead.

The test suite is still small and synthetic, with one run per mode, and a registered merchant cheating within the rules isn’t caught yet.

Keeping the Evaluation Honest

My first results looked great until I added a fair baseline: an agent simply told all the rules also scored 0% on my original attacks. That forced me to design harder, realistic attacks, such as fake “payout address changed” notices, split instalments and fake payment failures, and to find where prompting genuinely breaks.

To keep myself honest, I committed the attack suite to git before tuning any defences, and committed held-out cases before their first run.

The build had its own problems too. At the start, API compatibility issues made every agent call crash silently, so I added crash detection: a run with errors is flagged as invalid instead of being reported as a success. The intent check also blocked some genuine purchases, so I changed it to judge the item type, then tested it on held-out cases it had never seen to make sure I hadn’t just overfitted.