CartPilot
An AI shopping assistant that turns a cooking goal into a scaled recipe and a price-compared grocery list, with browser agents searching several stores without ever buying anything.
Project Details
About the Project
We’ve all been there: halfway through baking a cake, you realise you’re out of eggs. Or you buy a 1kg bag of cocoa powder when the recipe only needed 40g, and the rest sits in the cupboard for a year.
Comparing groceries is surprisingly hard too. Stores sell the same item in different pack sizes (“500 g”, “6 x 50ml”, “12’s”), so the sticker price is misleading, and look-alike products like “cocoa drink mix” and “cocoa powder” are easy to mix up. I wanted an assistant that does the tedious part, working out what you need, checking what you already have and comparing stores, while leaving every decision to you.
What It Does
CartPilot turns a cooking goal into a price-compared shopping list and a step-by-step recipe:
- tell it what you want to make — for example, “bake a chocolate cake for 6”, and the AI suggests 2–3 recipes with ingredients scaled to your servings;
- snap your fridge — the AI checks which ingredients you already have, marking unsure items Please check instead of assuming;
- approve the list — nothing automated happens before this;
- watch the agents — AI agents search several stores in parallel, live, in a grid of browser screens with an activity log and running totals;
- get the best product — chosen for each ingredient by price per unit and smart pack size, with alternatives you can swap in; and
- cook and use up leftovers — a Leftovers panel suggests recipes for what you’ll have extra of, and numbered, beginner-friendly steps take you through the recipe.
CartPilot never adds to cart, logs in or buys anything. The final output is a list of product pages, and you decide what to buy.
How It Works
The backend is Python and FastAPI, streaming live browser screenshots to the page over a WebSocket. Google Gemini handles recipes, fridge photos, product match reviews and leftover ideas, with every response in JSON mode and validated by Pydantic. Playwright runs a pool of parallel browser pages, and I built 3 mock grocery stores with different layouts, brands, prices and messy pack sizes, so the demo is reliable and doesn’t depend on scraping real stores.
Every pack size is converted into a standard unit and compared by cost per 100g, 100ml or piece. CartPilot then chooses the cheapest combination of packs that covers what the recipe needs, not just the cheapest pack, so you don’t overbuy.
Responsible AI
Automation only starts after you approve, and a Stop button halts every agent instantly. A guard in code, with its own tests, means the agent physically cannot click anything like “add to cart”, “checkout”, “pay” or “login”. The AI’s confidence is also combined with rule-based checks, and low-confidence items must be confirmed by you.
Project Satisfaction
I’m most proud that the AI can never spend your money: nothing runs until you approve the list, there’s a Stop button, and the code blocks the agent from clicking “add to cart” or “checkout”. Watching three stores get searched live at the same time was a really cool moment, and a test suite of nearly 400 tests backs it all up.
Biggest Challenge
Comparing prices was harder than I expected, because every store writes pack sizes differently, like “500 g”, “6 x 50ml” or just “Value Tin”. I wrote a parser to convert them into standard units, with lots of tests to catch mistakes. Real grocery sites also have logins and bot detection, so instead of fighting that, I built mock stores behind a common interface that real stores can plug into later through official APIs.
Key Takeaway
AI is great at understanding goals and photos, but simple maths and rules are more reliable for prices and units. I also learnt that responsible AI is mostly about design choices: approval steps, confidence flags and hard limits in code.
Build a project with us
Join AI Lodge, build with your track through the semester, and show your project at Hack Day. We will feature it here.