Catdex
A neighbourhood cat-collecting game that uses computer vision to detect, classify and recognise individual cats from photos, turning each new cat into an AI-written trading card.
Project Details
About the Project
What if the cats in your neighbourhood had their own Pokédex?
I built Catdex to turn spotting cats into a collection game. Snap a photo with your phone, and the app finds the cat, files it by coat pattern and checks whether you’ve met it before. New cats get a name, a personality and their own trading card, while familiar cats have their profile updated with every sighting.
Rationale
Catdex was inspired by my love for cats and the Telegram channel ‘cats of smu’. I thought it would be cool to keep track of the community cats around campus through something like a Pokédex for real cats.
How It Works
I built Catdex in Python using only pretrained models, with no custom training:
- YOLO11 detects the cat in each photo, rejecting dogs and group shots, while OpenCV rejects blurry images;
- CLIP classifies the coat zero-shot into one of nine coat pages, such as tabby, calico or tuxedo, refined with hand-labelled examples;
- a pet re-identification model creates a “fingerprint” for each cat and compares it against the database to recognise cats seen before; and
- Claude Haiku 4.5 with structured outputs writes each new cat’s personality, and Pillow draws the card.
Everything comes together in a phone-first Gradio web app with custom CSS and JavaScript, complete with a cat-ball capture button, card reveals, badges, sighting histories and a collection view.
Telling Cats Apart
Because Catdex is meant to tell individual cats apart, different cats with the same coat pattern were sometimes merged into one entry. I added two safety nets: when the similarity score falls between the match and ask thresholds, the app asks the user whether it’s a cat they’ve already met; and if a cat is still filed under the wrong entry, users can correct it manually from the cat’s profile.
Project Satisfaction
I’m proud that the app actually works end to end, from photo to trading card, in a single app — and that the interface turned out very cute.
Biggest Challenge
Model accuracy was a key challenge. Zero-shot CLIP often confused similar patterns, so I wrote a labelling rulebook, built dev/test splits and added few-shot examples. Black cats were rarely classified correctly and the blur check wrongly rejected dark photos; adding contrast stretching before the blur check, plus better prompts and examples, made dark cats much more reliable.
I also learnt to evaluate models honestly rather than trusting a good demo.
Areas for Improvement
With more time, I would keep improving coat classification for white and black cats, and teach the app to recognise more animals than just cats and dogs. I would also deploy it on a permanent link — right now, the public version only works while the code is running on my laptop.
Build a project with us
Join AI Lodge, build with your track through the semester, and show your project at Hack Day. We will feature it here.