Mural Recognition App
A scan-to-learn app for the "150 Years of Women at UC Berkeley" installation: 41 portraits, full biographies on tap.
About this project
The 150 Years of Women at UC Berkeley mural lives in the Undergraduate Academic Building and celebrates 41 distinguished women whose work, scholarship, and activism have shaped the university: Jennifer Doudna, June Jordan, Maxine Hong Kingston, Julia Morgan, Beverly Cleary, Alice Waters, Barbara Lee, and Judith Butler, alongside alumnae, faculty, friends of the university, and staff. The mural was designed by Twin Walls Mural Company (2020–2022) and is the centerpiece of UC Berkeley's "150 Years of Women" commemoration.
I collaborated with Professor Oliver O'Reilly, Vice Provost for Undergraduate Education, and a cross-functional team to design and ship the companion app: a scan-to-learn experience that lets passersby photograph the mural and instantly access annotations and full biographies for each of the 41 women portrayed. The goal was simple: turn a static work of public art into an interactive companion piece that travels with the visitor.
I defined the product scope and the end-to-end user experience, then built the computer-vision pipeline and the full-stack app in Next.js with PostgreSQL for serving biographies, citations, and context in real time. Recognition runs entirely in the browser with OpenCV.js: rather than face recognition, it matches ORB features and solves a homography to figure out which region of the mural is in view, then maps that region to the people painted there. No camera frames ever leave the device.
Tapping a detected portrait opens that alumna's profile, and an optional voice mode reads her story aloud: Claude drafts a short, first-person narration from her biography and ElevenLabs speaks it. The result is a quiet, low-friction layer on top of the installation: point your phone, and the wall starts talking back.