Capstone
helm
AI-powered surgical instrument tracking system
A surgical instrument-readiness system that uses infrared-responsive ink and tray scanners to track when and where each instrument was last handled, so OR teams catch missing equipment before it becomes a delay.
ROLE
Project Lead
TIMELINE
20 Weeks
TEAM
6 UX Designers
TOOLS
UX Design
UX Research
System Design
AI Integration
Autonomous Systems
Brand Identity
THE PROBLEM
Equipment issues cause more OR delays than any other single factor.
This was confirmed through multiple rounds of primary and secondary research, from published studies to direct interviews with OR staff.
“Humans being humans is probably one of the biggest reasons for the smaller delays. The bigger ones? Probably equipment problems.”
— CHIEF OF ANESTHESIA · DIRECT INTERVIEW
45.9%
of OR delays traced to equipment or instrument issues, independent of human factors.
WUBBEN ET AL., 2010
THE DEFINING QUESTION
How might we…
Use AI to reduce the compounding effect of small delays in surgical workflows, improving experiences for both patients and clinicians?
THE SOLUTION
An AI-powered system that catches delays before they compound.
One missed instrument doesn't just cost the OR team time, it raises real patient risk. helm's AI layer closes that gap before it grows.
THE SYSTEM
Physical hardware and digital software work as one ecosystem.
The tray senses what's happening in the room. The dashboard tells the team what to do next.
THE DESKTOP APP
A digital dashboard that tracks instruments as cases progress.
helm pulls case data from Epic to build a fuller dashboard, locate any instrument, and surface risk alerts and next-step guidance for OR nurses and scrub techs.
THE TRAY
A physical tray that scans instruments as they're placed.
Instruments are tagged with invisible NIR ink and read the instant they land on the tray, confirmed by an AI layer for accuracy. This makes manual counts no longer necessary, while being cheaper than RFID.
THE AI BOUNDARY
Recommend, never decide.
Every clinician drew the same line on AI, independently. Knowing where the system shouldn't act mattered as much as knowing where it should, so helm surfaces what it detects and leaves the decision to the OR team.
EASY RETRIEVAL
A missing instrument is never more than a glance away.
Nurses and scrub techs can pull up any tagged instrument's last location and status through the dashboard's map tab on the sidebar.
VOICE TO TEXT
Sterile hands still get full access to helm.
Scrub techs speak directly to helm instead of touching a screen. If the OR nurse steps out, they can complete tasks on their own that used to be off-limits while sterile.
IMPACT
How helm benefits hospitals long-term
Reduced risk
Lower risk of missing instruments, less time lost searching.
Reduced cognitive load
Less reliance on manual counting, more attention for the patient.
Increased efficiency
Fewer delays before procedures even begin.
THE PROCESS
Getting here meant getting our hands dirty.
Twenty weeks of user research, prototyping, and iteration, three pivots when the data hit dead ends, and one Demo Day to test our product.
RESEARCH & DISCOVERY
CONCEPT DEVELOPMENT
TESTING & DEMO DAY
FINAL STUDIO CRITIQUE
RESEARCH & DISCOVERY
Interviews and affinity mapping turned assumptions into direction.
We shadowed OR staff, interviewed surgeons and nurses across specialties, and synthesized it into one clear direction: equipment readiness was the problem worth solving.
15+
interviews with surgeons, nurses, and anesthesiologists across 20 weeks.
3
peer-reviewed studies backed our findings, alongside direct interviews with OR staff.
FROM CONCEPT TO PROTOTYPE
Build detection into the tray that's already at the center of every case.
We reframed the problem: put instrument detection into the Mayo tray itself, avoiding the need for another device to track. From there the concept survived three real pivots, each forced by what the work actually demanded.
Three decisions turned early sketches into what actually shipped.
The tray reframe
Shadowing an OR showed most rooms had no cameras, ruling out our original halo-shaped design. A handheld wand scanner came next, but still required manual scanning. The tray reframe solved both problems.
RFID → NIR ink
RFID meant competing with hospitals' entrenched tagging systems. Dr. Maged Henary's near-infrared ink, patented and clinically validated for retained surgical items, offered a custom-writable alternative never applied to instrument readiness before.
A layered pipeline
A single force-sensitive resistor drifted on its own, so we reframed it as a first-pass signal, narrowed by a YOLOv8n detection layer and confirmed with a final contextual check.
Build a dashboard that shows instrument readiness first.
Early wireframes tested whether case status was clear at a glance. Usability sessions flagged where that clarity broke down, and the mid-fidelity screens got rebuilt around fixing it.
The difference maker: NIR ink = a fraction of RFID's per-tag cost
$2+
SURGICAL RFID · COST PER TAG
Sterilizable, autoclave-rated tags used in systems like STANLEY Healthcare's STM.
INDUSTRY PRICING DATA, 2026
5–10¢
NIR INK · COST PER TAG
Custom-writable, applied per instrument, and clinically validated for real surgical environments.
DR. MAGED HENARY · DIRECT INTERVIEW
TESTING & DEMO DAY
Testing exposed what the wireframes couldn't predict.
More than fifteen OR staff tested and/or provided feedback on early prototypes across two rounds, surfacing interface gaps and confirming the physical build held up under real handling.
WHAT TESTING CAUGHT
Early dashboards buried case-readiness status three clicks deep. Usability sessions flagged the same friction twice, so the home screen got rebuilt around it.
MY ROLE
Leading the project meant owning the direction from initial research through Demo Day.
Research & synthesis
Led primary research, including OR shadowing and interviews with surgeons, nurses, and anesthesiologists.
Concept & framing
Reframed the initial camera concept into the tray-based direction the team built toward.
System design
Directed the physical and digital system architecture across all three pivots.
Testing & validation
Ran usability sessions with OR staff and validated the final build ahead of Demo Day.
REFLECTION
A good concept only survives contact with real users if you're willing to rebuild it. Every pivot in this project came from listening past the first answer, and that discipline now shapes how I approach every system I design.
See how the entire process actually unfolded.
The complete 142-slide process book lives on Figma and covers every round of research, testing, and iteration behind helm.