Skip to main content
MR Finder - Spatial Intelligence for Real-World Search

Never Lose Anything Again. Anywhere.

Team

  • Sutanuka Jashu (Researcher)
  • Arthur Baney (SDE)
  • Sowilo Xiong (SDE)
  • Jiachen Zeng (PM)

My Contribution

  • Built Figma-Unity Design Workflows
  • Designed Voice-Driven AR Navigation
  • Led Cross-Team UX Alignment

Tech Stack

Unity + MRTK, Wit.ai, Meta Quest SDK, SLAM, Cognitive3D
Developed at MIT Reality Hack 2024

Overview

We lose critical items daily—passports buried in drawers, keys swallowed by clutter, Wi-Fi codes on vanishing sticky notes. MR Finder tackles this chaos through AI-driven spatial anchors, blending voice commands and persistent mixed reality markers to turn any space into a searchable inventory.

As a five-person team at MIT Reality Hack, we reimagined lost-and-found by merging SLAM-based 3D mapping (Meta Quest 3) and Unity MRTK interactions with conversational AI. Users tag items via voice/photo and retrieve them months later with AR arrows—no computer vision or extra hardware required.

By anchoring digital memory to physical spaces, MR Finder redefines lost-and-found for real-world chaos.

Github Code Sample

● Why It Matters

Lost your keys... again?
We've all wasted hours searching for lost keys, passports, or that one missing charger. MR Finder tackles this universal frustration by reimagining how humans interact with physical spaces.

Airbnb Hosts

"Guests call me at 2AM for Wi-Fi codes"

Frequent Travelers

"I hid my passport somewhere safe..."

Caregivers

"I spend 30 mins daily searching for pills"

Our Solution

1

Map Your Space

Using your Quest 3 headset, scan your room for about 20 seconds. Behind the scenes, our advanced SLAM engine builds a precise 3D map of your environment with 1cm accuracy, establishing digital anchors on walls and floors.

2

Tag Critical Items

Simply speak a command like "Tag my passport here" or capture a photo of an item. The system records its location at pinpoint-accurate, GPS-free coordinates, ensuring the tag remains intact even if the room is rearranged.

3

Voice-Activated Search

When you need to locate something, just say, "Where's my medication?" Our GPT-4–powered system processes your natural voice command, matches it to your tagged items, and retrieves their exact coordinates instantly.

4

Follow AR Guide

AR cues take over as virtual arrows, audio prompts, or a heatmap overlay appear in your headset, guiding you seamlessly along an optimized navigation path rendered by MRTK.

● Airbnb Use Case

Effortless Hospitality, No More Notes
Airbnb hosts pre-tag essentials—Wi-Fi cards, extra towels, coffee machines—so guests simply wear a headset, speak their needs, and follow AR cues. No more handwritten notes or phone calls; language barriers vanish for a truly seamless stay.
Airbnb use case illustration

● Tech Highlights

○ SLAM Precision: Continuous scanning locks virtual tags to real objects, preventing AR drift.

○ ChatGPT Integration: Enables natural, conversational queries for a more intuitive experience.

○ Unity MRTK: Powers 3D menus and smooth interactions from hardware inputs.

○ Cognitive3D: Captures spatial data for actionable insights and digital twin potential.

● Next Steps

○ Automatic Object Recognition: Use AI-based image analysis to tag items without manual snapshots.

○ Expanded Collaboration: Share digital inventories across multiple headsets, ideal for offices, museums, or senior living communities.

○ Cloud Sync & Analytics: Integrate with cloud platforms to track usage patterns, enabling future digital twin capabilities.