Platform

A modular headset only works if the software doesn't care which modules it's running on. Swap the camera, the display or the processor, and every app should keep working. That requirement shaped all the software we built around HoloFlect.

The problem underneath

The hardware made the problem obvious. Each part brought its own operating system support and its own API: the tracking camera ran on one set of systems, the neural compute sticks on another, the Snapdragon board on Debian and Android (see Compute and Sensors). Each depth camera had its own SDK. Each controller reported input its own way. An app written for one HoloFlect configuration would break on the next.

And beyond HoloFlect, the AR market was split across phones, tablets and headsets, with no platform that dominated. Developers had to pick one and rebuild for the rest.

AquariOS

AquariOS, which we also called HoloFlect OS, was our design for the layer between the operating system and the apps. In the Molecules architecture it lives on the Compute Cluster, alongside the interface layer and storage. Its main ideas:

• Device independence. An abstraction layer presents spatial awareness, controllers, gestures and other triggers without the app knowing which hardware produced them.
• Several apps at once. An app controller lets multiple AR and VR apps share one space, the way desktop apps share a screen.
• A standard interface. Common dialogs, settings and controls on every element, instead of a new game-style interface for each app.
• Security. Signed messages, encrypted storage, and isolated panels for logins and payments.

It was meant to run on HoloFlect and on everyone else's devices as well: mobile, standalone and tethered headsets alike.

The SDK and The Blue Vale

The practical piece of this was a HoloFlect SDK: a compatibility layer that translates input from each platform into one common interface and uses whichever AR toolkit fits the device.

That SDK became the foundation of The Blue Vale, our augmented reality tabletop roleplaying game. The game ran on ARKit iPhones and iPads, Tango devices, HoloLens, Surface tablets and Epson Moverio glasses, and on older phones using a printed game mat, all through the layer that came from HoloFlect. A HoloFlect tier was planned as a stretch goal for the game's Kickstarter.

Where it led

We explored many personal, everyday uses for a headset like this. Our conclusion was that AR on its own is still hard to justify for daily use, and the strongest applications are around connected devices, home hubs and instrumented spaces, with distributed compute doing much of the work.

Building HoloFlect also showed us what was missing underneath. Commercial depth sensing rarely scales past one room, sensors don't talk to each other, and AR development platforms rarely go beyond simple visualizations and games for specific devices. That's where our xR Ecosystem research picked up: a nodal programming interface, sensor data standardized into unified types, and distributed compute as a core part of the design. The same thinking runs through the MRiot Vision Pro Module.