Interacting in 2D and 3D with SpatialMouse

When doing general computing, you probably use a conventional mouse or trackpad. Then if you want to work in 3D, such as when using a VR headset, you need to switch to motion-tracking controllers dedicated to spatial tasks. That switch represents inefficiency and an extra cost. SpatialMouse is a single device that performs both functions.

Imagine if you took a VR controller, such as for a Meta Quest, and glued it to a standard computer mouse. That is the basic idea behind SpatialMouse. Users can move it around a desktop and click like they would with any other computer mouse, but a VR headset can also track its movement in space for work in 3D environments. That’s particularly useful when interacting with simulated desktops and 2D applications within virtual 3D worlds.

SpatialMouse’s developers built their prototype using an Arduino Nano ESP32 board, a PMW3389 mouse sensor, and some inputs (buttons, switches, and a joystick). Those components fit into a custom 3D-printed enclosure that has a ¼”-20 screw on top for mounting a VR tracking unit.

The prototype uses an HTC Vive tracker, but users could mount any suitable tracker that works with their particular VR system.

To evaluate SpatialMouse, the developers created a 3D environment in Unity that includes a desktop monitor with a 2D screen. In testing with 12 participants, they found that SpatialMouse improved the perceived performance of task switching and usability.

SpatialMouse is open-source and the files are available on GitHub.