[ Back to home ]

Hand-tracked rehab for progressive muscle disease.

A standalone Meta Quest 2 Unity build for LGMD, FSHD, and myotonic dystrophy patients. Butterflies caught with bare hands in a Gaussian-splat garden, an overextension guardrail that redirects rather than punishes, and every parameter tunable from the Unity Inspector.

1. Abstract

A standalone Unity APK for the Meta Quest 2, with no controllers and no PC tether, built for the roughly 250,000 Americans living with progressive muscle disease, a population for whom exercise is one of the few interventions that slows decline. The patient stands in a World Labs Marble-generated Japanese garden and catches floating butterflies with bare hands. The design protects one outcome: keep the patient moving, never let them overextend.

The build is five C# scripts on Unity 2022.3 LTS and the Meta XR SDK, shipped end-to-end in an 8-hour hackathon. The time budget forced minimalism: every script does one job, every catchable lives off one tag, and the entire safety branch is one independent Update() loop. Hand tracking and the over-reach check run as separate branches that converge on a GameManager singleton owning score, timer, and UI.

The novelty is the patient population, not the engine. Every commercial VR rehab product is built for stroke, traumatic brain injury, or chronic pain; none target muscular dystrophy. AdaptVR does, and tilts hard toward "the patient cannot lose": generous catch radius, no fail screen, redirection rather than punishment.

• 5 C# scripts, built end-to-end in 8 hours, drive the entire loop: butterfly behaviour, spawning, hand colliders, overextension detection, session orchestration.
• 26 joints per hand via Meta's native hand tracking. No controllers, no haptic gloves, no buttons.
• 250,000+ Americans in the target population (LGMD, FSHD, myotonic dystrophy combined). Roughly 70% drop out of standard home-exercise programmes.
• 0.65 m overextension threshold on wrist-to-head distance, surfaced as a soft redirection cue rather than a fail state.
• Every clinical parameter on the Inspector. The therapist tunes once per patient and the configuration travels with the build.

2. Introduction

In Limb-Girdle Muscular Dystrophy (LGMD), Facioscapulohumeral Muscular Dystrophy (FSHD), and myotonic dystrophy, exercise is one of the only clinically validated ways to slow functional decline. It also sits inside a hard paradox.

• Adherence is poor. Roughly 70% of patients drop out of standard home-exercise programmes, which are generic, boring, and blind to how the disease progresses in each individual.
• Overexertion is dangerous. Where most populations benefit from "push harder," these patients suffer real muscle damage when they over-reach. The fatigue signal arrives hours late, so they cannot self-regulate by feel.
• The MRC grade 3 to 5 gap. Patients who move against gravity but tire quickly can benefit from exercise, but only through a designed system. They do not have one.

2.1 Why hand-tracked VR

The Meta Quest 2 makes a deployable answer structurally possible, and is cheap enough at retail to reach patients. Controllers are a real barrier here: trigger pulls need exactly the grip strength and finger isolation the disease takes away.

• No controllers. The 26-joint OVRSkeleton pipeline reads wrist position directly, so the patient never grips, squeezes, or presses anything.
• Standalone hardware. One Android APK on a Quest 2: no PC, no cables, no calibration. The headset goes home with the patient.
• Immersive distraction. A Gaussian-splat garden, not a flat exercise screen. Exergaming achieves 3 to 5 times better adherence in neuromuscular populations than standard home programmes; immersion carries that gain.

2.2 What is actually new

The hand-tracking SDK and the VR engine are off-the-shelf. The contribution is the target population and the safety posture it requires.

• The population gap. XRHealth, REAL System, MindMaze, and Karuna Labs are the major commercial VR rehab platforms. None mention muscular dystrophy in their clinical materials, a gap verifiable in 60 seconds of searching their pages.
• Redirection, not punishment. Past a tunable reach threshold the patient gets a soft cue: no fail state, no penalty, no points lost. The system redirects the body, not the will.
• The patient cannot lose. A generous catch radius, slow sine-wave bobbing, and 1.2 second respawns keep the score climbing steadily. Frustration that maps onto disease severity is the dropout dynamic the build is designed to prevent.
• Therapist-tunable surface. Every clinical parameter lives on the Unity Inspector, not in code, set once per patient.

Section 3 is the engineering that holds those four commitments on the Quest 2.

3. System

3.1 Scene composition

• OVRCameraRig with OVRManager.HandTrackingSupport = HandsOnly. LeftHandAnchor / RightHandAnchor carry OVRHand and OVRSkeleton; no controllers are wired in.
• Skybox. The World Labs garden as a panoramic image on a Skybox/Panoramic material.
• World-space canvas at (0.4, 1.2, 1.5) with ScoreText (72 pt), TimerText (36 pt), and WarningText (28 pt, hidden by default), floating at the patient's natural sightline.
• Butterfly prefab at a 5 to 10 cm wingspan, with a 15 cm trigger SphereCollider and a kinematic Rigidbody so trigger callbacks fire without physics interference.

3.2 The catch loop: ButterflyController.cs

• Each butterfly floats toward a random point inside its zoneCenter / zoneRadius sphere, picking a new target once within 5 cm of the current one.
• A Y-axis sine bob (1.2 Hz, 0.06 m amplitude) and a yaw toward the direction of travel keep the motion alive, not mechanical.
• The float target Y is clamped above the floor (_target.y >= zoneCenter.y - zoneRadius * 0.5), so butterflies never sink below a comfortable reach line.
• OnTriggerEnter checks for tag "PlayerHand". On a catch: optional particle effect and sound, a call to GameManager.Instance.RegisterCatch(), renderer and collider hidden for 1.2 seconds, then a respawn at a fresh point in the same zone.

3.3 Initial placement: ButterflySpawner.cs

One empty GameObject. At Start() it computes a spawn center at cameraTransform.position + flattened-forward * distFromHead (default 1.2 m), instantiates count butterflies (default 5) at random points inside a sphere of zoneRadius = 0.9 m around it, writes zoneCenter and zoneRadius into each controller, and steps back. Ongoing motion is the controller's problem.

3.4 Hands as colliders: HandCollider.cs

One per hand. Every Update() walks handSkeleton.Bones, finds Hand_WristRoot, and snaps the GameObject to it. A kinematic Rigidbody (required for OnTriggerEnter under Unity's physics rules) and a 10 cm trigger SphereCollider tagged "PlayerHand" form the only handshake with the butterfly script.

3.5 Safety guardrail: OverextensionDetector.cs

Independent of the catch loop. Every frame it measures Euclidean distance from centerEyeAnchor.position to each wrist bone. If either exceeds maxReachMeters (default 0.65 m), GameManager.ShowWarning() surfaces a soft cue for 2.5 seconds. No fail state; the score is untouched.

3.6 Session orchestrator: GameManager.cs

• Static singleton (GameManager.Instance), so the butterflies and the overextension detector find it without scene wiring.
• Owns the session state: _catches, _elapsed, _sessionActive, _warningTimer. Default session is 300 seconds.
• Drives the three TextMeshPro labels (score, timer, warning) and freezes the timer at "Done!" when the session ends.
• Reads OverextensionDetector.EitherOverextended every frame and calls ShowWarning() with a 2.5 second display window per cue.

3.7 Therapist controls

Reused across sessions, these scale difficulty to MRC grade and disease profile without recompiling.

• ButterflySpawner.count: butterflies at session start.
• distFromHead: spawn distance from the head. Closer for weaker patients, further for stronger.
• zoneRadius: how spread out the butterflies are inside the zone.
• Float speed, bob frequency, bob amplitude on the prefab: how briskly butterflies move.
• maxReachMeters: the overextension guardrail threshold.
• sessionDuration: session length in seconds.

4. Evaluation

Validated qualitatively on Quest 2 hardware. Butterflies respawn cleanly inside the zone after each catch. The 15 cm butterfly trigger radius is deliberately generous because patients with MRC 3 to 4 strength have imprecise distal control.

• Standalone Android APK. Single install, no tether, no per-session calibration.
• Hand tracking holds under the seated and standing reach patterns the protocol targets, in normal indoor lighting.
• Overextension cue at 0.65 m surfaces the redirection text inside 2.5 seconds of the threshold crossing, every time.
• Therapist parameter sweep. Moving distFromHead from 0.9 m to 1.5 m and zoneRadius from 0.6 m to 1.1 m on the Inspector produced the expected difficulty changes without code edits.

The 3 to 5 times adherence figure in §2.1 is from the literature. It motivates the design; it is not a measurement of AdaptVR.

5. Conclusion and Implications

The clinical case is short. The population needs to exercise, and every commercial VR rehab platform fails it. The Quest 2 is finally cheap and capable enough to close that gap with hand-tracked play that respects what the disease takes away. If even a fraction of the 70% home-programme dropout is recovered by a build patients want to put on, the result is muscle function preserved longer, slower loss of independence, and fewer of the cascade complications that follow when neuromuscular patients stop moving. No fail screen, redirection over punishment, therapist-tunable parameters, and no controllers are what make it deployable, rather than something that demos well and rehabilitates nobody.

5.1 Limitations and future work

• No clinical evaluation yet. Engineering correctness is validated on the headset; efficacy is unproven. Next: a longitudinal adherence study against a standard home programme, plus a therapist dashboard for remote configuration and progress tracking.
• Instantaneous safety signal. The cue fires on wrist-to-head distance, not fatigue, so a patient reaching just inside 0.65 m all session never trips it. Next: reach distance, hand velocity, and jitter over a rolling 30 second window, with adaptive spawn-zone resizing.
• Partial per-condition profiles. LGMD posture-locked seating and FSHD overhead-reach exclusion are still set by the therapist. Next: LGMD seated, FSHD no-overhead, and General balanced profiles picked on the headset.
• Single environment. One garden skybox ships. Next: multiple World Labs splats (beach, meadow, garden) rotated per session to keep long-term use varied.
• Hand tracking degrades in low light, a Meta SDK constraint the build inherits. Sessions need normal overhead lighting.

6. Acknowledgments

The garden environment is a Gaussian-splat export from World Labs Marble. Hand tracking is the Meta XR SDK. The clinical framing draws on published guidance from TREAT-NMD on exercise in progressive muscle disease.