Controlling complex 3D CAD software requires constant switching between keyboard shortcuts, 2D mice, and 3D controllers. Our system solves this problem: a wireless forearm band detects surface electromyography (sEMG) signals from muscles, evaluates them locally on an ESP32-S3 via an artificial neural network, and translates hand gestures into CAD commands in real time.
Test the EMG armband live in the browser. Select different gestures on the left. On the top screen, you will see the simulated **raw biosignals** (muscle voltage over time). On the bottom screen, you will see the reaction in the **CAD viewport (3D wireframe model)**.
How the system works behind the scenes. The synergy of noise-free analog technology and resource-saving edge ML algorithms.
Eight gold-plated dry electrodes surround the forearm. A medical analog front-end converter (ADS1298) digitizes the minimal microvolt signals of muscle contractions at a sampling rate of 1000 Hz per channel.
Our Convolutional Neural Network (CNN) was trained in PyTorch and optimized via TensorFlow Lite Micro for the ESP32-S3 microcontroller. It classifies 6 gestures with a latency of under 15 milliseconds directly on the device.
The armband behaves via Bluetooth LE either as a standardized Human Interface Device (HID) or sends direct control data via Websocket to a SolidWorks/Fusion360 plugin that controls the 3D camera.