Gesture as Inclusive Interface
#Gesture #Interface #ComputerVision #GestureRecognition #MachineLearning #HumanComputerInteraction #SpeculativeDesign #TouchDesigner #MediaPipe #TeachableMachine #Culture #DiversityThis work began with a question regarding the fundamental limitations that standardized interface functions impose on interaction systems. Specifically, gesture-based interfaces enable touchless, natural interactions through various movements of body parts, yet they still rely on traditional touchscreen input methods such as tap, pinch, and swipe. This dependence creates barriers for users with disabilities and often fails to account for culturally diverse hand gestures. In Gesture, Czech philosopher and writer Vilém Flusser argues that gestures are a mediating process between apparatus and human, carrying cultural and ideological meanings. He emphasizes that gestures reveal our behaviors, perceptions, and relationship to the world. As interactive design increasingly influences how we view ourselves and interact with the world, it is essential to unlock its full capacity to respond ethically and empathically to users’ diverse identities and cultural backgrounds. Building on this idea, my project re-imagines the current gesture-based interface by embedding diverse cultures, values, and worldviews into a more immersive and accessible form of HCI.
Project Objectives
This speculative project explores a range of finger, hand, and body gestures. It demonstrates how customized gestures can replace and perform existing navigational, action, and transformational functions by harnessing pose and gesture recognition through camera vision.
1. Diversity
The gesture interface should consider the cultural meanings of various body movements.
2. Inclusivity
The gesture interface should consider users with varying physical abilities and dexterity.
3. Functionality
The gesture controls should perform the three primary actions: navigation, zoom, and select.
4. Feedback
The gesture interface should provide clear visual feedback to users with sensory impairments.
Process & Methods
Hand Gesture Explorations
Body Gesture Explorations
Hand Gesture Controls Demo
Body Gesture Controls Demo
Critical Inquiry
→ The Veils of Asian Women
→ Black Rain White Light
→ Mirror Mirror
→ The Digitalization of the Creative Market
Biometrics / Emotions / Sensory
→ Pulse to Light
→ Graphic Language for Emotion-Based Interface
→ Sensitive SurfaceEnhanced User Experience
→ Gesture as Inclusive Interface
→ Natural Sound Mixer
© 2026 Dho Yee Chung