What was built?
A cutting-edge Human-Robot Interaction (HRI) project integrating spatial-temporal deep learning models to enable autonomous social robots to perceive, analyze, and react to human gestures and facial expressions in real time.
Problem Statement
Conventional robotic vision systems often struggle with lighting variance, occlusion, and real-time inference latency. In social HRI environments, delayed response times disrupt natural human communication and degrade user trust.
Engineering Approach & Architecture
We developed a lightweight MobileNetV3 + Bi-LSTM neural network architecture connected via WebSocket streams to a responsive web dashboard. This achieved 48 FPS real-time processing while running on low-power edge GPUs.
Key Engineering Obstacles
Obstacle: High Latency During Multi-Person Tracking
Implemented frame-skipping heuristics and dynamic region-of-interest (ROI) cropping to prioritize the primary speaker, reducing GPU memory bandwidth by 40%.
Obstacle: False Positives in Low-Light Conditions
Applied adaptive histogram equalization and temporal smoothing across 5-frame rolling windows to stabilize detection confidence scores.
Technologies & Tools Used
Core libraries, frameworks, and deployment services.