AI build · May 12, 2025
WhisperAI: real-time communication overlay
Created a real-time communication intelligence overlay analyzing facial expression, tone, pose, and sentiment.
Overview
I built an overlay that uses the webcam, the microphone, and AI models to read how people show up on Zoom calls, from posture to voice energy to emotional presence. It works as a real-time coaching layer for high-stakes conversations.
What I built
- Real-time read: facial sentiment and voice tone detection
- Multimodal overlay: combines pose, emotion, and audio features
- Modular frontend: draggable, resizable UI
Stack
- TensorFlow.js
- Azure Speech
- Vite
- Next.js
- TypeScript
What I learned
- Multimodal analysis can turn any video call into a feedback loop. The hard part is keeping the UX low friction.
Outcome
Live at whisperxai.com.