Jarvis V2 — AI Voice Assistant
A Python and Flask voice assistant with a Gemini-inspired web interface: it listens for voice commands, predicts intent using a trained Stacking model, and executes the matching action — opening an app, searching the web, or fetching system info — falling back to a browser search when no local match is found.
Problem
A voice assistant that only recognizes a fixed command list breaks the moment a user phrases something slightly differently or asks for an app that isn't installed.
Solution
Trained a Stacking classifier to predict intent from voice commands, and built a fallback path so anything the classifier can't confidently resolve locally still resolves to something useful — a browser search or web equivalent of the requested action.
Architecture
- 01Speech-to-text captures the voice command and normalizes it to text
- 02A trained Stacking classifier predicts the intended action from the transcribed command
- 03Matched intents trigger local actions (opening apps, fetching system info)
- 04Unmatched or low-confidence commands fall back to an equivalent browser action
- 05Flask backend serves a Gemini-themed web interface for text-to-speech feedback and interaction
Implementation
- ▸Trained the Stacking intent classifier on a labeled command dataset
- ▸Built the app-launch handler with a browser fallback for commands with no local match
- ▸Wired speech-to-text input and text-to-speech feedback into a single Flask-served interface
Challenges
- ▸Covering enough intent variety in training data for natural phrasing without over-fitting to exact command wording
- ▸Deciding when to fall back to the browser versus reporting a command as unrecognized
Result
A working voice assistant with local intent prediction and a browser fallback, published as an open-source project.
More projects
MCPify
Analyzes any AI agent's URL, detects its framework, and generates ready-to-use MCP configs — and exposes itself as an MCP server.
NirmanAI 2.0
An AI agent for Indian home construction planning — RAG-grounded chat plus deterministic engineering calculators for BOQ, Vastu, and loan schedules.
IntelliDocs
A multi-user RAG platform: upload PDFs, ask questions, and get AI answers with page-level citations. Live and deployed.