Handy Ai App
Handy Ai is an open-source Android AI agent that uses Claude to look at the screen, then tap, type and swipe through apps to complete tasks. It is designed for blind, low-vision and motor-impaired users, and also works for hands-free control of Android apps.
You can speak or type a task, hear each step read aloud, and stop it at any time. The app needs Android 11 or later and accessibility permission. It also supports included Claude steps or your own Anthropic API key, plus on-device OCR for reading screen text.
Voice input, text-to-speech narration, named aliases, local chat history and an easy mode with one large mic button help make the interface simpler to use. A floating accessibility shortcut can open the app from anywhere.
Features
- Voice-first task input with speech recognition
- Claude-powered tapping, typing, swiping and waiting
- Step-by-step narration with text-to-speech
- On-device OCR for reading screen text
- Accessibility shortcut button for quick access
- Easy Mode with a single large mic button
- Named aliases for contacts, orders and other shortcuts
- Local chat history with multiple sessions
- Destructive-action warning before sending, buying or deleting
