DeVoice

DeVoice is a mobile app for Android and iOS that transcribes speech, translates it, and plays it back — with your audio staying on the device for every step that matters. Speak in one language, read the translation, and optionally hear it spoken in a voice matched to yours.

The problem

Most translation apps assume you always have a connection, send your audio to the cloud, and read results back in a generic system voice. That works in a café with Wi‑Fi. It falls apart on a plane, in a basement, or anywhere you care about privacy.

I wanted something simpler and more personal: record speech, get a transcript, translate it offline, and hear the result without feeling like you handed a stranger your microphone.

What it does

The core loop is intentionally small:

1. Tap or hold the mic to record
2. DeVoice transcribes what you said on device
3. The transcript is translated into your chosen target language
4. You can edit either side, then play the translation back

Behind that simple flow sits a pipeline built for unreliable real-world audio — noisy rooms, multiple speakers, mixed languages, and phones that vary wildly in compute power.

How it works

On-device speech recognition

Transcription runs through Whisper, packaged for mobile via sherpa-onnx and executed entirely on the phone. Users pick a model size that fits their device — from a 75 MB “fastest” variant up to large-v3 for maximum accuracy.

Before audio reaches Whisper, it passes through a speech enhancement stage that reduces background noise. In practice this makes a noticeable difference in cafés, cars, and anywhere the mic picks up more than just the speaker.

Offline translation

Translations use Google ML Kit with downloadable language packs, so the app keeps working without a network connection once packs are installed. The UI exposes dozens of target languages, remembers recent choices, and can suggest a target based on automatic language detection.

Each identified speaker can also have their own preferred translation language — useful when two people are taking turns in a conversation.

Speaker recognition

DeVoice identifies who is speaking using on-device speaker embeddings. New voices get a friendly random display name (“Sunny Finch”, “Calm River”, and so on) that you can rename later. Known speakers are matched on future recordings and their settings persist locally in a SQLite database via Drift.

Voice playback

Reading a translation is always available. Hearing it spoken in your own voice is optional and requires explicit consent.

When enabled, DeVoice sends the translated text plus a short reference clip from your recording to a Cloudflare Worker that handles voice-cloned synthesis. The core transcription and translation pipeline never depends on the network — playback is the one feature that can reach out when you ask it to.

Architecture

DeVoice is a Flutter app organized by feature: startup, transcribe, and settings. Heavy work stays off the UI thread.

• ASR worker isolate — owns the recording session, coordinates denoising, and streams waveform levels back to the UI
• Whisper sub-isolate — runs inference without blocking interaction
• Speaker sub-isolate — computes and matches speaker embeddings in parallel
• SpeakController — orchestrates recording, transcription, translation, playback, and error recovery in one place

On first launch, the app downloads required ONNX models after the user consents — with a clear size estimate upfront. Older 32-bit ARM devices get a smaller default Whisper model automatically.

The interface is localized in eight languages (English, Spanish, French, Portuguese, German, Italian, Dutch, and Polish), matching the kind of cross-border conversations the app is built for.

Interaction design

The mic button supports two modes that people already understand from messaging apps:

• Tap to start and stop (with voice-activity detection handling natural pauses)
• Press and hold to record only while your finger is down, with a slide-to-cancel gesture if you change your mind

Transcripts and translations live in editable cards. Errors surface inline with retry actions instead of silent failures — important when model loading or permissions are involved.

What I learned

Building DeVoice meant wrestling with real mobile constraints: model sizes measured in hundreds of megabytes, inference times that vary by device, microphone permissions, and the expectation that everything should feel instant even when it cannot be.

The biggest design decision was drawing a hard line between what runs on device and what does not. Transcription, translation, denoising, and speaker ID stay local. Voice playback is opt-in and transparent about what leaves the phone. That trade-off keeps the app honest about privacy while still delivering the feature that makes it feel different from every other translator.

Stack

• Flutter — cross-platform UI
• sherpa-onnx — Whisper ASR, speech denoising, speaker embeddings
• Google ML Kit — on-device translation and language identification
• Drift — local persistence for speakers, settings, and preferences
• Cloudflare Workers — optional voice-cloned TTS synthesis