How to Implement Whisper Speech-to-Text in Flutter

Speech-to-text can make a Flutter app feel almost magical: users speak naturally, and useful text appears a moment later. But sending every recording to a cloud API is not always the right trade-off. It adds network latency, creates an ongoing service cost, and may be unacceptable when users are dictating private notes, interviews, medical information, or internal business data.

That is where whisper_cpp_flutter_plus comes in. It runs whisper.cpp directly on Android and iOS, so transcription can continue without an internet connection after the model is installed. Audio stays on the device, while your Flutter code gets access to file transcription, live microphone updates, timestamps, translation, model management, voice activity detection, and subtitle export.

In this guide, we will build the foundation for a production-friendly, offline speech-to-text feature in Flutter. We will download a verified model, load it once, transcribe a WAV file, stream text from the microphone, and cover the performance and lifecycle details that matter on real phones. The examples use whisper_cpp_flutter_plus 0.4.0.

What we are building

Our implementation has four parts:

Install and configure the Flutter plugin.
Download a checksum-verified Whisper model into app storage.
Keep a WhisperEngine alive while the feature is in use.
Use that engine for recorded files or live microphone transcription.

The package currently supports Android API 24 and later and iOS 14 and later. It does not currently target web, macOS, Windows, or Linux, so plan a platform-specific fallback if your Flutter product also ships on desktop or web.

1. Add the package

From the root of your Flutter project, run:

flutter pub add whisper_cpp_flutter_plus

Then import its public API wherever you manage transcription:

import 'dart:io';

import 'package:whisper_cpp_flutter_plus/whisper_cpp_flutter_plus.dart';

Whisper model files are intentionally not bundled with the plugin. They are large, and the best model depends on the languages, accuracy, download size, and speed your app needs. We will download one in a later step.

2. Configure microphone access

Android

The plugin contributes android.permission.RECORD_AUDIO to the merged Android manifest. You do not need to duplicate it. At runtime, either call WhisperRecorder.requestPermission() yourself or let transcribeMicrophone() request access when recording starts.

As with any permission, explain the value before showing the system dialog. “Allow microphone access to create private, offline transcripts” is much more reassuring than asking without context.

iOS

Add a clear microphone usage description to ios/Runner/Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>Microphone access is used to create private, offline transcripts.</string>

The plugin works with CocoaPods and Flutter's Swift Package Manager integration. Flutter uses the dependency manager already configured for the application, so there is no separate native Whisper package to add manually.

3. Download and load a verified model

For a first English-language implementation, the Tiny English model is a sensible mobile starting point. It is smaller and faster than larger models, although larger models can improve recognition accuracy. The package's verified model catalog is preferable to scattering a mutable download URL through your app: catalog entries pin an upstream revision and SHA-256 checksum.

class SpeechToTextService {
  final WhisperModelManager _models = WhisperModelManager();
  WhisperEngine? _engine;

  WhisperEngine get engine {
    final value = _engine;
    if (value == null) {
      throw StateError('SpeechToTextService has not been initialized.');
    }
    return value;
  }

  Future<void> initialize({
    void Function(double fraction)? onDownloadProgress,
  }) async {
    final descriptor = WhisperModelCatalog.tinyEnglish;

    var model = await _models.findCatalogModel(descriptor);
    if (model == null) {
      await for (final progress
          in _models.downloadCatalogModel(descriptor)) {
        onDownloadProgress?.call(progress.fraction);
      }
      model = await _models.findCatalogModel(descriptor);
    }

    if (model == null) {
      throw StateError('The Whisper model could not be installed.');
    }

    _engine = await WhisperEngine.load(model.path);
  }

  void dispose() {
    _engine?.dispose();
    _models.close();
  }
}

Call initialize() before enabling the record button. In the UI, show an honest download state—model downloads can be noticeable on mobile data—and keep the fraction for a progress indicator:

final speech = SpeechToTextService();

await speech.initialize(
  onDownloadProgress: (fraction) {
    setState(() => downloadProgress = fraction);
  },
);

WhisperEngine.load() uses the model at its existing local path; it does not copy or own the file. Do not delete or replace that file until the engine has been disposed. Also avoid loading the model for every button tap. Loading once and reusing the engine removes unnecessary delay and memory churn.

4. Transcribe an existing WAV file

whisper.cpp expects mono, 16 kHz floating-point PCM. Fortunately, you do not need to hand-write an audio conversion pipeline for common WAV input. WhisperAudio.readWav() decodes supported WAV files, mixes channels, and resamples the audio for you.

Future<String> transcribeWav(
  SpeechToTextService speech,
  File audioFile,
) async {
  final samples = await WhisperAudio.readWav(audioFile);

  final options = const TranscribeOptions(
    language: 'en',
    tokenTimestamps: true,
  ).withPerformanceMode(WhisperPerformanceMode.balanced);

  final task = speech.engine.transcribe(samples, options: options);

  final progressSubscription = task.progress.listen((percent) {
    print('Transcription progress: $percent%');
  });

  try {
    final result = await task.result;
    return result.text;
  } finally {
    await progressSubscription.cancel();
  }
}

The transcription job is non-blocking and exposes progress separately from its final result. This is a good fit for Flutter: update a progress bar from the stream, await the result, and keep the interface responsive. If the user leaves the screen or presses Cancel, call task.cancel() rather than leaving native inference running in the background.

The result contains more than one string. Depending on your product, you can use segments and timestamps to build an editable transcript, highlight the active phrase during playback, or export the output:

final result = await speech.engine.transcribe(samples).result;

final plainText = result.toPlainText();
final json = result.toJsonString();
final srt = result.toSrt();
final webVtt = result.toVtt();

5. Add live microphone transcription

For dictation, captions, or voice notes, the package can manage microphone capture and windowed streaming. The default configuration decodes every two seconds with a 30-second rolling window and treats the latest four seconds as provisional.

Future<void> startListening(SpeechToTextService speech) async {
  final task = await speech.engine.transcribeMicrophone(
    options: const TranscribeOptions(language: 'en'),
  );

  final updates = task.updates.listen((update) {
    setState(() {
      transcript = update.text;
    });
  });

  // Store `task` and `updates` as State fields in a real screen so the
  // Stop button and dispose() method can reach them.
}

There is one important UI rule here: replace the displayed live text on every update. Do not append update.text. The newest words are provisional and may change when a later audio window gives Whisper more context. The update's confirmed content is append-only, but the combined text is designed to replace your current preview.

When the user presses Stop, flush the remaining audio and use the completed result:

final complete = await task.stop();
await updates.cancel();

setState(() {
  transcript = complete.confirmedText;
});

Use stop() for the normal “finish my transcript” flow. Use cancel() when the operation should be abandoned. Cancellation completes with a WhisperException, so handle it distinctly from an unexpected failure.

6. Turn the snippets into a solid Flutter screen

A good transcription screen is mostly state management. At minimum, model these states explicitly:

Preparing: checking or downloading the model.
Ready: the engine is loaded and the user can record.
Listening: the microphone is active and live text is updating.
Finalizing: remaining audio is being flushed.
Complete: the final transcript can be edited, copied, or exported.
Error: show a useful message for denied permission, download failure, invalid audio, or insufficient storage.

Disable actions that conflict with the current state. A one-shot or streaming task reserves its engine until that job finishes, so do not start two jobs on the same engine simultaneously. If your product truly needs parallel transcription, load separate engine instances and be mindful of the much higher memory cost.

Finally, clean up every resource. Cancel active subscriptions or tasks when the screen goes away, dispose the WhisperEngine when the feature no longer needs it, and close a WhisperModelManager that owns its HTTP client.

Performance tips for real devices

Start small

Test the Tiny English model first for an English-only live experience. Larger models trade storage, memory, and inference time for accuracy. For multilingual recognition, select a multilingual GGML model rather than an .en model and set language to a language code or auto.

Choose a performance mode intentionally

The package provides three reusable modes for offline transcription:

responsive prioritizes latency with four threads, greedy best-of-one decoding, and timestamps disabled.
balanced preserves the standard defaults.
efficient uses two threads and less decoding work, but it does not guarantee faster results or lower battery use on every device.

Benchmark on physical low-, mid-, and high-end devices in release mode. Emulator results and debug builds are poor predictors of the experience your users will get.

Design around the first download

Tell users the model size before downloading, support retry, and offer a Wi-Fi-only choice if the model is large. The model manager supports resumable downloads, and its verified catalog protects you from silently accepting a corrupted or unexpected file.

Use VAD when silence is expensive

For long recordings with large silent sections, Silero voice activity detection (VAD) can identify speech ranges before transcription. The plugin supports both integrated and standalone VAD. It requires a separate Silero model, so evaluate the additional download and complexity against the performance gain for your use case.

Common mistakes to avoid

Appending live updates: replace provisional text instead, or users will see duplicated phrases.
Reloading the model for every recording: initialize once and reuse the engine.
Deleting a model while it is loaded: the engine reads from that path until disposal.
Testing only in debug mode: use a physical device and a release build for meaningful performance measurements.
Ignoring lifecycle events: stop or cancel microphone work and dispose native resources.
Choosing an English-only model for multilingual audio: model capability and the language option must agree.
Hiding the model download: communicate the size, progress, storage cost, and offline benefit.

Where to go next

Once the basic flow works, you can add segment-level editing, word highlighting, SRT or WebVTT export, transcription queues, translation, custom PCM streams, or Silero VAD. The package also exposes lower-level decoding controls when your product needs more than the convenient defaults.

On-device speech recognition is not simply a cloud feature moved into a phone. You have to design for model downloads, CPU and memory limits, provisional live text, permissions, and lifecycle cleanup. In return, you get a compelling user promise: fast, private speech-to-text that keeps working when the network does not.

Explore the latest package documentation and example app on pub.dev. If you need help integrating offline transcription, optimizing a Whisper model for mobile, or building a complete AI-powered Flutter experience, get in touch.

How to Implement Whisper Speech-to-Text in a Flutter Application