How to Implement Offline Speech-to-Text in Flutter with flutter_gemma

Speech-to-text does not have to send a user's voice to a remote API. With flutter_gemma, Flutter apps can run AI models locally, and its opt-in flutter_gemma_speech module adds on-device speech recognition through LiteRT and Dart FFI. Once the model and tokenizer are installed, recorded audio can be transcribed without a cloud speech service.

This tutorial builds a private, offline speech-to-text flow using flutter_gemma 1.5.9, flutter_gemma_speech 0.4.3, and the Moonshine Tiny model. We will register the speech backend, download the model with progress, record 16 kHz mono PCM, create a recognizer, transcribe the captured bytes, and clean up native resources correctly.

Quick answer: install both Flutter Gemma packages, register LiteRtSttBackend during application startup, install a Moonshine model and tokenizer with FlutterGemma.installStt(), then pass 16-bit, 16 kHz mono PCM bytes to SpeechRecognizer.transcribe().

Why use Flutter Gemma for speech-to-text?

Cloud transcription is convenient, but it introduces network latency, usage-based costs, and a privacy decision every time audio leaves the device. Local transcription can be a better fit for private notes, field apps, interviews, accessibility tools, offline workflows, and products that need predictable inference costs.

Flutter Gemma is broader than speech recognition. Its modular ecosystem supports on-device language models, embeddings, RAG, agents, and speech. The speech code is deliberately separated into flutter_gemma_speech, so apps that only need text inference do not ship the STT and TTS runtime unnecessarily.

There is an important distinction: Gemma itself is a language-model family, while the STT module uses a dedicated automatic speech recognition model. In the current stable documentation, Moonshine Tiny is the end-to-end STT option. Your app still uses the Flutter Gemma model-management and backend APIs, but a Gemma chat model is not required just to produce a transcript.

What we are building

1. Add the core, speech, and recording dependencies.
2. Register the LiteRT speech backend before the app starts.
3. Download and activate the Moonshine model and tokenizer.
4. Capture raw 16-bit PCM from the microphone.
5. Transcribe the audio locally and display the result.

STT currently works through FFI on Android, iOS, macOS, Windows, and Linux. The speech package exposes a web stub, but web STT currently throws UnsupportedError. If your Flutter app targets browsers, design a separate web fallback.

1. Add the Flutter packages

Run these commands from the root of your Flutter project:

flutter pub add flutter_gemma
flutter pub add flutter_gemma_speech
flutter pub add record

Import the APIs used by the transcription service:

import 'dart:async';
import 'dart:typed_data';

import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_gemma_speech/flutter_gemma_speech.dart';
import 'package:record/record.dart';

You do not need to add flutter_gemma_litertlm directly for this setup. flutter_gemma_speech depends on it and uses its shared native LiteRT bundle.

2. Configure microphone permission

Android

Add microphone permission to android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.RECORD_AUDIO" />

iOS

Add a clear purpose string to ios/Runner/Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>Microphone access is used to create private, on-device transcripts.</string>

Ask for permission when the user starts the feature, not immediately at launch. The record package's hasPermission() method can trigger the platform request while keeping the permission check close to the action that needs it.

3. Register the speech backend

Flutter Gemma uses opt-in backend providers. Register LiteRtSttBackend before requesting a recognizer. A good place is main(), before runApp():

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();

  await FlutterGemma.initialize(
    sttBackends: [LiteRtSttBackend()],
  );

  runApp(const MyApp());
}

Registering only the STT backend keeps the setup focused. If the same application later needs offline text-to-speech, add LiteRtTtsBackend() to the corresponding ttsBackends list.

4. Download and activate Moonshine Tiny

Speech recognition requires two files: the model and its tokenizer. The installation builder can load them from the network, Flutter assets, bundled native resources, or existing files. The network example below uses the same public sources as the official Flutter Gemma example.

class GemmaSpeechService {
  static const _modelUrl =
      'https://huggingface.co/litert-community/moonshine-tiny/'
      'resolve/main/moonshine_tiny_5s_f32.tflite';

  static const _tokenizerUrl =
      'https://huggingface.co/UsefulSensors/moonshine/'
      'resolve/main/ctranslate2/tiny/tokenizer.json';

  SpeechRecognizer? _recognizer;

  SpeechRecognizer get recognizer {
    final value = _recognizer;
    if (value == null) {
      throw StateError('GemmaSpeechService has not been initialized.');
    }
    return value;
  }

  Future<void> initialize({
    void Function(String file, int percent)? onProgress,
  }) async {
    await FlutterGemma.installStt()
        .modelFromNetwork(_modelUrl)
        .tokenizerFromNetwork(_tokenizerUrl)
        .ofType(SttModelType.moonshine)
        .withModelProgress(
          (percent) => onProgress?.call('model', percent),
        )
        .withTokenizerProgress(
          (percent) => onProgress?.call('tokenizer', percent),
        )
        .install();

    _recognizer = await FlutterGemma.getActiveStt();
  }

  Future<String> transcribe(Uint8List pcm16kMono) {
    return recognizer.transcribe(pcm16kMono);
  }

  Future<void> dispose() async {
    await _recognizer?.close();
    _recognizer = null;
  }
}

Installation is idempotent. If both files are already installed, Flutter Gemma skips their downloads and activates the existing STT model. The full-precision Moonshine Tiny model used here is approximately 109 MB, so show the download state and avoid hiding the storage cost from users.

The model and tokenizer progress values are separate. Label them independently instead of showing one bar that appears to jump from 100 percent back to zero when the second file begins.

5. Record 16 kHz mono PCM

SpeechRecognizer.transcribe() expects raw PCM bytes, not an MP3 file or a WAV file with a container header. Configure the recorder to stream signed 16-bit PCM at 16 kHz with one channel:

class PcmRecorder {
  final AudioRecorder _recorder = AudioRecorder();
  final List<int> _bytes = <int>[];

  StreamSubscription<Uint8List>? _subscription;
  Completer<void>? _streamFinished;

  Future<void> start() async {
    if (!await _recorder.hasPermission()) {
      throw StateError('Microphone permission was not granted.');
    }

    _bytes.clear();
    _streamFinished = Completer<void>();

    final stream = await _recorder.startStream(
      const RecordConfig(
        encoder: AudioEncoder.pcm16bits,
        sampleRate: 16000,
        numChannels: 1,
      ),
    );

    _subscription = stream.listen(
      _bytes.addAll,
      onDone: () => _streamFinished?.complete(),
      onError: (Object error, StackTrace stackTrace) {
        _streamFinished?.completeError(error, stackTrace);
      },
    );
  }

  Future<Uint8List> stop() async {
    await _recorder.stop();
    await _streamFinished?.future;
    await _subscription?.cancel();
    _subscription = null;
    return Uint8List.fromList(_bytes);
  }

  Future<void> dispose() async {
    await _subscription?.cancel();
    await _recorder.dispose();
  }
}

Keep recordings short for the example Moonshine model and prevent two recording or transcription operations from running at the same time. For longer dictation, segment the source into manageable chunks and design an overlap or stitching strategy so words at chunk boundaries are not lost.

6. Connect recording and transcription to the UI

The button handlers can now remain small. Initialize the model before enabling Record, then send the captured PCM to the recognizer when the user stops:

final speech = GemmaSpeechService();
final recorder = PcmRecorder();

Future<void> prepareSpeech() async {
  await speech.initialize(
    onProgress: (file, percent) {
      setState(() {
        downloadLabel = 'Downloading $file: $percent%';
      });
    },
  );

  setState(() => isReady = true);
}

Future<void> startRecording() async {
  await recorder.start();
  setState(() => isRecording = true);
}

Future<void> stopAndTranscribe() async {
  setState(() {
    isRecording = false;
    isTranscribing = true;
  });

  try {
    final pcm = await recorder.stop();
    final text = await speech.transcribe(pcm);
    setState(() => transcript = text);
  } finally {
    setState(() => isTranscribing = false);
  }
}

Dispose both services when the screen or feature owner is destroyed:

@override
void dispose() {
  recorder.dispose();
  speech.dispose();
  super.dispose();
}

Flutter's widget dispose() cannot be awaited. Start the asynchronous cleanup as shown, or move resource ownership into a longer-lived service with an explicit awaited shutdown method when your architecture requires confirmation that native resources have closed.

Production considerations

Make the model download a product decision

Show the approximate download size, current file, progress, retry state, and offline benefit. Let users wait for Wi-Fi if needed. For controlled deployments, you can use modelFromAsset(), modelFromFile(), or a private authenticated URL instead of the public network source.

Keep audio private beyond inference

Local inference prevents the transcription request from going to a cloud API, but your own code still controls the recording. Avoid leaving temporary audio behind, encrypt saved recordings when appropriate, and do not send transcripts to analytics or crash reports accidentally.

Test release builds on physical devices

Model load time, memory pressure, and inference speed vary substantially across devices. Test the slowest supported phone and each desktop target in release mode. Also test interrupted downloads, denied microphone permission, empty audio, backgrounding, and rapid repeated button taps.

Model the UI as explicit states

A robust screen should distinguish downloading, ready, recording, transcribing, complete, and error states. Disable conflicting actions and guard asynchronous gaps so a double tap cannot start two recorders or recognizers.

Common mistakes to avoid

• Installing only flutter_gemma: the current speech backend lives in the opt-in flutter_gemma_speech package.
• Forgetting backend registration: call FlutterGemma.initialize() with LiteRtSttBackend before requesting the recognizer.
• Passing compressed audio: convert input to raw signed 16-bit, 16 kHz mono PCM.
• Using the wrong model type: pair Moonshine files with SttModelType.moonshine.
• Expecting web support: Flutter Gemma supports web generally, but the current speech module's STT implementation is native-only.
• Ignoring cleanup: close the recognizer, stop recording, and dispose the recorder.

Frequently asked questions

Does flutter_gemma provide speech-to-text by itself?

The core package provides the shared model-management and STT interfaces. The working LiteRT speech backend is delivered through the separate flutter_gemma_speech package, so a speech-to-text app should add both dependencies.

Does this speech-to-text implementation work offline?

Yes. After the model and tokenizer are available on the device, transcription runs locally and does not require a cloud speech API. The initial model installation needs network access unless you provide the files as assets, bundled resources, or local files.

Do I need a Gemma LLM to transcribe audio?

No. The STT recognizer uses its own automatic speech recognition model. A Gemma language model is useful if you want to summarize, classify, correct, or chat about the transcript afterward, but it is not required for the transcription step.

Can flutter_gemma_speech transcribe MP3 or WAV directly?

The recognizer accepts raw PCM bytes. Record in the required format directly or decode and resample MP3, WAV, M4A, and other sources to signed 16-bit, 16 kHz mono PCM before transcription.

How is this different from whisper_cpp_flutter_plus?

Both approaches run speech recognition locally, but they use different runtimes, model formats, APIs, and platform matrices. Flutter Gemma's speech module integrates with its larger on-device AI ecosystem, while whisper_cpp_flutter_plus focuses specifically on whisper.cpp. See the Flutter Whisper speech-to-text guide for that implementation.

Where to go next

Once transcription works, you can pass the result into an on-device Gemma chat model for summarization, structured note extraction, commands, or a private voice assistant. The speech package also supports text-to-speech and a VoiceSession abstraction for an STT → LLM → TTS push-to-talk loop.

Review the latest Flutter Gemma package documentation and speech module documentation before shipping, because model support and platform capabilities are evolving quickly. If you need help designing an offline AI workflow or integrating speech recognition into an existing Flutter product, get in touch.