Adding Google Tensor NPU Support to Flutter Edge AI: An Engineering Case Study

In October 2026, my contribution adding Google Tensor NPU support to Flutter Edge AI was merged and released. The final integration uses LiteRT-LM bundle metadata to select a compatible NPU vendor, isolates the Google dispatch library from other vendor stacks, and reaches the device-provided Pixel runtime without shipping that proprietary runtime in the package.

The released combination is flutter_edge_ai 2.4.1, flutter_edge_ai_litertlm 1.15.0, and native runtime native-v0.18.0-f. This article explains the engineering path from my original PR #670 to the production shape refined through related maintainer work. It is an implementation case study, not a performance claim: I did not establish latency, throughput, battery, or thermal improvements.

Why on-device AI needs hardware-aware execution

“On-device” describes where inference happens, not how it runs. A phone may offer CPU, GPU, and NPU execution, but those backends are not interchangeable. A generic model can often be planned for GPU and then CPU. An NPU-compiled bundle can contain accelerator-specific sections that only the matching vendor stack understands.

That makes backend selection a compatibility decision before it becomes an optimization decision. If a Tensor-compiled model is sent to a CPU or GPU path, retrying does not make the model portable. The correct behavior is to reject the mismatch clearly, rather than report a fallback that cannot execute the compiled sections.

Understanding Flutter Edge AI and LiteRT-LM

Flutter Edge AI provides the application-facing model installation and inference APIs. Inference engines are separate packages; flutter_edge_ai_litertlm supplies the LiteRT-LM engine for .litertlm bundles. Applications register that engine once, install a model, and then ask the shared API for an active model.

Google Tensor support fits behind the existing PreferredBackend.npu abstraction. It did not require a public googleTensor backend enum. The model metadata and the runtime environment determine which NPU vendor is valid, while activeBackend reports what the engine initialized.

Release component	Version	Role
flutter_edge_ai	2.4.1	Public model and backend API
flutter_edge_ai_litertlm	1.15.0	LiteRT-LM engine and Tensor integration
Native runtime	native-v0.18.0-f	Released native libraries, including the Tensor dispatch bridge

The original problem: Google Tensor NPU dispatch

My starting point was straightforward to describe: Flutter Edge AI already exposed an NPU backend, but a Google Tensor model bundle could not reach the Tensor NPU through the Android LiteRT-LM integration. The missing path crossed Dart planning, Android packaging, native dispatch discovery, and a system runtime supplied by the Pixel device.

The first implementation in PR #670 added Google Tensor as another NPU vendor, built the open-source dispatch bridge from the repository's pinned LiteRT sources, declared the Pixel runtime libraries, and added regression coverage. Review then aligned the contribution with newer bundle inspection and centralized vendor architecture that had landed while the work was in progress.

Model metadata and NPU compatibility

Inspecting the bundle before engine creation

PR #674 introduced readLiteRtLmBundleInfo. It reads the container header and compiled-section stamps without reading the model weights. That lets the engine reject incomplete files and invalid backend requests before creating an engine.

final info = await readLiteRtLmBundleInfo(modelPath);

if (!info.isComplete) {
  throw StateError('The LiteRT-LM bundle is incomplete.');
}

print('NPU compiled: ${info.isCompiledForNpu}');
print('NPU target: ${info.npuTarget}');

This is a real native API in flutter_edge_ai_litertlm 1.15.0; it is not available on the web. For normal loading, applications do not need to duplicate the planner—the engine performs the inspection itself. The API is useful when an app wants to explain compatibility before loading.

Vendor-specific execution

A compatible Tensor G6 bundle uses a filename such as …_Google_Tensor_G6.litertlm, but the filename is guidance rather than the source of truth. The embedded LiteRT stamp identifies the vendor and SoC target. Flutter Edge AI maps that information to its internal Google Tensor vendor entry and confirms that the host and enabled native stack can serve it.

Why an NPU bundle cannot fall back to CPU or GPU

A Tensor-compiled bundle is planned for that NPU only. An explicit CPU or GPU request is refused, and an unavailable Tensor stack fails instead of silently retrying another backend. Fallback still makes sense for generic bundles, but not for vendor-compiled sections. This distinction prevents a misleading “fallback succeeded” result when the requested model cannot run there at all.

Integrating Google Tensor into the shared NPU architecture

Flutter Application→ Flutter Edge AI→ LiteRT-LM Engine→ Model Bundle Inspection→ NPU Vendor Resolution→ Compatibility Validation→ Google Tensor Dispatch Bridge→ Pixel System Runtime→ Tensor NPU Inference

Vendor detection and dispatch isolation

PR #683 centralized per-vendor answers in one internal NPU table. My integration was rebased onto that design: Google Tensor became one vendor entry instead of a parallel routing mechanism. The same table connects the bundle stamp, supported ABI, build opt-in, dispatch library, and Android system libraries.

Isolation matters because LiteRT loads a dispatch from a directory and the choice is effectively process-wide. If libraries for multiple vendors are presented together, discovery order can select the wrong one. The final architecture stages only the resolved vendor's dispatch libraries into the directory handed to LiteRT and refuses a request that mixes NPU vendors in one load.

Model-aware execution planning

The plan combines what the bundle says with what the build and device provide. A Tensor stamp is supported only on Android arm64, the app must opt in, and the device probe follows LiteRT's own Google Tensor rule before attempting to open the system runtime. The Tensor generation encoded by the bundle is ultimately checked by LiteRT when the engine is created.

Native build and Android packaging

The dispatch bridge and Pixel system runtime are different libraries

• libLiteRtDispatch_GoogleTensor.so is the native dispatch bridge built from the pinned LiteRT sources and shipped through Flutter Edge AI's native archive.
• libedgetpu_litert.so is the Tensor system runtime provided by a compatible Pixel device. The package declares and probes it; it does not redistribute it.

Keeping that boundary explicit avoids both a packaging mistake and an attribution mistake. The project owns how it builds and stages the bridge. The operating system owns the device runtime that actually talks to the Tensor accelerator.

Source-based compilation and build opt-in

The project does not accept an opaque prebuilt bridge in source control. The Android build creates it from pinned LiteRT sources, CI verifies the native archive manifest, and the Dart build hook checks the released artifact. The follow-up PR #686 also made Tensor support opt-in so ordinary Android apps do not carry or require the dispatch.

How to enable Google Tensor NPU support in Flutter

Add the released core and LiteRT-LM packages:

dependencies:
  flutter_edge_ai: ^2.4.1
  flutter_edge_ai_litertlm: ^1.15.0

Enable the Tensor dispatch in the application pubspec.yaml:

hooks:
  user_defines:
    flutter_edge_ai_litertlm:
      google_tensor_npu: true

Register the engine once during startup:

import 'package:flutter/widgets.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_litertlm/flutter_edge_ai_litertlm.dart';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await FlutterEdgeAi.initialize(
    inferenceEngines: const [LiteRtLmEngine()],
  );
  runApp(const MyApp());
}

Install a compatible …_Google_Tensor_G6.litertlm file. The explicit file type matters: the installer otherwise defaults to the MediaPipe task format.

await FlutterEdgeAi.installModel(
  modelType: ModelType.gemmaIt,
  fileType: ModelFileType.litertlm,
).fromFile(modelPath).install();

Then request the shared NPU backend and verify the initialized backend:

final model = await FlutterEdgeAi.getActiveModel(
  preferredBackend: PreferredBackend.npu,
);

if (model.activeBackend != PreferredBackend.npu) {
  throw StateError('The model did not initialize on the NPU.');
}

For a Tensor-compiled bundle, the planner already enforces NPU-only execution. Checking activeBackend is still useful as an application-level assertion and for diagnostics.

Testing on physical Tensor hardware

Automated validation

The final PR #670 validation reported 187 targeted tests, 706 LiteRT-LM tests, and green workspace testing across 18 packages. Those counts cover regression and integration behavior; they are not performance benchmarks. Native Android compilation, APK packaging, alignment, and single-vendor dispatch isolation were also checked.

Pixel device and release-signed validation

I validated the integration on a physical Pixel 11 Pro with Tensor G6. The pre-release debug run completed Tensor NPU inference and produced native logs showing Google dispatch loading, Darwinn firmware activity, and EdgeTPU command execution. After the official packages were published, I repeated the work with a clean, release-signed APK: NPU inference succeeded after a cold restart, ordinary GPU inference also passed, and teardown and application restart completed successfully.

The final release-signed run did not reproduce an earlier EdgeTPU allow-list warning observed during debug teardown. That is useful evidence for this tested configuration, but it is not proof of universal device compatibility.

Known limitations

• Tensor G5 hardware was not physically validated in this work.
• The verified NPU model used a Tensor G6-specific bundle; vendor bundles are not portable across arbitrary SoCs.
• No controlled latency, throughput, battery, or thermal study was performed.
• Applications should surface clear compatibility errors instead of promising an automatic CPU/GPU fallback for an NPU-compiled model.

Lessons from open-source collaboration

The strongest part of this contribution was not preserving my first design. It was adapting the work as the project evolved. DenisovAV pointed the implementation toward bundle-driven planning, strict handling for compiled models, source-built native artifacts, and then the centralized vendor table. Those changes removed duplicate logic and made the Tensor path consistent with Qualcomm and Intel without erasing their platform differences.

Attribution matters here. I authored the original Google Tensor contribution in PR #670. DenisovAV and other Flutter Edge AI contributors designed, maintained, reviewed, and extended the surrounding ecosystem, including PRs #674, #683, #686, and #688. This is exactly why mature open-source work is often better described as a sequence of compatible improvements than as one isolated patch.

Frequently asked questions

Does enabling google_tensor_npu make every model use the Tensor NPU?

No. The flag makes the dispatch bridge available to the build. The model bundle still needs compatible Google Tensor metadata, and the device must pass the runtime checks.

Can a Tensor G6 bundle fall back to the GPU?

No. A bundle compiled for the Tensor NPU contains vendor-specific sections. Use a separate generic CPU/GPU bundle when your product needs a fallback path.

Is the Pixel system runtime bundled in the APK?

No. Flutter Edge AI ships the dispatch bridge; a compatible Pixel system supplies libedgetpu_litert.so.

Do these results prove that NPU inference is faster or more efficient?

No. The validation establishes functional execution and packaging for the tested hardware and software versions. It does not establish comparative performance, battery savings, or thermal improvement.

Conclusion and developer resources

Google Tensor NPU support in Flutter Edge AI is now a released, model-aware path rather than a device-name shortcut. A Tensor bundle identifies its target, the planner validates compatibility, the build opt-in supplies the dispatch bridge, and the Pixel system supplies the runtime. That separation is what makes the integration understandable and maintainable.

• My original Google Tensor contribution, PR #670
• Flutter Edge AI repository and project documentation
• flutter_edge_ai on pub.dev and flutter_edge_ai_litertlm on pub.dev
• Related reading: hybrid on-device and cloud routing, offline speech-to-text with Flutter Gemma, and offline Whisper speech-to-text in Flutter
• Gurwinder Singh on GitHub