TensorFlow Lite for Microcontrollers on NXP Neutron

Browse source code on GitHub

Overview

A sample application that demonstrates how to run an image classification inference using the TFLM framework and the NXP Neutron NPU.

The sample application runs a MobileNet-based model that has been converted to a TFLite flatbuffer. The model is compiled into the application as a C header array. The Neutron NPU accelerates supported operators via a custom NEUTRON_GRAPH operator registered in the TFLM MicroMutableOpResolver.

Supported Platforms

This sample supports NXP platforms with the Neutron NPU:

  • MIMXRT798S (i.MX RT700 series)

  • MIMXRT2663 (i.MX RT2600 series)

  • FRDM-MCXN947 (MCXN series)

A platform-specific model is selected at build time via the CMakeLists.txt.

Generating Neutron-converted model

Follow the steps below to convert a quantized TFLite model for the Neutron NPU and generate the header files used by this sample.

Software requirements

  • eIQ Toolkit (available for Windows and Linux)

  • GCC, Python, and Git dependencies

Converting the model using neutron-compiler

After installing the eIQ neutron SDK, the neutron-compiler tool is located in the eIQ Neutron SDK directory (for example C:\eiq-neutron-sdk-windows-3.2.2\bin). Add it to your executable path.

To list the available Neutron targets:

neutron-compiler --show-targets

Some of the supported targets are:

  • imxrt700 : NXP i.MX RT700 MCU

  • imxrt2660 : NXP i.MX RT2660 Application Processor

  • mcxn94x : NXP MCX N94x MCU

To convert a quantized TFLite model for the Neutron NPU:

neutron-compiler --input model_quant.tflite --output model_npu.tflite \
--target imxrt700 --use-sequencer

The neutron-compiler takes a TFLite file as input and produces another TFLite file as output, where the operators supported by the Neutron NPU have been replaced by a NeutronGraph custom operator. Any layers that were not converted run on the Cortex-M33 core.

Converting to C array

Use xxd to convert the TFLite model and input/output data to C header files:

xxd -c 16 -i model_npu.tflite model_npu.tflite.h
xxd -c 16 -i input_data.bin input_data.h
xxd -c 16 -i output_data.bin output_data.h

Alternatively, the neutron-compiler can generate header files directly using the --dump-header-file-input and --dump-header-file-output arguments.

Synchronizing to this sample

Copy the generated header files to the appropriate model directory:

  • For RT700: src/models/rt700/model.hpp

  • For RT2660: src/models/rt2660/model.hpp

  • For MCXN: src/models/mcxn/model.hpp

Building and running

Add the tflite-micro module to your West manifest and pull it:

west config manifest.project-filter -- +tflite-micro
west update

Fetching NXP Neutron blobs

This sample requires NXP Neutron NPU driver and firmware binary blobs from the hal_nxp module. Fetch them before building:

west blobs fetch hal_nxp

Build the sample for an RT700 board:

# From the root of the zephyr repository
west build -b mimxrt798s/mimxrt700_evk samples/boards/nxp/tflm_neutron

Then flash the image:

west flash

Building for RT2660

# From the root of the zephyr repository
west build -b mimxrt2660_evk/mimxrt2663/cm85 samples/boards/nxp/tflm_neutron

RT2660 memory map

The MIMXRT2663 NPU cannot access TCM. The sample uses the following memory layout:

Region

Address

Attribute

Contents

SRAM01_NPU (512 KB)

0x22000000

non-cacheable

Tensor arena and Neutron DMA-shared data

SRAM2 (256 KB)

0x22080000

cacheable

Model copy

PSRAM (32 MB)

0x80000000

cacheable

System RAM: application data, BSS, heap, and stacks

The board overlay combines SRAM0 and SRAM1 into SRAM01_NPU, and the nocache.ld fragment places .data.neutron_driver in this region. At setup() the model is copied from flash into SRAM2 and the D-cache is flushed, so the NPU reads it from on-chip RAM: about 50 us faster per inference than a copy in PSRAM, and over 1 ms faster than reading it straight from the XIP flash.

Running a larger model

If the model does not fit in SRAM2 (256 KB), place the copy in PSRAM instead: remove the SRAM2_SECTION attribute from g_model_sram in src/main_functions.cpp so it becomes a plain .bss array, and keep the cleanCache_by_Addr() call. PSRAM is the system RAM on this board (zephyr,sram = &psram), so the linker allocates the array there with the rest of the image data. Do not place it in a dedicated psram linker section instead: the PSRAM and RAM regions share the same origin (0x80000000) and the linker does not check for overlaps between different memory regions, so such a section would be linked on top of .data, .bss and the stacks.

Keep the tensor arena and the driver data in SRAM01_NPU in every configuration; if necessary, increase kTensorArenaSize without exceeding the region’s 512 KB capacity.

Building for MCXN

# From the root of the zephyr repository
west build -b frdm_mcxn947/mcxn947 samples/boards/nxp/tflm_neutron

Sample output

The application prints the top 5 classification results with confidence percentages to the console:

=== TFLM NXP Neutron Starting ===

Inference #1
----------------------------------------
Running inference...
Inference complete

Top 5 Results:
----------------------------------------
1. ship                           99.61%
2. automobile                      0.00%
3. bird                            0.00%
4. cat                             0.00%
5. deer                            0.00%
----------------------------------------

Inference complete!