TensorFlow Lite for Microcontrollers on NXP Neutron
Overview
A sample application that demonstrates how to run an image classification inference using the TFLM framework and the NXP Neutron NPU.
The sample application runs a MobileNet-based model that has been converted to a
TFLite flatbuffer. The model is compiled into the application as a C header
array. The Neutron NPU accelerates supported operators via a custom
NEUTRON_GRAPH operator registered in the TFLM MicroMutableOpResolver.
Supported Platforms
This sample supports NXP platforms with the Neutron NPU:
MIMXRT798S (i.MX RT700 series)
MIMXRT2663 (i.MX RT2600 series)
FRDM-MCXN947 (MCXN series)
A platform-specific model is selected at build time via the CMakeLists.txt.
Generating Neutron-converted model
Follow the steps below to convert a quantized TFLite model for the Neutron NPU and generate the header files used by this sample.
Software requirements
eIQ Toolkit (available for Windows and Linux)
GCC, Python, and Git dependencies
Converting the model using neutron-compiler
After installing the eIQ neutron SDK, the neutron-compiler tool is located in
the eIQ Neutron SDK directory (for example
C:\eiq-neutron-sdk-windows-3.2.2\bin). Add it
to your executable path.
To list the available Neutron targets:
neutron-compiler --show-targets
Some of the supported targets are:
imxrt700: NXP i.MX RT700 MCUimxrt2660: NXP i.MX RT2660 Application Processormcxn94x: NXP MCX N94x MCU
To convert a quantized TFLite model for the Neutron NPU:
neutron-compiler --input model_quant.tflite --output model_npu.tflite \
--target imxrt700 --use-sequencer
The neutron-compiler takes a TFLite file as input and produces another
TFLite file as output, where the operators supported by the Neutron NPU have
been replaced by a NeutronGraph custom operator. Any layers that were not
converted run on the Cortex-M33 core.
Converting to C array
Use xxd to convert the TFLite model and input/output data to C header files:
xxd -c 16 -i model_npu.tflite model_npu.tflite.h
xxd -c 16 -i input_data.bin input_data.h
xxd -c 16 -i output_data.bin output_data.h
Alternatively, the neutron-compiler can generate header files directly
using the --dump-header-file-input and --dump-header-file-output
arguments.
Synchronizing to this sample
Copy the generated header files to the appropriate model directory:
For RT700:
src/models/rt700/model.hppFor RT2660:
src/models/rt2660/model.hppFor MCXN:
src/models/mcxn/model.hpp
Building and running
Add the tflite-micro module to your West manifest and pull it:
west config manifest.project-filter -- +tflite-micro
west update
Fetching NXP Neutron blobs
This sample requires NXP Neutron NPU driver and firmware binary blobs from the
hal_nxp module. Fetch them before building:
west blobs fetch hal_nxp
Build the sample for an RT700 board:
# From the root of the zephyr repository
west build -b mimxrt798s/mimxrt700_evk samples/boards/nxp/tflm_neutron
Then flash the image:
west flash
Building for RT2660
# From the root of the zephyr repository
west build -b mimxrt2660_evk/mimxrt2663/cm85 samples/boards/nxp/tflm_neutron
RT2660 memory map
The MIMXRT2663 NPU cannot access TCM. The sample uses the following memory layout:
Region |
Address |
Attribute |
Contents |
|---|---|---|---|
|
|
non-cacheable |
Tensor arena and Neutron DMA-shared data |
|
|
cacheable |
Model copy |
|
|
cacheable |
System RAM: application data, BSS, heap, and stacks |
The board overlay combines SRAM0 and SRAM1 into SRAM01_NPU, and the
nocache.ld fragment places .data.neutron_driver in this region.
At setup() the model is copied from flash into SRAM2 and the D-cache is
flushed, so the NPU reads it from on-chip RAM: about 50 us faster per
inference than a copy in PSRAM, and over 1 ms faster than reading it straight
from the XIP flash.
Running a larger model
If the model does not fit in SRAM2 (256 KB), place the copy in PSRAM instead:
remove the SRAM2_SECTION attribute from g_model_sram in
src/main_functions.cpp so it becomes a plain .bss array, and keep the
cleanCache_by_Addr() call. PSRAM is the system RAM on this board
(zephyr,sram = &psram), so the linker allocates the array there with the
rest of the image data. Do not place it in a dedicated psram linker
section instead: the PSRAM and RAM regions share the same origin
(0x80000000) and the linker does not check for overlaps between different
memory regions, so such a section would be linked on top of .data, .bss
and the stacks.
Keep the tensor arena and the driver data in SRAM01_NPU in every
configuration; if necessary, increase kTensorArenaSize without exceeding
the region’s 512 KB capacity.
Building for MCXN
# From the root of the zephyr repository
west build -b frdm_mcxn947/mcxn947 samples/boards/nxp/tflm_neutron
Sample output
The application prints the top 5 classification results with confidence percentages to the console:
=== TFLM NXP Neutron Starting ===
Inference #1
----------------------------------------
Running inference...
Inference complete
Top 5 Results:
----------------------------------------
1. ship 99.61%
2. automobile 0.00%
3. bird 0.00%
4. cat 0.00%
5. deer 0.00%
----------------------------------------
Inference complete!