Arm NN

Software Developer Kit (SDK)

Arm NN is an inference engine for CPUs, GPUs and NPUs. It bridges the gap between existing NN frameworks and the underlying IP. It enables efficient translation of existing neural network frameworks, such as TensorFlow and Caffe, allowing them to run efficiently, without modification, across Arm Cortex-A CPUs, Arm Mali GPUs and Arm Ethos NPUs.

Arm NN is free of charge.

Download Arm NN SDK

About Arm NN SDK

Arm NN SDK is a set of open-source Linux software and tools that enables machine learning workloads on power-efficient devices. It provides a bridge between existing neural network frameworks and power-efficient Cortex-A CPUs, Arm Mali GPUs and Arm Ethos NPUs.

Arm NN SDK utilizes the Compute Library to target programmable cores, such as Cortex-A CPUs and Mali GPUs, as efficiently as possible. Arm NN does not provide support for Cortex-M CPUs.

The latest release supports Caffe, TensorFlow, TensorFlow Lite, and ONNX. Arm NN takes networks from these frameworks, translates them to the internal Arm NN format and then, through the Compute Library, deploys them efficiently on Cortex-A CPUs, and, if present, Mali GPUs such as the Mali-G71 and Mali-G72.

In September 2018, Arm donated Arm NN to the Linaro Machine Intelligence Initiative, where it is now developed fully in open source. To find out more, visit mlplatform.org.

 

 

Neural Network SDK Diagram.
Neural Network Diagram for Android.

Arm NN for Android

Also available is Arm NN for NNAPI, Google’s interface for accelerating neural networks on Android devices, made available in Android O. By default, NNAPI runs neural network workloads on the device’s CPU cores, but also provides a Hardware Abstraction Layer (HAL) that can target other processor types, such as GPUs. Arm NN for Android NNAPI provides this HAL for Mali GPUs. A further release adds support for Arm Ethos-N NPUs. 

Arm support for Android NNAPI gives >4x performance boost.

Learn more

Download Arm NN for Android sources. 

Download here

Arm NN performance relative to other NN frameworks

  • Arm NN open-source collaboration enables optimal third-party implementations
  • Deployed in multiple production devices (>250Mu)

Support for Cortex-M CPUs

Machine learning support for Cortex-M microcontrollers is provided by TensorFlow Lite Micro. Further optimization is available via CMSIS-NN, a collection of efficient neural network kernels developed to maximize the performance and minimize the memory footprint of neural networks on Cortex-M processor cores.

Download CMSIS-NN

Arm NN future roadmap

Future releases of Arm NN will enable support for other machine learning frameworks as inputs, and other forms of processor cores as targets. This includes processor cores and accelerators from Arm’s partners, assuming availability of suitable extensions.

Information on the Machine Learning Platform.

Webinar - Project Trillium: Optimizing ML Performance for any Application

Project Trillium is a suite of Arm IP designed to deliver scalable ML and neural network functionality at any point on the performance curve, from sensors, to mobile, and beyond. 

 

Find out more

Community Forums

Answered Inconsistent shareability domain on tlbi instructions
  • Cortex-A72
  • Cortex-A53
0 votes 496 views 3 replies Latest yesterday by josecm Answer this
Not answered Memory violation calling ATSAM3X8 0 votes 154 views 0 replies Started yesterday by ccandido Answer this
Suggested answer Instruction Count and Memory Access
  • CoreSight Debug and Trace
  • Musca-A
  • Cortex-M33
0 votes 260 views 2 replies Latest 2 days ago by Lica Answer this
Suggested answer Understanding interrupt latency and jitter in Cortex-M
  • Interrupt Handling
  • Cortex-M7
  • Cortex-M
  • Interrupt
0 votes 961 views 7 replies Latest 2 days ago by 42Bastian Schick Answer this
Suggested answer PSA Certified Devices
  • SoC Designer
  • Secure Microcontroller
  • Platform Security Architecture (PSA)
  • Security
  • Security System Software
  • Internet of Things (IoT)
0 votes 499 views 1 replies Latest 3 days ago by adrianlshaw Answer this
Not answered SWD: Cannot connect to MEM-AP on Cortex-M0 Designstart Eval
  • Cortex-M0
  • DesignStart
  • SWD
0 votes 77 views 0 replies Started 3 days ago by Bernhard Lang Answer this
Answered Inconsistent shareability domain on tlbi instructions Latest yesterday by josecm 3 replies 496 views
Not answered Memory violation calling ATSAM3X8 Started yesterday by ccandido 0 replies 154 views
Suggested answer Instruction Count and Memory Access Latest 2 days ago by Lica 2 replies 260 views
Suggested answer Understanding interrupt latency and jitter in Cortex-M Latest 2 days ago by 42Bastian Schick 7 replies 961 views
Suggested answer PSA Certified Devices Latest 3 days ago by adrianlshaw 1 replies 499 views
Not answered SWD: Cannot connect to MEM-AP on Cortex-M0 Designstart Eval Started 3 days ago by Bernhard Lang 0 replies 77 views