You may have come across TensorFlow Lite while going through Edge AI development boards or AI acceleration projects.
TensorFlow Lite is a framework of software packages that enables ML training locally on the hardware. This on-device processing and computing allow developers to run their models on targeted hardware. The hardware includes development boards, hardware modules, embedded and IoT devices.
Overview of the TensorFlow Lite Framework
TensorFlow is a popular term in deep learning, as many ML developers use this framework for various use cases. It provides the ease of implementing machine learning models and inferences for AI applications.
But TensorFlow Lite is a deep learning framework for local inference, specifically for the low computational hardware. It allows on-device machine learning by assisting developers in running their models on compatible hardware and IoT devices.
A developer needs to select a suitable model depending on the use case. The framework also gives an option of retraining the existing model on a custom dataset as well. As TensorFlow’s protocol buffer model comes with a large size and requires advanced computational power, thus it enables the conversion of the TensorFlow model to the TensorFlow Lite model.
The customization of optimizing and quantizing parameters allows the reduction in model size and latency.
Apart from the latency and size benefits of TensorFlow Lite, the framework provides the security of data as training occurs locally on the device. Additionally, there is no need for internet connectivity. Thus, the deployment of applications is not restricted to specific areas with connectivity.
These factors ultimately reduce the power consumption load on the device by eliminating the connectivity factor and increasing the efficiency of the deep learning inference.
Models of the TensorFlow Lite framework exist in a cross-platform format known as FlatBuffers. It is a serialization library that stores hierarchical data in a flat binary buffer so that direct access is possible without unpacking. You can also observe the “.tflite” extension for the TensorFlow Lite models. This technique of representation allows optimizations in computations and reduces the memory requirements. Hence, making it much better than TensorFlow models
TinyML on TensorFlow Lite Micro
As TensorFlow Lite is compatible with various platforms for Edge AI applications, the need of further converging the library was necessary. Hence, the organization came up with a subset library of the TensorFlow Lite, known as TensorFlow Lite Micro. TensorFlow Lite Micro specifically runs machine learning models on microcontrollers locally with minimum memory requirements of around a few kilobytes.
The core runtime of the procedure integrates with 16KB on an Arm Cortex M3 and can work on various models. The framework does not require additional OS support or other high-level language libraries as dependencies for running the inference on the device.
The development of the TensorFlow Lite Micro roots to C++ 11, which needs 32-bit architecture for compatibility. Talking more about the architectures, the library works fine on a robust range of processors based on the Arm Cortex-M Series architecture to other design architectures like ESP32.
Work Flow for TensorFlow Lite Micro Use Cases
The training process of the neural network requires high computational hardware. Thus, it is trained on the general TensorFlow model. However, training is only required if a custom dataset fits a deep learning model, whereas pre-trained models on the framework can also be used for the applications.
Assuming a custom use case with the application-specific dataset, the user trains the model on the general TensorFlow framework with high processing capacity and architecture. Once the training is over, the model evaluation using testing techniques verify the accuracy and reliability of the model. Further, the process is followed by converting the TensorFlow model to hardware compatible TensorFlow Lite model in the .tflite format.
The .tflite format is a flat buffer file common to the TensorFlow Lite framework and compatible hardware. The model can further be used for inference training on the real-time data received on the model. The inference training optimized the models for robust use cases. Hence, the option of inference training is crucial for edge AI applications.
Most of the microcontroller’s firmware do not support the native filesystem for directly embedding the flat buffer format of the TensorFlow Lite model. Hence, the conversion of the .tflite file is necessary to an array structure format, which is compatible with the microcontrollers.
Including the program in the C array followed by normal compilation is an easy technique for such conversion. The resulting format acts as a source file and consists of a character array compatible with the microcontrollers.
Devices Supporting TensorFlow Lite Micro
TensorFlow Lite is suitable for powerful devices, but it comes with the drawback of the larger workload on the processor. Although the TensorFlow Lite Micro has small size files prone to underfitting, optimising the file size that fits the memory can significantly improve output for low power and low processing hardware such as microcontrollers.
Here is the list of development boards from the official TensorFlow documentation that supports TensorFlow Lite Micro:
- Arduino Nano 33 BLE Sense
- SparkFun Edge
- STM32F746 Discovery kit
- Adafruit EdgeBadge
- Adafruit TensorFlow Lite for Microcontrollers Kit
- Adafruit Circuit Playground Bluefruit
- Espressif ESP32-DevKitC
- Espressif ESP-EYE
- Wio Terminal: ATSAMD51
- Himax WE-I Plus EVB Endpoint AI Development Board
The TensorFlow Lite Micro is also available as an Arduino library for expanded support for microcontrollers. It can also build projects for hardware development environments similar to Mbed.
TensorFlow Lite Offers A Lot
TensorFlow Lite deep learning framework opens up the possibilities to a number of edge AI applications. As the framework is open-source for AI enthusiasts, the community support makes it even more popular for machine learning use cases. The overall platform of TensorFlow Lite enhances the environment for the growth of edge applications for embedded and IoT devices
Additionally, there are various examples for beginners to assist them with the hands-on use cases on the framework. Some of these examples include person detection depending on the data collected by the image sensor of the development board and the standard hello world program for all the development boards. The examples also include applications like gesture detection and speech recognition for specific development boards as well.
For more information on the TensorFlow Lite and TensorFlow Lite Micro, you can visit the official documentation page of the organization. There are a lot of conceptual as well as tutorial sections for a better understanding of the framework.
About The Author