A few weeks ago at the PyTorch conference, the team released ExecuTorch, which is a tool that runs PyTorch models on devices like smartphones, wearables, and embedded systems.
Four years ago, PyTorch Mobile was introduced for a similar purpose but ExecuTorch uses a significantly smaller memory size and a dynamic memory footprint resulting in superior performance and portability compared to PyTorch Mobile.
ExecuTorch does not rely on TorchScript, and instead leverages PyTorch 2 compiler and export functionality for on-device execution of PyTorch models. It isn’t just a rewrite of PyTorch Mobile; it leverages the PyTorch 2 compiler, which is a significant advancement. Not restricted to mobiles, the performance includes hardware capabilities of CPUs, NPUs, and DSPs.
The team said that it was challenging for devices to adapt for TorchScript compatibility, so newer models increasingly opt for the PyTorch 2 compiler for improved performance which means the already unpopular Pytorch Mobile will see fewer users and an automatic support for ExecuTorch.
On the other hand, TensorFlow Lite which was released in 2017, is also a tool that converts TensorFlow models into a more efficient format that can be run on edge devices. It does this by using a compiler called the TensorFlow Lite Converter to convert the model into a flatbuffer format that can be executed by a lightweight runtime.
Now to compare the two systems, it is imperative to also understand how the frameworks are used in machine learning.
TensorFlow Lite vs ExecuTorch
ExecuTorch and TensorFlow Lite are both tools designed for deploying machine learning models on edge devices, such as smartphones, wearables, and embedded systems. However, these tools exhibit significant differences because of the frameworks they’re built on.
PyTorch is undeniably more favourable than Tensorflow and most industry experts and researchers prefer it over its more cumbersome counterpart. While PyTorch Mobile was limited in its compatibility in edge devices the introduction of ExecuTorch has filled that gap.
ExecuTorch is built on the foundation of PyTorch 2.0. This is more popular than Torch Script as it is user-friendly and the compiler is supported by a wider range of devices than TensorFlow. One of its standout features is its compatibility with Android devices, making it an attractive option for those new to machine learning deployment or in need of Android device support.
In contrast, TensorFlow Lite, based on the TensorFlow framework, has established itself as a reliable choice known for its exceptional performance and efficiency within TensorFlow’s framework. To improve its adaptability TensorFlow updated the deployment of LLM models on Android.
ExecuTorch is lauded for its user-friendly nature, extensive model compatibility, and specific support for Android devices. In contrast, TensorFlow Lite, grounded in TensorFlow, excels in performance and boasts compatibility with a wide range of devices.
ExecuTorch is a practical choice for broad model compatibility, or Android device support. On the other hand, TensorFlow Lite may be the more suitable option for if your priority is top-tier performance on device.
A step up from PyTorch Mobile
ExecuTorch surpasses PyTorch Mobile in several key areas. Firstly, it demonstrates superior performance and portability due to its smaller memory size and dynamic memory footprint. The compiler used by ExecuTorch optimises the model for the target device, and the export functionality generates a smaller model file. It uses a technique called memory allocation on demand, which means it only allocates the memory it needs when it needs it.
This is in contrast to PyTorch Mobile, which has a static memory footprint. This means that PyTorch Mobile allocates all of the memory it needs upfront, even if it doesn’t need it all right away. This can lead to performance problems and memory crashes on devices with limited memory.
ExecuTorch also excels in ease of use. Unlike PyTorch Mobile, it doesn’t rely on TorchScript, a potentially complex compiler that requires changes to model code. Instead, Executorch utilises the PyTorch 2 compiler and export functionality, simplifying the deployment process.
In addition to these core advantages, ExecuTorch is actively maintained and updated, in contrast to the stagnation of PyTorch Mobile’s development. Its larger user and developer community provides a valuable support network.
Furthermore, ExecuTorch seamlessly integrates with the PyTorch ecosystem, ensuring consistency in tools and libraries for model development and deployment.
Be the first to comment