TensorFlow is an open-source ML framework developed by the Google Brain team. Its main focus is on building, training, and deploying ML models, especially deep learning models. It also supports numerical operations similar to NumPy but with distributed and parallelized support.
TensorFlow offers two main modes of execution:
- Eager Execution
- Graph Execution
Eager Execution
Eager execution is nothing but sequential execution of the code in the traditional way. Every line is executed and the values are assigned (if any) before the compiler jumps onto the next line.
Graph Execution
In this approach, a computational graph is constructed, and operations are added to the graph representing the computation flow for better performance.
Before execution, the graph is optimized (constant folding, dead code elimination etc) and can make use of hardware-specific library implementation like cuDNN for NVIDIA GPUs or Eigen for CPUs. All the computations can be parallelized across multiple cores or machines in a distributed manner.
The compiled graphs are platform-independent and can be exported to a different machine with different hardware configs without any code changes for execution.
Apart from this, Tensorflow also has XLA (Accelerated Linear Algebra) compiler support. It takes the model (from PyTorch, Tensorflow, etc)and optimizes it for high-performance execution across different hardware like GPUs, CPUs, and ML accelerators
Internally TF relies on highly optimized C++ code to run the computations. Each operation like addition, multiplication, convolution etc, has multiple implementations called “Kernels”
These are low-level, hardware-specific implementations of mathematical operations that form the core functionality of TensorFlow. They are written in C++ and optimized for different hardware — CPU, GPU, TPU.
Dear reader, Thank you so much for reading. If you liked what you read, please don’t forget to give claps.
Be the first to comment