Important Terms in Machine Learning | by Amit Kumar | Nov, 2023

Artificial Intelligence

• Originated: 1950
• This is Simulated Intelligence in Machines

Machine Learning

• Originated: 1960
• This is Machine making decisions without being programmed

Deep Learning

• Originated: 1970
• This is machine Using Neural networks to solve complex problems

Without an activation function(eg: relu, softmax), Dense layer would always perform linear operations(a dot product, an addition) on input tensors.

Adding activation function to a layer introduces non-linearity into the model, allowing it to learn more complex relationships between the input and output data.

Non-linear activation functions such as ReLU, Sigmoid, and Tanh can help the model to better fit the training data and make more accurate predictions on new data.

Miniconda is the recommended approach for installing TensorFlow with GPU supportIt creates a separate environment to avoid changing any installed software in your system.

This is also the easiest way to install the required software especially for the GPU setup.

Tells how parameters should be tuned to make model produce expected output. Presently ML model is showing loss function.

Goal of gradient descent is to identify the model parameters that provide the maximum accuracy.

Gradient-descent process must be based on a single scalar loss value; so, for multiloss networks, all losses are combined (via averaging) into a single scalar quantity.

Means that the model performs well on the training data, but it does not generalize well(ie produces good results on real world/unseen data), because there is too of much unnecessary data(noise) in training data.

Regularization: Constraining a model to make it simpler and reduce the risk of overfitting.

We should use a training data set that is representative of the cases we want model to predict.

if the sample is too small, you will have sampling noise (i.e., nonrepresentative data as a result of chance), but even very large samples can be nonrepresentative if the sampling method is flawed. This is called sampling bias.

This is Microsoft Cognitive Toolkit (CNTK) backend, plugged with keras.

Library(in Python) which provides functions/APIs to build deep-learning models. Different backends can be plugged with keras

            Keras
Tensorflow / Theano / CNTK
CUDA           BLAS,Eigen
GPU            CPU

Neural network is created by cascading multiple layers.

Function that compares expected and actual values. Measures how well the neural network models the training data. Loss function should be minimum.

 Loss function = (Actual O/P) - (Expected output)

Neural network is created by cascading multiple layers.

Function that compares expected and actual values. Measures how well the neural network models the training data. Loss function should be minimum.

Loss function = (Actual O/P) - (Expected output)

This is matrix(as in maths).

Multi-dimensional numpy arrays used to store numbers during computation.

  /////////// 2-D Tensor //////////////
Shape: (2,3)
Dimension/Rank/Axis/Ndim: 2
| 1,2,3 |
| 4,5,6 |
/////////// 3-D Tensor example. Packing 2-D matrices //////////////
Shape: (3,3,5)
Dimension/Rank/Axis/Ndim: 3
>> x = np.array([[
[5, 78, 2, 34, 0],
[6, 79, 3, 35, 1],
[7, 80, 4, 36, 2]
],
[
[5, 78, 2, 34, 0],
[6, 79, 3, 35, 1],
[7, 80, 4, 36, 2]
],
[
[5, 78, 2, 34, 0],
[6, 79, 3, 35, 1],
[7, 80, 4, 36, 2]
]])

Tensor Terms

Data types(dtype)

Type of the data contained in the tensor; for instance, a tensor’s type could be float32, uint8, float64, and so on.

String tensors don’t exist in Numpy (or in most other libraries), because tensors are preallocated contiguous memory segments, and strings, being variable length.

Rank/Axis/Dimension/ndimDimension of matrix

For instance, a 3D tensor has three axes, and a matrix has two axes. This is also called the tensor’s ndim in Python libraries such as Numpy.

Shape

Tells how many size tensor has along each axis.For instance, the previous matrix example has shape (3, 5), and the 3D tensor example has shape (3, 3, 5).

This is ML Open source library(EXPOSING APIs) for numerical computation and large-scale ML supports CPUs & GPUs.

Python Front-end APIs & backend written in c++ for high performance.

  //Install conda https://docs.conda.io/projects/miniconda/en/latest/
C:\Users\amitk\source\repos\Python> mkdir venv_ml1
C:\Users\amitk\source\repos\Python> cd venv_ml1
C:\Users\amitk\source\repos\Python\venv_ml1>"c:\Users\amitk\miniconda3\Scripts\activate" venv_ml1
//Env is created here: C:\Users\amitk\miniconda3\envs
(venv_ml1) C:\Users\amitk\source\repos\Python\venv_ml1>
(venv_ml1) C:\Users\amitk\source\repos\Python\venv_ml1>"c:\Users\amitk\miniconda3\condabin\deactivate.bat"  //deactivate
(venv_ml1) C:\Users\amitk\source\repos\Python\venv_ml1>pip install tensorflow
Downloading tensorflow-2.6.2-cp36-cp36m-win_amd64.whl (423.3 MB)

Does not produces good results on traning data.

Variance is the tendency to learn random things unrelated to the real signal