Introduction to Tensors in Neural Networks
In Deep Learning, we store our data in the form of Tensors.
A Tensor is a multidimensional NumPy array in which we store our input data. It is a container for data - almost always numerical data. So, it’s a container for numbers.
In general, all current machine-learning systems use tensors as their basic data structure.
For example, we have matrices which are 2D tensors. Tensors can also be defined as a generalization of matrices to an arbitrary number of dimensions.
Different Types of Tensors -
Scalars (0D Tensors)
A tensor that contains only one number is called a scalar (or scalar-tensor, or 0-dimensional tensor, or 0D tensor).
In Numpy, a float32 or float64 number is a scalar-tensor (or scalar array). You can display the number of axes of a Numpy tensor via the ndim attribute; a scalar-tensor has 0 axes (ndim == 0).
The number of axes of a tensor is also called it's rank.
>>> import numpy as np >>> x = np.array(15) >>> x array(15) >>> x.ndim 0
Vectors (1D tensors)
An array of numbers is called a vector, or 1D tensor. A 1D tensor is said to have exactly one axis.
>>> x = np.array([15, 2, 7, 4]) >>> x array([15, 2, 7, 4]) >>> x.ndim 1
This vector has four entries and so is called a 4-dimensional vector.
Note: Don’t confuse a 4D vector with a 4D tensor! A 4D vector has only one axis and has five dimensions along its axis, whereas a 4D tensor has five axes (and may have any number of dimensions along each axis).
Matrices (2D tensors)
An array of vectors is a matrix or a 2D tensor. A matrix has two axes (often referred to as rows and columns). You can visually interpret a matrix as a rectangular grid of numbers.
>>> x = np.array([[5, 78, 2, 34, 0], [6, 79, 3, 35, 1], [7, 80, 4, 36, 2]]) >>> x.ndim 2
The entries from the first axis are called the rows, and the entries from the second axis are called the columns.