Tensors are multi-dimensional data structures commonly used in mathematics, physics, and deep learning. In deep learning, tensors play a fundamental role in representing data and carrying out computations in neural networks. Here's a checklist of terminology related to tensors with examples:
Scalar (0D Tensor):
A single numerical value.
Example: 5
Vector (1D Tensor):
An ordered array of numbers.
Example: [1, 2, 3]
Matrix (2D Tensor):
A 2D array of numbers arranged in rows and columns.
Example:
lua
Copy code
[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]
Tensor (N-Dimensional Tensor):
A generalization of matrices to more than two dimensions.
Example: A color image can be represented as a 3D tensor with dimensions (height, width, channels).
Shape:
The dimensions of a tensor, represented as a tuple or list.
Example: A matrix with shape (3, 4) has 3 rows and 4 columns.
Rank (or Order):
The number of dimensions in a tensor.
Example: A rank-2 tensor is a matrix, while a rank-3 tensor is a 3D array.
Element-wise Operation:
Operations performed on each element of a tensor separately.
Example: Element-wise addition of two matrices, where each corresponding element is added together.
Broadcasting:
Implicitly expanding the dimensions of tensors to make them compatible for element-wise operations.
Example: Adding a scalar to a matrix, where the scalar is broadcast to match the shape of the matrix.
Transpose:
Changing the dimensions of a tensor by switching its rows and columns.
Example: Transposing a matrix A changes the rows of A to columns and vice versa.
Reshape:
Changing the shape of a tensor while keeping the number of elements constant.
Example: Reshaping a 2x3 matrix into a 3x2 matrix.
Concatenation:
Combining tensors along a specified axis (dimension).
Example: Concatenating two 2D tensors along the rows or columns.
Tensor Product (or Outer Product):
A mathematical operation that combines two tensors to create a new tensor.
Example: The tensor product of two vectors results in a matrix.
Dot Product (Inner Product):
A specific type of element-wise multiplication and summation operation between tensors.
Example: The dot product of two vectors is a scalar value.
Reduction Operation:
Operations that reduce the dimensions of a tensor by aggregating values, e.g., sum, mean, max.
Example: Summing all elements of a tensor to get a scalar value.
Slicing:
Selecting specific elements or sub-tensors from a larger tensor.
Example: Extracting a portion of an image or a row from a matrix.
Sparse Tensor:
A tensor where most elements are zero or have negligible values.
Example: A document-term matrix in natural language processing where most entries are zeros.
Dense Tensor:
A tensor where most elements have non-zero values.
Example: A color image with RGB channels.
Gradient:
The derivative of a tensor with respect to some variable, used in training neural networks.
Example: Calculating the gradient of a loss function with respect to the model's parameters.
Placeholder:
A special type of tensor used to hold values that will be provided later, commonly used in TensorFlow and similar frameworks.
Example: A placeholder for input data in a deep learning model.
Top comments (0)