What is a tensor
Published: 2023-03-11 19:40:00

---
title: What is a tensor
published: true
description: Explaining what a tensor is in the context of machine learning
tags: 
cover_image: https://blog-files-prod.s3.us-east-2.amazonaws.com/public/what-is-a-tensor/what-is-a-tensor.jpg
# Use a ratio of 100:42 for best results.
published_at: 2023-03-11 19:40 +0000
---

Used as the building block of machine learning models, a tensor is an N-dimensional container of data. Two very common Python machine learning libraries, [PyTorch](https://pytorch.org/) and [Tensorflow](https://www.tensorflow.org/), have their own object-oriented abstractions of tensors that are used to build PyTorch and Tensorflow machine learning models, respectively. In this blog post, we will cover basic tensor operations (in the context of the Python PyTorch and Tensorflow libraries), along with examples of using tensors in our own machine learning code here at Crypto Clamor.

## Basic tensor operations

Tensors in both PyTorch and Tensorflow have a variety of operations available to use in various computations. One of these common operations makes use of operator overloading.

### Operator overloading

Tensors in both PyTorch and Tensorflow make heavy use of Python's [operator overloading](https://stackoverflow.com/questions/2400635/comprehensive-guide-to-operator-overloading-in-python) functionality. Operator overloading is the ability of a programming language to override default operators (e.g., "+" and "-") to provide custom functionality to a class.

For example, to "add 1" to a tensor with Tensorflow with 1 dimension:

```python
import tensorflow as tf

# initialize a 1-D tensor
rank_1_tensor = tf.constant([1,1,1])

# add "1" to each element in the rank_1_tensor
x = rank_1_tensor + 1
print(x)
# log output:
# tf.Tensor([2 2 2], shape=(3,), dtype=int32)
```

and in PyTorch:

```python
import torch

# initialize a 1-D tensor
rank_1_tensor = torch.tensor([1,1,1])

# add "1" to each element in the rank_1_tensor
x = rank_1_tensor + 1
print(x)
# log output:
# tensor([2, 2, 2])
```

## Examples of tensor usage in a machine learning model

Here at Crypto Clamor, one example of using tensors we can share is from how we initially fine-tuned our BERT model. We used a labeled (meaning, tweet text with associated sentiment score) csv data file to load into a Tensorflow dataset. This dataset we are using is a iterable data structure, where each element is a batch and each batch is a tuple of tensors.

```python
import tensorflow as tf
from transformers import BertTokenizer, TFBertForSequenceClassification, InputFeatures
import pandas as pd
import matplotlib.pyplot as plt
import tarfile
import os

CSV_PATH = './labeled_model_data.csv'

NUM_EPOCHS = 10
DATASET_SIZE = 2500
BATCH_SIZE = 25
AUTOTUNE = tf.data.experimental.AUTOTUNE

dataset = tf.data.experimental.make_csv_dataset(
    CSV_PATH,
    batch_size=BATCH_SIZE,
    column_names=['score','timestamp', 'datestring', 'N/A', 'user', 'tweet'],
    label_name='score',
    select_columns=['score', 'tweet'],
    num_epochs=NUM_EPOCHS,
    header=False,
    shuffle_seed=0,
    shuffle=True,
    num_rows_for_inference=1600000,
    ignore_errors=True,).prefetch(AUTOTUNE)
```

Here, we are creating a Tensorflow dataset using the experimental API's `make_csv_dataset` function. The output stored in our `dataset` variable is a Tensorflow `Dataset`, where each element in the tensorflow dataset is a batch (of size set to `batch_size`). Each element (that is, each batch) in the dataset is comprised of a tuple `(features, labels)` where `features` is a Tensor containing the corresponding feature data, and `labels` is a Tensor containing the batch's corresponding label data.

## Conclusion

This blog post has just barely scratched the surface of what tensors are, and especially how they are used in building machine learning models. Hopefully you've learned a thing or two from this blog post on tensors.

Questions or comments? Connect with us on Twitter, LinkedIn or Facebook!