"Neural networks comprise of layers/modules that perform operations on data. The torch.nn namespace provides all the building blocks you need to build your own neural network. Every module in PyTorch subclasses the nn.Module. A neural network is a module itself that consists of other modules (layers). This nested structure allows for building and managing complex architectures easily."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import torch\n",
"from torch import nn\n",
"from torch.utils.data import DataLoader\n",
"from torchvision import datasets, transforms"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We want to be able to train our model on a hardware accelerator like the GPU, if it is available. Let’s check to see if torch.cuda is available, else we continue to use the CPU.\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Using cuda device\n"
]
}
],
"source": [
"device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
"print(f\"Using {device} device\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We define our neural network by subclassing nn.Module, and initialize the neural network layers in __init__. Every nn.Module subclass implements the operations on input data in the forward method."
"# Create an instance of NeuralNetwork , and move it to the device \n",
"model = NeuralNetwork().to(device)\n",
"# Print its structure\n",
"print(model)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"To use the model, we pass it the input data. This executes the model’s forward, along with some background operations. Do not call model.forward() directly!\n",
"\n",
"Calling the model on the input returns a 2-dimensional tensor with dim=0 corresponding to each output of 10 raw predicted values for each class, and dim=1 corresponding to the individual values of each output. We get the prediction probabilities by passing it through an instance of the nn.Softmax module."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Predicted class: tensor([9], device='cuda:0')\n"
]
}
],
"source": [
"X = torch.rand(1,28,28, device=device)\n",
"logits = model(X)\n",
"pred_probab = nn.Softmax(dim=1)(logits)\n",
"y_pred = pred_probab.argmax(1)\n",
"print(f\"Predicted class: {y_pred}\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Let’s break down the layers in the FashionMNIST model. To illustrate it, we will take a sample minibatch of 3 images of size 28x28 and see what happens to it as we pass it through the network."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([3, 28, 28])\n"
]
}
],
"source": [
"input_image = torch.rand(3,28,28)\n",
"print(input_image.size())"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We initialize the nn.Flatten layer to convert each 2D 28x28 image into a contiguous array of 784 pixel values ( the minibatch dimension (at dim=0) is maintained)."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([3, 784])\n"
]
}
],
"source": [
"flatten = nn.Flatten()\n",
"flat_image = flatten(input_image)\n",
"print(flat_image.size())"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The linear layer is a module that applies a linear transformation on the input using its stored weights and biases."
"Non-linear activations are what create the complex mappings between the model’s inputs and outputs. They are applied after linear transformations to introduce nonlinearity, helping neural networks learn a wide variety of phenomena.\n",
"\n",
"In this model, we use nn.ReLU between our linear layers, but there’s other activations to introduce non-linearity in your model."
"nn.Sequential is an ordered container of modules. The data is passed through all the modules in the same order as defined. You can use sequential containers to put together a quick network like seq_modules."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"seq_modules = nn.Sequential(\n",
" flatten,\n",
" layer1,\n",
" nn.ReLU(),\n",
" nn.Linear(20,10)\n",
")\n",
"\n",
"input_image = torch.rand(3,28,28)\n",
"logits = seq_modules(input_image)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The last linear layer of the neural network returns logits - raw values in [-infty, infty] - which are passed to the nn.Softmax module. The logits are scaled to values [0, 1] representing the model’s predicted probabilities for each class. dim parameter indicates the dimension along which the values must sum to 1."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"softmax = nn.Softmax(dim=1)\n",
"pred_probab = softmax(logits)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Many layers inside a neural network are parameterized, i.e. have associated weights and biases that are optimized during training. Subclassing nn.Module automatically tracks all fields defined inside your model object, and makes all parameters accessible using your model’s parameters() or named_parameters() methods."
"Data does not always come in its final processed form that is required for training machine learning algorithms. We use transforms to perform some manipulation of the data and make it suitable for training.\n",
"\n",
"All TorchVision datasets have two parameters -transform to modify the features and target_transform to modify the labels - that accept callables containing the transformation logic. The torchvision.transforms module offers several commonly-used transforms out of the box.\n",
"\n",
"The FashionMNIST features are in PIL Image format, and the labels are integers. For training, we need the features as normalized tensors, and the labels as one-hot encoded tensors. To make these transformations, we use ToTensor and Lambda"
"ToTensor converts a PIL image or NumPy ndarray into a FloatTensor. and scales the image’s pixel intensity values in the range [0., 1.]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Lambda transforms apply any user-defined lambda function. Here, we define a function to turn the integer into a one-hot encoded tensor. It first creates a zero tensor of size 10 (the number of labels in our dataset) and calls scatter_ which assigns a value=1 on the index as given by the label y."
Nigel Barink