Developing an AI application

Going forward, AI algorithms will be incorporated into more and more everyday applications. For example, you might want to include an image classifier in a smart phone app. To do this, you'd use a deep learning model trained on hundreds of thousands of images as part of the overall application architecture. A large part of software development in the future will be using these types of models as common parts of applications.

In this project, you'll train an image classifier to recognize different species of flowers. You can imagine using something like this in a phone app that tells you the name of the flower your camera is looking at. In practice you'd train this classifier, then export it for use in your application. We'll be using this dataset of 102 flower categories, you can see a few examples below.

The project is broken down into multiple steps:

• Load and preprocess the image dataset
• Train the image classifier on your dataset
• Use the trained classifier to predict image content

We'll lead you through each part which you'll implement in Python.

When you've completed this project, you'll have an application that can be trained on any set of labeled images. Here your network will be learning about flowers and end up as a command line application. But, what you do with your new skills depends on your imagination and effort in building a dataset. For example, imagine an app where you take a picture of a car, it tells you what the make and model is, then looks up information about it. Go build your own dataset and make something new.

First up is importing the packages you'll need. It's good practice to keep all the imports at the beginning of your code. As you work through this notebook and find you need to import a package, make sure to add the import up here.

# Download the dataset
# This cell has to run only once. 
# NO need to run every time you arrive on this notebook. 

import requests
import tarfile
import os
import shutil

# Define the URL and folder paths
url = "https://s3.amazonaws.com/content.udacity-data.com/nd089/flower_data.tar.gz"
folder_name = "flowers"
file_name = "flower_data.tar.gz"
file_path = os.path.join(folder_name, file_name)

# Remove the folder or symbolic link if it already exists (equivalent to `rm -rf flowers`)
try:
    if os.path.islink(folder_name) or os.path.isfile(folder_name):
        os.remove(folder_name)  # Remove the symbolic link or file
    elif os.path.isdir(folder_name):
        shutil.rmtree(folder_name)  # Remove the directory
    print(f"Removed existing {folder_name} folder/file/soft link, if any.")
except FileNotFoundError:
    pass  # If the file or directory does not exist, do nothing

# Create the folder
os.makedirs(folder_name)
print(f"Created folder: {folder_name}")

# Download the file
response = requests.get(url, stream=True)

# Save the file in the 'flowers' folder
with open(file_path, "wb") as file:
    for chunk in response.iter_content(chunk_size=1024):
        if chunk:
            file.write(chunk)

print(f"Downloaded {file_name} to {folder_name}")

# Extract the file in the 'flowers' folder
if file_path.endswith("tar.gz"):
    with tarfile.open(file_path, "r:gz") as tar:
        tar.extractall(path=folder_name)
        print(f"Extracted {file_name} to {folder_name}")

# Clean up by removing the tar.gz file after extraction
os.remove(file_path)
print(f"Removed the downloaded tar.gz file: {file_path}")

Removed existing flowers folder/file/soft link, if any.
Created folder: flowers
Downloaded flower_data.tar.gz to flowers
Extracted flower_data.tar.gz to flowers
Removed the downloaded tar.gz file: flowers/flower_data.tar.gz

# Imports here
# Imports here
import torch
from torch import nn, optim
import torch.nn.functional as F
from torchvision import datasets, transforms, models
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image

Load the data

Here you'll use torchvision to load the data (documentation). The data should be included alongside this notebook, otherwise you can download it here. The dataset is split into three parts, training, validation, and testing. For the training, you'll want to apply transformations such as random scaling, cropping, and flipping. This will help the network generalize leading to better performance. You'll also need to make sure the input data is resized to 224x224 pixels as required by the pre-trained networks.

The validation and testing sets are used to measure the model's performance on data it hasn't seen yet. For this you don't want any scaling or rotation transformations, but you'll need to resize then crop the images to the appropriate size.

For all three sets you'll need to normalize the means and standard deviations of the images to what the network expects. For the means, it's [0.485, 0.456, 0.406] and for the standard deviations [0.229, 0.224, 0.225]. This converts the values of each color channel to be between -1 and 1 instead of 0 and 1.

data_dir = 'flowers'
train_dir = data_dir + '/train'
valid_dir = data_dir + '/valid'
test_dir = data_dir + '/test'

# TODO: Define your transforms for the training, validation, and testing sets
from torchvision import transforms
from torchvision import datasets
from torch.utils.data import DataLoader

data_transforms = {
    'train': transforms.Compose([
        transforms.RandomResizedCrop(224),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),
    
    'valid': transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),

    'test': transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
}


# TODO: Load the datasets with ImageFolder
image_datasets = {
    'train': datasets.ImageFolder(data_dir + '/train', transform=data_transforms['train']),
    'valid': datasets.ImageFolder(data_dir + '/valid', transform=data_transforms['valid']),
    'test': datasets.ImageFolder(data_dir + '/test', transform=data_transforms['test'])
}

# TODO: Using the image datasets and the trainforms, define the dataloaders
dataloaders = {
    'train': DataLoader(image_datasets['train'], batch_size=64, shuffle=True),
    'valid': DataLoader(image_datasets['valid'], batch_size=64, shuffle=False),
    'test': DataLoader(image_datasets['test'], batch_size=64, shuffle=False)
}

Label mapping

You'll also need to load in a mapping from category label to category name. You can find this in the file cat_to_name.json. It's a JSON object which you can read in with the json module. This will give you a dictionary mapping the integer encoded categories to the actual names of the flowers.

import json

with open('cat_to_name.json', 'r') as f:
    cat_to_name = json.load(f)

Building and training the classifier

Now that the data is ready, it's time to build and train the classifier. As usual, you should use one of the pretrained models from torchvision.models to get the image features. Build and train a new feed-forward classifier using those features.

We're going to leave this part up to you. If you want to talk through it with someone, chat with your fellow students! You can also ask questions on the forums or join the instructors in office hours.

Refer to the rubric for guidance on successfully completing this section. Things you'll need to do:

• Load a pre-trained network (If you need a starting point, the VGG networks work great and are straightforward to use)
• Define a new, untrained feed-forward network as a classifier, using ReLU activations and dropout
• Train the classifier layers using backpropagation using the pre-trained network to get the features
• Track the loss and accuracy on the validation set to determine the best hyperparameters

We've left a cell open for you below, but use as many as you need. Our advice is to break the problem up into smaller parts you can run separately. Check that each part is doing what you expect, then move on to the next. You'll likely find that as you work through each part, you'll need to go back and modify your previous code. This is totally normal!

When training make sure you're updating only the weights of the feed-forward network. You should be able to get the validation accuracy above 70% if you build everything right. Make sure to try different hyperparameters (learning rate, units in the classifier, epochs, etc) to find the best model. Save those hyperparameters to use as default values in the next part of the project.

# TODO: Build and train your network
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import models
from torch.cuda.amp import autocast, GradScaler

# Load the pre-trained VGG16 model (without the classifier)
vgg16 = models.vgg16(pretrained=True)

# Freeze the convolutional layers
for param in vgg16.parameters():
    param.requires_grad = False

# Define a new feed-forward classifier
classifier = nn.Sequential(
    nn.Linear(25088, 512),  # 25088 is the output size of VGG16's last convolutional layer
    nn.ReLU(),
    nn.Dropout(0.2),
    nn.Linear(512, 102),  # Output layer with 102 classes (flower categories)
    nn.LogSoftmax(dim=1)  # LogSoftmax for multi-class classification
)

# Replace the pre-trained classifier with our new classifier
vgg16.classifier = classifier

# Define loss function and optimizer
criterion = nn.NLLLoss()  # Negative Log-Likelihood Loss for classification
optimizer = optim.Adam(vgg16.classifier.parameters(), lr=0.001)  # Adam optimizer

# Set device to GPU if available
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
vgg16.to(device)

# Function to train the model
def train_model(model, dataloaders, criterion, optimizer, num_epochs=5):
    for epoch in range(num_epochs):
        model.train()
        running_loss = 0
        correct = 0
        total = 0
        
        for inputs, labels in dataloaders['train']:
            inputs, labels = inputs.to(device), labels.to(device)

            optimizer.zero_grad()  # Zero the parameter gradients
            
            # Forward pass
            outputs = model(inputs)
            loss = criterion(outputs, labels)
            
            # Backward pass
            loss.backward()
            optimizer.step()
            
            running_loss += loss.item()
            
            # Calculate accuracy
            _, preds = torch.max(outputs, 1)
            correct += torch.sum(preds == labels.data)
            total += labels.size(0)
        
        epoch_loss = running_loss / len(dataloaders['train'])
        epoch_acc = correct.double() / total
        
        print(f"Epoch {epoch+1}/{num_epochs} - Loss: {epoch_loss:.4f} - Accuracy: {epoch_acc:.4f}")

# Train the model
train_model(vgg16, dataloaders, criterion, optimizer, num_epochs=5)

/opt/conda/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/opt/conda/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to /home/student/.cache/torch/hub/checkpoints/vgg16-397923af.pth
100%|██████████| 528M/528M [00:02<00:00, 264MB/s]  

Epoch 1/5 - Loss: 2.2618 - Accuracy: 0.4731
Epoch 2/5 - Loss: 1.0173 - Accuracy: 0.7225
Epoch 3/5 - Loss: 0.8420 - Accuracy: 0.7703
Epoch 4/5 - Loss: 0.7065 - Accuracy: 0.8031
Epoch 5/5 - Loss: 0.6826 - Accuracy: 0.8107

Testing your network

It's good practice to test your trained network on test data, images the network has never seen either in training or validation. This will give you a good estimate for the model's performance on completely new images. Run the test images through the network and measure the accuracy, the same way you did validation. You should be able to reach around 70% accuracy on the test set if the model has been trained well.

# TODO: Do validation on the test set

# Set the model to evaluation mode
vgg16.eval() 

# Initialize variables to track correct predictions and total images
correct = 0
total = 0

# No need for gradient calculation during inference
with torch.no_grad():
    # Iterate over the test data
    for inputs, labels in dataloaders['test']:
        # Move inputs and labels to the device (GPU or CPU)
        inputs, labels = inputs.to(device), labels.to(device)
        
        # Forward pass: Get predictions from the model
        outputs = vgg16(inputs)  # تغییر model به vgg16
        
        # Get the predicted class (max probability)
        _, predicted = torch.max(outputs, 1)
        
        # Update the total and correct counters
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

# Calculate and print the accuracy
accuracy = 100 * correct / total
print(f'Test Accuracy: {accuracy:.2f}%')

Test Accuracy: 87.42%

Save the checkpoint

Now that your network is trained, save the model so you can load it later for making predictions. You probably want to save other things such as the mapping of classes to indices which you get from one of the image datasets: image_datasets['train'].class_to_idx. You can attach this to the model as an attribute which makes inference easier later on.

model.class_to_idx = image_datasets['train'].class_to_idx

Remember that you'll want to completely rebuild the model later so you can use it for inference. Make sure to include any information you need in the checkpoint. If you want to load the model and keep training, you'll want to save the number of epochs as well as the optimizer state, optimizer.state_dict. You'll likely want to use this trained model in the next part of the project, so best to save it now.

# TODO: Save the checkpoint 
epochs = 5
# Add the class_to_idx to the model
vgg16.class_to_idx = image_datasets['train'].class_to_idx

# Save the model checkpoint
checkpoint = {
    'input_size': 25088,  # Size of input layer (depends on the architecture you used, e.g., VGG16)
    'output_size': 102,   # Number of classes (adjust as needed)
    'hidden_units': 512,  # Number of units in the hidden layers of your classifier
    'arch': 'vgg16',      # Pre-trained model architecture used (VGG16 in this case)
    'model_state_dict': vgg16.state_dict(),
    'optimizer_state_dict': optimizer.state_dict(),
    'epochs': epochs,
    'class_to_idx': vgg16.class_to_idx,
    'classifier': vgg16.classifier
}

# Save to file
torch.save(checkpoint, 'flower_classifier.pth')
print("Checkpoint saved!")

Checkpoint saved!

Loading the checkpoint

At this point it's good to write a function that can load a checkpoint and rebuild the model. That way you can come back to this project and keep working on it without having to retrain the network.

# TODO: Write a function that loads a checkpoint and rebuilds the model

import torch
from torchvision import models

def load_checkpoint(filepath):
    """
    Loads a checkpoint and rebuilds the model.
    
    Parameters:
        filepath (str): Path to the checkpoint file.
    
    Returns:
        model (torch.nn.Module): The trained model.
        optimizer (torch.optim.Optimizer): The optimizer with its state loaded.
    """
    # Load the checkpoint
    checkpoint = torch.load(filepath)
    
    # Rebuild the model (use the same architecture)
    model = models.vgg16(pretrained=True)
    
    # Modify the classifier if needed
    model.classifier = checkpoint['classifier']
    
    # Load model state
    model.load_state_dict(checkpoint['model_state_dict'])
    
    # Load the optimizer state
    optimizer = torch.optim.Adam(model.classifier.parameters())  # Define an optimizer
    optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
    
    # Load class_to_idx
    model.class_to_idx = checkpoint['class_to_idx']
    
    # Set the model to evaluation mode
    model.eval()
    
    return model, optimizer

model, optimizer = load_checkpoint('flower_classifier.pth')

model

VGG(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU(inplace=True)
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (6): ReLU(inplace=True)
    (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): ReLU(inplace=True)
    (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (13): ReLU(inplace=True)
    (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): ReLU(inplace=True)
    (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): ReLU(inplace=True)
    (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (20): ReLU(inplace=True)
    (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (22): ReLU(inplace=True)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): ReLU(inplace=True)
    (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (27): ReLU(inplace=True)
    (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (29): ReLU(inplace=True)
    (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
  (classifier): Sequential(
    (0): Linear(in_features=25088, out_features=512, bias=True)
    (1): ReLU()
    (2): Dropout(p=0.2, inplace=False)
    (3): Linear(in_features=512, out_features=102, bias=True)
    (4): LogSoftmax(dim=1)
  )
)

Inference for classification

Now you'll write a function to use a trained network for inference. That is, you'll pass an image into the network and predict the class of the flower in the image. Write a function called predict that takes an image and a model, then returns the top $K$ most likely classes along with the probabilities. It should look like

probs, classes = predict(image_path, model)
print(probs)
print(classes)
> [ 0.01558163  0.01541934  0.01452626  0.01443549  0.01407339]
> ['70', '3', '45', '62', '55']

First you'll need to handle processing the input image such that it can be used in your network.

Image Preprocessing

You'll want to use PIL to load the image (documentation). It's best to write a function that preprocesses the image so it can be used as input for the model. This function should process the images in the same manner used for training.

First, resize the images where the shortest side is 256 pixels, keeping the aspect ratio. This can be done with the thumbnail or resize methods. Then you'll need to crop out the center 224x224 portion of the image.

Color channels of images are typically encoded as integers 0-255, but the model expected floats 0-1. You'll need to convert the values. It's easiest with a Numpy array, which you can get from a PIL image like so np_image = np.array(pil_image).

As before, the network expects the images to be normalized in a specific way. For the means, it's [0.485, 0.456, 0.406] and for the standard deviations [0.229, 0.224, 0.225]. You'll want to subtract the means from each color channel, then divide by the standard deviation.

And finally, PyTorch expects the color channel to be the first dimension but it's the third dimension in the PIL image and Numpy array. You can reorder dimensions using ndarray.transpose. The color channel needs to be first and retain the order of the other two dimensions.

# TODO: Process a PIL image for use in a PyTorch model
from PIL import Image
import numpy as np

def process_image(image_path):
    ''' Scales, crops, and normalizes a PIL image for a PyTorch model,
        returns a Numpy array
    '''
    # Open the image
    pil_image = Image.open(image_path)
    
    # Resize the image so the shortest side is 256 pixels, maintaining aspect ratio
    pil_image.thumbnail((256, 256))
    
    # Crop the center of the image to 224x224
    width, height = pil_image.size
    left = (width - 224) / 2
    top = (height - 224) / 2
    right = (width + 224) / 2
    bottom = (height + 224) / 2
    
    pil_image = pil_image.crop((left, top, right, bottom))
    
    # Convert image to numpy array
    np_image = np.array(pil_image) / 255  # Convert pixel values to range [0, 1]
    
    # Normalize the image: subtract mean and divide by std
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    
    np_image = (np_image - mean) / std
    
    # Rearrange dimensions so the color channel is first (PyTorch expects this order)
    np_image = np_image.transpose((2, 0, 1))
    
    return np_image

image_1 = process_image('flowers/test/1/image_06743.jpg')

image_1

array([[[-2.11790393, -2.11790393, -2.11790393, ..., -2.11790393,
         -2.11790393, -2.11790393],
        [-2.11790393, -2.11790393, -2.11790393, ..., -2.11790393,
         -2.11790393, -2.11790393],
        [-2.11790393, -2.11790393, -2.11790393, ..., -2.11790393,
         -2.11790393, -2.11790393],
        ...,
        [-2.11790393, -2.11790393, -2.11790393, ..., -2.11790393,
         -2.11790393, -2.11790393],
        [-2.11790393, -2.11790393, -2.11790393, ..., -2.11790393,
         -2.11790393, -2.11790393],
        [-2.11790393, -2.11790393, -2.11790393, ..., -2.11790393,
         -2.11790393, -2.11790393]],

       [[-2.03571429, -2.03571429, -2.03571429, ..., -2.03571429,
         -2.03571429, -2.03571429],
        [-2.03571429, -2.03571429, -2.03571429, ..., -2.03571429,
         -2.03571429, -2.03571429],
        [-2.03571429, -2.03571429, -2.03571429, ..., -2.03571429,
         -2.03571429, -2.03571429],
        ...,
        [-2.03571429, -2.03571429, -2.03571429, ..., -2.03571429,
         -2.03571429, -2.03571429],
        [-2.03571429, -2.03571429, -2.03571429, ..., -2.03571429,
         -2.03571429, -2.03571429],
        [-2.03571429, -2.03571429, -2.03571429, ..., -2.03571429,
         -2.03571429, -2.03571429]],

       [[-1.80444444, -1.80444444, -1.80444444, ..., -1.80444444,
         -1.80444444, -1.80444444],
        [-1.80444444, -1.80444444, -1.80444444, ..., -1.80444444,
         -1.80444444, -1.80444444],
        [-1.80444444, -1.80444444, -1.80444444, ..., -1.80444444,
         -1.80444444, -1.80444444],
        ...,
        [-1.80444444, -1.80444444, -1.80444444, ..., -1.80444444,
         -1.80444444, -1.80444444],
        [-1.80444444, -1.80444444, -1.80444444, ..., -1.80444444,
         -1.80444444, -1.80444444],
        [-1.80444444, -1.80444444, -1.80444444, ..., -1.80444444,
         -1.80444444, -1.80444444]]])

To check your work, the function below converts a PyTorch tensor and displays it in the notebook. If your process_image function works, running the output through this function should return the original image (except for the cropped out portions).

def imshow(image, ax=None, title=None):
    """Imshow for Tensor."""
    if ax is None:
        fig, ax = plt.subplots()
    
    # PyTorch tensors assume the color channel is the first dimension
    # but matplotlib assumes is the third dimension
    image = image.transpose((1, 2, 0))
    
    # Undo preprocessing
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    image = std * image + mean
    
    # Image needs to be clipped between 0 and 1 or it looks like noise when displayed
    image = np.clip(image, 0, 1)
    
    ax.imshow(image)
    
    return ax

imshow(image_1)
plt.show()

Class Prediction

Once you can get images in the correct format, it's time to write a function for making predictions with your model. A common practice is to predict the top 5 or so (usually called top-$K$) most probable classes. You'll want to calculate the class probabilities then find the $K$ largest values.

To get the top $K$ largest values in a tensor use x.topk(k). This method returns both the highest k probabilities and the indices of those probabilities corresponding to the classes. You need to convert from these indices to the actual class labels using class_to_idx which hopefully you added to the model or from an ImageFolder you used to load the data (see here). Make sure to invert the dictionary so you get a mapping from index to class as well.

Again, this method should take a path to an image and a model checkpoint, then return the probabilities and classes.

probs, classes = predict(image_path, model)
print(probs)
print(classes)
> [ 0.01558163  0.01541934  0.01452626  0.01443549  0.01407339]
> ['70', '3', '45', '62', '55']

# TODO: Implement the code to predict the class from an image file
import torch
from torch import nn
import numpy as np

def predict(image_path, model, topk=5):
    ''' Predict the class (or classes) of an image using a trained deep learning model.'''
    
    # Process the image using the process_image function
    image = process_image(image_path)
    
    # Convert image to PyTorch tensor and add batch dimension
    image_tensor = torch.from_numpy(image).float()
    image_tensor = image_tensor.unsqueeze(0)  # Add batch dimension
    
    # Move the image to the same device as the model (CPU or GPU)
    device = 'cuda' if torch.cuda.is_available() else 'cpu'
    image_tensor = image_tensor.to(device)
    model.to(device)
    
    # Set the model to evaluation mode
    model.eval()
    
    # Forward pass through the model to get the output
    with torch.no_grad():
        outputs = model(image_tensor)
    
    # Apply softmax to get probabilities
    probs = nn.Softmax(dim=1)(outputs)
    
    # Get the top k probabilities and their corresponding indices
    top_probs, top_indices = torch.topk(probs, topk)
    
    # Convert topk results to numpy arrays
    # Convert topk results to numpy arrays (Move to CPU first)
    top_probs = top_probs.cpu().numpy().flatten()
    top_indices = top_indices.cpu().numpy().flatten()
    
    # Invert the class_to_idx mapping to get index to class
    idx_to_class = {v: k for k, v in model.class_to_idx.items()}
    
    # Get the class labels corresponding to the top indices
    top_classes = [idx_to_class[idx] for idx in top_indices]
    
    return top_probs, top_classes

image_path = "flowers/test/1/image_06743.jpg" 
probs, classes = predict(image_path, model)

print("Top probabilities:", probs)
print("Predicted classes:", classes)

Top probabilities: [0.5241119  0.36719608 0.03511214 0.03192949 0.01827037]
Predicted classes: ['83', '1', '76', '51', '98']

Sanity Checking

Now that you can use a trained model for predictions, check to make sure it makes sense. Even if the testing accuracy is high, it's always good to check that there aren't obvious bugs. Use matplotlib to plot the probabilities for the top 5 classes as a bar graph, along with the input image. It should look like this:

You can convert from the class integer encoding to actual flower names with the cat_to_name.json file (should have been loaded earlier in the notebook). To show a PyTorch tensor as an image, use the imshow function defined above.

# TODO: Display an image along with the top 5 classes
import matplotlib.pyplot as plt
import json
from PIL import Image

def display_prediction(image_path, model, topk=5):
    ''' Display an image and its top k predicted classes with probabilities. '''
    
    # Process the image
    probs, classes = predict(image_path, model, topk)
    
    # Load category to name mapping (cat_to_name.json should be loaded already)
    with open('cat_to_name.json', 'r') as f:
        cat_to_name = json.load(f)
    
    # Get the class names from the indices
    class_names = [cat_to_name[str(cls)] for cls in classes]
    
    # Load the image using PIL for displaying
    pil_image = Image.open(image_path)
    
    # Plot the image
    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 6))
    
    # Plot the image
    ax1.imshow(pil_image)
    ax1.axis('off')
    ax1.set_title("Image")
    
    # Plot the top k classes and their probabilities
    ax2.barh(class_names, probs)
    ax2.set_xlabel('Probability')
    ax2.set_title('Top 5 Predicted Classes')
    
    plt.tight_layout()
    plt.show()

image_path = "flowers/test/1/image_06743.jpg" 
display_prediction(image_path, model)

import json

# Path to the category mapping file
cat_to_name_path = "cat_to_name.json"

# Load the category-to-name mapping
with open(cat_to_name_path, 'r') as f:
    cat_to_name = json.load(f)

# Extract the true label from the image path
image_path = "flowers/test/1/image_06743.jpg"
true_label_idx = image_path.split("/")[-2]  # The folder before the image name represents the class index.

# Convert the class index to the actual category name
true_label = cat_to_name[true_label_idx]

print(f"🔹 True label of the image: {true_label}")

🔹 True label of the image: pink primrose