\( \DeclareMathOperator*\argmax{argmax} \)

Due Friday, Mar. 1 at 11:59 pm EST on Gradescope. Please follow the general guidelines regarding homework assignments.

1. ResNet architecture.

The goal of this exercise is to understand the architecture of Residual Networks, or ResNets. This is an example of a network that uses shortcut or skip connections. You can find links to the original ResNet paper and PyTorch implementation at Papers With Code. Submit your answers to these questions:

  • Which class implements Figure 2 of the paper?
  • Which line numbers correspond to \(\cal F\)?
  • Which line number combines the output of \(\cal F\) with the output of the shortcut pathway?
  • Figure 3 says that “The dotted shortcuts increase dimensions.” This is explained in Eq. (2), but the precise specification is in the code itself. Which lines of code implement the “dotted shortcut”?

This annotated ResNet code from labml.ai might also be helpful for understanding.

2. Inference with a pre-trained ResNet

PyTorch comes with ResNets that were already trained on 1000 object classes from ImageNet. These are known as models with “pre-trained” weights, or pre-trained models.

Work through the ResNet notebook and submit your answers to the following questions:

  • ResNet18 is said to have 18 layers. Which of the convolutional layers are not included in this count? Why? Your answer should refer to the list of the layers in the notebook subsection “To list the hierarchy of modules inside ResNet”.

  • Write a line of code that returns the kernels of the seventh convolutional layer as a tensor.

3. Batch Normalization

The conclusion of the notebook shows that ResNet gives very different classifications in the train and eval modes. Using the wrong mode is a bug commonly encountered in practice. The plots in the Batch Normalization section show that the operation is a linear scaling. This problem guides you to a more precise understanding of the linear scaling.

  • The documentation contains an equation for Batch Normalization. How is this equation related to the following tensors? You should answer this question for both train and eval modes.

      torch.mean(torch.squeeze(x), (1, 2))
  torch.var(torch.squeeze(x), (1, 2), unbiased=False)
  model.bn1.running_mean
  model.bn1.running_var
  model.bn1.weight
  model.bn1.bias

  • The slopes and intercepts of the plots in the Batch Normalization section of the notebook are simple formulas of the above tensors. Write the formulas for both train and eval modes. You can check your answer with numerical experiments.

Additional practice not for submission

  • Work through the PyTorch tutorial on transfer learning. You will need to add this code to the Colab notebook to download the dataset.

      !wget https://download.pytorch.org/tutorial/hymenoptera_data.zip
  !unzip hymenoptera_data.zip -d data

  • Read about PyTorch object detection, including the SSD implementation. Work through the object detection notebook. The code currently prints the predicted class in the bounding box. Modify the code to also print the prediction score in the bounding box.

  • Try out a drag and drop front end to ResNet18, courtesy of Gradio and Hugging Face. Challenge: can you find an image that is classified wrong?