{"cells":[{"cell_type":"markdown","metadata":{"id":"8nMtWnwwm-gF"},"source":["# **Exercise for Introduction to PyTorch Fundamentals**"]},{"cell_type":"markdown","metadata":{"id":"LT1YCYN4m-gI"},"source":["### IMPORTANT NOTE: To avoid version mismatch errors with Autolab, please ensure you are using commands that are from the following versions of [*NumPy*](https://numpy.org/) and [*Torch*](https://pytorch.org/) : \n","`numpy==1.16.4` \n","`torch==1.2.0` \n","if your installation fails, make sure you are using a virtual environment with `python v3.7.13` installed\n"," \n","For example,\n","Consider using `torch.mul(x, y)` instead of `torch.multiply(x, y)`.\n","\n","Why? `torch.mul(x, y)` is available in version `torch==1.2.0`, whereas `torch.multiply(x, y)` is not. Using `torch==1.2.0` will help us avoid unpleasant issues with AutoLab's default versioning.\n","\n","---\n","\n","\n"]},{"cell_type":"markdown","metadata":{"id":"llpFMKCwm-gI"},"source":["> PyTorch is an open source deep learning platform that provides a seamless path from research prototyping to production deployment.\n","> - *Hybrid Front-End:* A new hybrid front-end seamlessly transitions between eager mode and graph mode to provide both flexibility and speed.\n","> - *Distributed Training:* Scalable distributed training and performance optimization in research and production is enabled by the torch.distributed backend.\n","> - *Python-First:* Deep integration into Python allows popular libraries and packages to be used for easily writing neural network layers in Python.\n","> - *Tools & Libraries:* A rich ecosystem of tools and libraries extends PyTorch and supports development in computer vision, NLP and more.\n",">\n","> —*[About PyTorch](https://pytorch.org/)*\n","\n","One consideration as to why we are using PyTorch is most succinctly summerized by Andrej Karpathy, Former Director of Artificial Intelligence and Autopilot Vision at Tesla. The technical summary can be found [here](https://twitter.com/karpathy/status/868178954032513024?lang=en).\n","\n","You also refer a brief summary on PyTorch by Nvidia [here](https://www.nvidia.com/en-us/glossary/data-science/pytorch/).\n","\n","For this notebook you can refer to the documentation provided by PyTorch [here](https://pytorch.org/docs/stable/nn.html)."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"FmSsfMaAm-gJ"},"outputs":[],"source":["import numpy as np\n","print(np.__version__)\n","import os\n","import time\n","import torch\n","print(torch.__version__)\n","\n","# Expected outputs for local (used for HW part1s):\n","# 1.16.4\n","# 1.2.0\n","\n","# Expected outputs for colab (used for HW part2s):\n","# 1.23.5\n","# 2.1.0+cu121"]},{"cell_type":"markdown","metadata":{"id":"2qdEM_GTm-gK"},"source":["### **1. Interconversion**"]},{"cell_type":"markdown","metadata":{"id":"gtd1mrZom-gL"},"source":["#### 1.1 Converting from NumPy to PyTorch Tensor\n","In this task, you will implement a conversion function from arrays to tensors.\n","\n","The function should take a numpy ndarray and convert it to a PyTorch tensor.\n","\n","*Function torch.tensor is one of the simple ways to implement it but please do not use it this time. The PyTorch environment installed on Autolab is not an up-to-date version and does not support this function.*\n","\n","**Your Task**: Implement the function `numpy2tensor`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"AOztJhtsm-gL"},"outputs":[],"source":["def numpy2tensor(x):\n"," \"\"\"\n"," Creates a torch.Tensor from a numpy.ndarray.\n","\n"," Parameters:\n"," x (numpy.ndarray): 1-dimensional numpy array.\n","\n"," Returns:\n"," torch.Tensor: 1-dimensional torch tensor.\n"," \"\"\"\n","\n"," return torch.tensor(x) #NotImplemented # TODO"]},{"cell_type":"markdown","metadata":{"id":"I-CCu4Ibm-gL"},"source":["##### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"EySRUXjCm-gL","colab":{"base_uri":"https://localhost:8080/"},"outputId":"c5c09dcc-1c03-4780-def9-0862e4a04650"},"outputs":[{"output_type":"stream","name":"stdout","text":["\n"]}],"source":["X = np.random.randint(-1000, 1000, size=3000)\n","\n","print(type(numpy2tensor(X)))"]},{"cell_type":"markdown","metadata":{"id":"P7rVaFHim-gM"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n","
<class 'torch.Tensor'>
"]},{"cell_type":"markdown","metadata":{"id":"yy63znfRm-gM"},"source":["#### 1.2 Converting from PyTorch Tensor to NumPy\n","\n","In this task, you will implement a conversion function from tensors to arrays.\n","\n","The function should take a PyTorch tensor and convert it to a numpy ndarray.\n","\n","**Your Task**: Implement the function `tensor2numpy`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"sVy1x9pem-gM"},"outputs":[],"source":["def tensor2numpy(x):\n"," \"\"\"\n"," Creates a numpy.ndarray from a torch.Tensor.\n","\n"," Parameters:\n"," x (torch.Tensor): 1-dimensional torch tensor.\n","\n"," Returns:\n"," numpy.ndarray: 1-dimensional numpy array.\n"," \"\"\"\n","\n"," return NotImplemented # TODO"]},{"cell_type":"markdown","metadata":{"id":"ekVnQTUHm-gM"},"source":["##### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"QFNQcTS8m-gN"},"outputs":[],"source":["X = np.random.randint(-1000, 1000, size=3000)\n","X = torch.from_numpy(X)\n","\n","print(type(tensor2numpy(X)))"]},{"cell_type":"markdown","metadata":{"id":"ZJ-AbDmvm-gN"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n","
<class 'numpy.ndarray'>
"]},{"cell_type":"markdown","metadata":{"id":"ulMyp9z6m-gN"},"source":["### **2. Vectorization**\n","\n","Lists are a foundational data structure in Python, allowing us to create simple and complex algorithms to solve problems. However, in mathematics and particularly in linear algebra, we work with vectors and matrices to model problems and create statistical solutions. Through these exercises, we will begin introducing you to how to think more mathematically through the use of PyTorch by starting with a process known as vectorization.\n","\n","Index chasing is a very valuable skill, and certainly one you will need in this course, but mathematical problems often have simpler and more efficient representations that use vectors. The process of converting from an implimentation that uses indicies to one that uses vectors is known as vectorization. Once vectorized, the resulting implementation often yields to the user faster and more readable code than before.\n","\n","In the following problems, we will ask you to practice reading mathematical expressions and deduce their vectorized equivalent along with their implementation in Python. You will use the PyTorch array object as the Python equivalent to a vector, and in later sections you will work with sets of vectors known as matrices."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"pwJu6cNAm-gO"},"outputs":[],"source":["def PYTORCH_dot(x, y):\n"," \"\"\"\n"," Dot product of two tensors.\n","\n"," Parameters:\n"," x (torch.Tensor): 1-dimensional torch tensor.\n"," y (torch.Tensor): 1-dimensional torch tensor.\n","\n"," Returns:\n"," torch.int64: scalar quantity.\n"," \"\"\"\n","\n"," return NotImplemented # TODO"]},{"cell_type":"markdown","metadata":{"id":"WGgHsqmDm-gO"},"source":["##### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"JqUL7tQCm-gO"},"outputs":[],"source":["np.random.seed(0)\n","X = np.random.randint(-1000, 1000, size=3000)\n","Y = np.random.randint(-1000, 1000, size=3000)\n","\n","\n","X = numpy2tensor(X)\n","Y = numpy2tensor(Y)\n","\n","print(PYTORCH_dot(X,Y))"]},{"cell_type":"markdown","metadata":{"id":"p17MVBlPm-gO"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n"," \n","
PYTORCH_dot(X,Y) 7082791
"]},{"cell_type":"markdown","metadata":{"id":"OehfROg0m-gO"},"source":["#### 2.2 Outer Product\n","\n","In this task, you will implement the outer product function for torch tensors.\n","\n","The outer product (also known as the tensor product) of vectors x and y is defined as\n","\n","$$\n","x \\otimes y =\n","\\begin{bmatrix}\n","x_1 y_1 & x_1 y_2 & … & x_1 y_n\\\\\n","x_2 y_1 & x_2 y_2 & … & x_2 y_n\\\\\n","⋮ & ⋮ & ⋱ & ⋮ \\\\\n","x_m y_1 & x_m y_2 & … & x_m y_n\n","\\end{bmatrix}\n","$$\n","\n","**Your Task**: Implement the function `PYTORCH_outer`.\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Lz7bIeLgm-gP"},"outputs":[],"source":["def PYTORCH_outer(x, y):\n"," \"\"\"\n"," Compute the outer product of two vectors.\n","\n"," Parameters:\n"," x (torch.Tensor): 1-dimensional torch tensor.\n"," y (torch.Tensor): 1-dimensional torch tensor.\n","\n"," Returns:\n"," torch.Tensor: 2-dimensional torch tensor.\n"," \"\"\"\n","\n"," return NotImplemented # TODO"]},{"cell_type":"markdown","metadata":{"id":"pMNJF977m-gP"},"source":["##### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"hR7BfA3im-gP"},"outputs":[],"source":["np.random.seed(0)\n","X = np.random.randint(-1000, 1000, size=3000)\n","Y = np.random.randint(-1000, 1000, size=3000)\n","\n","\n","X = numpy2tensor(X)\n","Y = numpy2tensor(Y)\n","\n","print(PYTORCH_outer(X,Y))"]},{"cell_type":"markdown","metadata":{"id":"nio3SbQim-gP"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n"," \n"," \n","
PYTORCH_outer(X,Y) \n"," [[  59092 -144096  136512 ...  -53088  -86268   53404]
\n","  [  82467 -201096  190512 ...  -74088 -120393   74529]
\n","  [-122111  297768 -282096 ...  109704  178269 -110357]
\n","  ...
\n","  [-144551  352488 -333936 ...  129864  211029 -130637]
\n","  [-179707  438216 -415152 ...  161448  262353 -162409]
\n","  [  88825 -216600  205200 ...  -79800 -129675   80275]]
\n","
"]},{"cell_type":"markdown","metadata":{"id":"GoF1xUtem-gP"},"source":["#### 2.3 Hadamard Product\n","\n","In this task, you will implement the Hadamard product function, `multiply`, for torch tensors.\n","\n","The Hadamard product (also known as the Schur product or entrywise product) of vectors x and y is defined as\n","\n","$$\n","x \\circ y =\n","\\begin{bmatrix}\n","x_{1} y_{1} & x_{2} y_{2} & … & x_{n} y_{n}\n","\\end{bmatrix}\n","$$\n","\n","**Your Task**: Implement the function `PYTORCH_multiply`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"2_qwqp2ym-gP"},"outputs":[],"source":["def PYTORCH_multiply(x, y):\n"," \"\"\"\n"," Multiply arguments element-wise.\n","\n"," Parameters:\n"," x (torch.Tensor): 1-dimensional torch tensor.\n"," y (torch.Tensor): 1-dimensional torch tensor.\n","\n"," Returns:\n"," torch.Tensor: 1-dimensional torch tensor.\n"," \"\"\"\n","\n"," return NotImplemented # TODO"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"BqU2UtZAm-gQ"},"outputs":[],"source":["np.random.seed(0)\n","X = np.random.randint(-1000, 1000, size=3000)\n","Y = np.random.randint(-1000, 1000, size=3000)\n","\n","\n","X = numpy2tensor(X)\n","Y = numpy2tensor(Y)\n","\n","print(PYTORCH_multiply(X,Y))"]},{"cell_type":"markdown","metadata":{"id":"9jLobTUMm-gQ"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n"," \n","
PYTORCH_multiply(X,Y) \n"," [  59092 -201096 -282096 ...  129864  262353   80275]\n","
"]},{"cell_type":"markdown","metadata":{"id":"Y281rHuem-gQ"},"source":["#### 2.4 Sum-Product\n","In this task, you will implement the sum-product function for torch tensors.\n","\n","The sum-product of vectors x and y, each with n real component, is defined as\n","\n","$$\n","f(x, y) =\n","{\n","\\begin{bmatrix}\n","1\\\\\n","1\\\\\n","⋮\\\\\n","1\n","\\end{bmatrix}^{\\;T}\n","%\n","\\begin{bmatrix}\n","x_1 y_1 & x_1 y_2 & … & x_1 y_n\\\\\n","x_2 y_1 & x_2 y_2 & … & x_2 y_n\\\\\n","⋮ & ⋮ & ⋱ & ⋮ \\\\\n","x_m y_1 & x_m y_2 & … & x_m y_n\n","\\end{bmatrix}\n","%\n","\\begin{bmatrix}\n","1\\\\\n","1\\\\\n","⋮\\\\\n","1\n","\\end{bmatrix}\n","} =\n","\\displaystyle\\sum_{i=1}^{n} \\displaystyle\\sum_{j=1}^{n} x_i \\cdot y_j\n","$$\n","\n","**Your Task**: Implement the function `PYTORCH_sumproduct`.\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"4zyqcVJPm-gQ"},"outputs":[],"source":["def PYTORCH_sumproduct(x, y):\n"," \"\"\"\n"," Sum over all the dimensions of the outer product of two vectors.\n","\n"," Parameters:\n"," x (torch.Tensor): 1-dimensional torch tensor.\n"," y (torch.Tensor): 1-dimensional torch tensor.\n","\n"," Returns:\n"," torch.int64: scalar quantity.\n"," \"\"\"\n","\n"," return NotImplemented # TODO"]},{"cell_type":"markdown","metadata":{"id":"Ixn7jT8Xm-gQ"},"source":["##### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"uk_y45xYm-gR"},"outputs":[],"source":["np.random.seed(0)\n","X = np.random.randint(-1000, 1000, size=3000)\n","Y = np.random.randint(-1000, 1000, size=3000)\n","\n","X = numpy2tensor(X)\n","Y = numpy2tensor(Y)\n","\n","print(PYTORCH_sumproduct(X,Y))"]},{"cell_type":"markdown","metadata":{"id":"dXl3lBNEm-gR"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n"," \n","
TORCH_sumproduct(X,Y) 265421520
"]},{"cell_type":"markdown","metadata":{"id":"rljBFhfIm-gS"},"source":["#### 2.5 ReLU\n","\n","In this task, you will implement the ReLU activation function torch tensors.\n","\n","The ReLU activation (also known as the rectifier or rectified linear unit) matrix Z resulting from applying the ReLU function to matrix X is defined such that for $X,Z \\in M_{m \\times n} (\\mathbb{R})$,\n","\n","\n","**Your Task:** Implement the function `PYTORCH_ReLU`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"nfaJ0bjNm-gS"},"outputs":[],"source":["def PYTORCH_ReLU(x):\n"," \"\"\"\n"," Applies the rectified linear unit function element-wise.\n","\n"," Parameters:\n"," x (torch.Tensor): 2-dimensional torch tensor.\n","\n"," Returns:\n"," torch.Tensor: 2-dimensional torch tensor.\n"," \"\"\"\n","\n"," return NotImplemented # TODO"]},{"cell_type":"markdown","metadata":{"id":"hGr4rDIum-gS"},"source":["##### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"JIrL1jfum-gT"},"outputs":[],"source":["np.random.seed(0)\n","X = np.random.randint(-1000, 1000, size=(3000,3000))\n","\n","\n","X = numpy2tensor(X)\n","\n","print(PYTORCH_ReLU(X))"]},{"cell_type":"markdown","metadata":{"id":"Qs1Imk9Mm-gT"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n"," \n"," \n","
PYTORCH_ReLU(X) \n"," [[  0   0 653 ... 773 961   0]
\n"," [  0 456   0 ... 168 273   0]
\n"," [936 475   0 ... 408   0   0]
\n"," ...
\n"," [  0 396 457 ... 646   0   0]
\n"," [645 943   0 ... 863   0 790]
\n"," [641   0 379 ... 347   0   0]]\n","
"]},{"cell_type":"markdown","metadata":{"id":"oFlfBxn9m-gT"},"source":["#### 2.6 Prime ReLU (derivative of ReLU)\n","\n","In this task, you will implement the derivative of the ReLU activation function for torch tensors.\n","\n","The derivative of the ReLU activation matrix Z resulting from applying the derivative of the ReLU function to matrix X is defined such that for $X,Z \\in M_{m \\times n} (\\mathbb{R})$,\n","\n","\n","\n","**Your Task:** Implement the function `PYTORCH_PrimeReLU`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Ar0b8Cmom-gU"},"outputs":[],"source":["def PYTORCH_PrimeReLU(x):\n"," \"\"\"\n"," Applies the derivative of the rectified linear unit function\n"," element-wise.\n","\n"," Parameters:\n"," x (numpy.ndarray): 2-dimensional torch tensor.\n","\n"," Returns:\n"," numpy.ndarray: 2-dimensional torch tensor.\n"," \"\"\"\n"," # TODO\n","\n"," return x"]},{"cell_type":"markdown","metadata":{"id":"x4-Tsua7m-gU"},"source":["##### Test Example:\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"E6FtWmGJm-gU"},"outputs":[],"source":["np.random.seed(0)\n","X = np.random.randint(-1000, 1000, size=(3000,3000))\n","\n","X = numpy2tensor(X)\n","\n","print(PYTORCH_PrimeReLU(X))"]},{"cell_type":"markdown","metadata":{"id":"65FcTZOFm-gV"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n"," \n","
PYTORCH_PrimeReLU(X) \n"," [[0 0 1 ... 1 1 0]
\n"," [0 1 0 ... 1 1 0]
\n"," [1 1 0 ... 1 0 0]
\n"," ...
\n"," [0 1 1 ... 1 0 0]
\n"," [1 1 0 ... 1 0 1]
\n"," [1 0 1 ... 1 0 0]]\n","
"]},{"cell_type":"markdown","metadata":{"id":"o3EBsZsim-gV"},"source":["### **3. Tensor Manipulation**\n","\n","PyTorch offers a wide range of tensor manipulation tasks that empower users to efficiently process and transform data within neural network workflows.\n","These tasks include :\n","1. Flatten\n","2. Unsqueeze\n","3. Squeeze\n","4. Reshape\n","5. Transpose\n","6. Permute\n","7. Concatenation\n","8. Stack\n","9. Padding"]},{"cell_type":"markdown","metadata":{"id":"fIzGVjK0m-gW"},"source":["#### 3.1 Flatten\n","\n","In this task, you will implement the flatten function for torch tensors.\n","\n","\n","\n","\n","**Your Task:** Implement the function `PYTORCH_flatten`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"bG7H_5ahm-gW"},"outputs":[],"source":["def PYTORCH_flatten(input_tensor):\n"," \"\"\"\n"," Reshapes a tensor into a 1-dimensional array while maintaining the order of elements.\n","\n"," Parameters:\n"," x (torch.Tensor): 3-dimensional torch tensor.\n","\n"," Returns:\n"," torch.Tensor: 1-dimensional torch tensor.\n"," \"\"\"\n"," m = torch.nn.Flatten()\n"," return m(input_tensor)\n"," # return NotImplemented #TODO"]},{"cell_type":"markdown","metadata":{"id":"ckVHYbr-m-gW"},"source":["##### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"6AxIPmtdm-gW","colab":{"base_uri":"https://localhost:8080/"},"outputId":"5dcfd6ba-b6af-401e-9864-2c5c7ada13d1"},"outputs":[{"output_type":"stream","name":"stdout","text":["tensor([[ 2, 5, -10, ..., -2, 4, 2],\n"," [ 3, -9, 3, ..., 4, 3, 6],\n"," [ 4, 9, 9, ..., 7, -4, 9],\n"," ...,\n"," [ -7, 1, 2, ..., 3, -2, 4],\n"," [ 1, -8, -2, ..., -5, -6, 7],\n"," [ -4, -6, -3, ..., 1, 3, -3]])\n"]}],"source":["np.random.seed(0)\n","X = np.random.randint(-10, 10, size=(100,100,100))\n","\n","X = numpy2tensor(X)\n","\n","print(PYTORCH_flatten(X))"]},{"cell_type":"markdown","metadata":{"id":"SNsni2Bhm-gW"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n"," \n","
PYTORCH_flatten(X) \n"," [ 2, 5, -10, ..., 1, 3, -3]\n","
"]},{"cell_type":"markdown","metadata":{"id":"3xEP2bXom-gW"},"source":["#### 3.2 Unsqueeze\n","\n","In this task, you will implement the unsqueeze function for torch tensors along given dimension.\n","\n","\n","\n","\n","**Your Task:** Implement the function `PYTORCH_unsqueeze`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"JWiQtP-Tm-gW"},"outputs":[],"source":["def PYTORCH_unsqueeze(X,dim):\n"," \"\"\"\n"," Adds a new dimension to a tensor at the specified position 'dim'.\n","\n"," Parameters:\n"," x (torch.Tensor): 2-dimensional torch tensor.\n","\n"," dim (int): scalar.\n","\n"," Returns:\n"," torch.Tensor: 3-dimensional torch tensor.\n"," \"\"\"\n","\n"," return NotImplemented #TODO"]},{"cell_type":"markdown","metadata":{"id":"Hu-n1qoom-gW"},"source":["##### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ERwRMBmim-gW","outputId":"cc7b8029-6ac6-4f71-81ed-ca9cb10624dd"},"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([[[4, 0, 3, ..., 2, 2, 3],\n"," [2, 3, 2, ..., 2, 3, 3],\n"," [1, 3, 4, ..., 0, 2, 4],\n"," ...,\n"," [2, 0, 1, ..., 4, 4, 3],\n"," [2, 4, 3, ..., 3, 0, 0],\n"," [2, 3, 4, ..., 4, 3, 1]]])\n","\n","\n","Shape before unsqueeze: torch.Size([100, 100])\n","Shape after unsqueeze: torch.Size([1, 100, 100])\n"]}],"source":["np.random.seed(0)\n","X = np.random.randint(0, 5, size=(100,100))\n","\n","X = numpy2tensor(X)\n","\n","print(PYTORCH_unsqueeze(X,0))\n","print(\"\\n\")\n","\n","print(\"Shape before unsqueeze: \",X.shape)\n","print(\"Shape after unsqueeze: \",PYTORCH_unsqueeze(X,0).shape)"]},{"cell_type":"markdown","metadata":{"id":"D-bSSKHwm-gX"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n"," \n","
PYTORCH_unsqueeze(X) \n"," [[[4 0 3 ... 2 2 3]
\n","  [2 3 2 ... 2 3 3]
\n","  [1 3 4 ... 0 2 4]
\n","  ...,
\n","  [2 0 1 ... 4 4 3]
\n","  [2 4 3 ... 3 0 0]
\n","  [2 3 4 ... 4 3 1]]]
\n","
\n","Shape before unsqueeze: torch.Size([100, 100])
\n","Shape after unsqueeze: torch.Size([1, 100, 100])\n","
"]},{"cell_type":"markdown","metadata":{"id":"fvx93Rb7m-gX"},"source":["#### 3.3 Squeeze\n","\n","In this task, you will implement the squeeze function for torch tensors along axis with dimension 1 (axis = 0).\n","\n","\n","\n","\n","**Your Task:** Implement the function `PYTORCH_squeeze`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"EK92xdp9m-gX"},"outputs":[],"source":["def PYTORCH_squeeze(X,dim):\n"," \"\"\"\n"," Removes dimension to a tensor at the specified position 'dim'.\n","\n"," Parameters:\n"," x (torch.Tensor): 2-dimensional torch tensor.\n","\n"," dim (integer): scalar.\n","\n"," Returns:\n"," torch.Tensor: 1-dimensional torch tensor.\n"," \"\"\"\n","\n"," return NotImplemented #TODO"]},{"cell_type":"markdown","metadata":{"id":"dsaF2J-7m-gX"},"source":["#### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"RMD-i948m-gX","outputId":"9d0cce2a-73c9-4f8d-bee0-cc4e2b50328e"},"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([5, 0, 3, 3, 7])\n","Shape of tensor before squeeze: torch.Size([1, 5])\n","Shape of tensor after squeeze: torch.Size([5])\n"]}],"source":["np.random.seed(0)\n","X = np.random.randint(0, 10, size=(1,5))\n","\n","X = numpy2tensor(X)\n","\n","\n","\n","print(PYTORCH_squeeze(X,0))\n","\n","print(\"Shape of tensor before squeeze: \",X.shape)\n","print(\"Shape of tensor after squeeze: \",PYTORCH_squeeze(X,0).shape )"]},{"cell_type":"markdown","metadata":{"id":"32q8s3Aym-gY"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n"," \n","
PYTORCH_squeeze(X) \n"," [5 0 3 3 7 ]
\n","Shape of tensor before squeeze: torch.Size([1, 5])
\n","Shape of tensor after squeeze: torch.Size([5])\n","
"]},{"cell_type":"markdown","metadata":{"id":"QzHpPfoWm-gY"},"source":["#### 3.4 Reshape\n","\n","In this task, you will implement the reshape function for torch tensors.\n","\n","\n","\n","\n","**Your Task:** Implement the function `PYTORCH_reshape`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"jij9cDVIm-gY"},"outputs":[],"source":["def PYTORCH_reshape(X,new_shape):\n"," \"\"\"\n"," Reorganizes the tensor's elements to match a specified shape while maintaining the same number of elements.\n","\n"," Parameters:\n"," x (torch.Tensor): 3-dimensional torch tensor.\n","\n"," new_shape (tuple): tuple with 3 elements which represents the new shape of the tensor.\n","\n"," Returns:\n"," torch.Tensor: torch tensor of dimension 'new_shape'.\n"," \"\"\"\n"," return NotImplemented #TODO"]},{"cell_type":"markdown","metadata":{"id":"19T3Ud8Im-gY"},"source":["#### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"CTBeU7bmm-gY","outputId":"f0f7ba25-694a-4a80-d377-27a87fc3dbf3"},"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([[[5, 0, 3, 3, 7],\n"," [9, 3, 5, 2, 4],\n"," [7, 6, 8, 8, 1]],\n","\n"," [[6, 7, 7, 8, 1],\n"," [5, 9, 8, 9, 4],\n"," [3, 0, 3, 5, 0]]])\n"]}],"source":["np.random.seed(0)\n","X = np.random.randint(0, 10, size=(3,10))\n","\n","X = numpy2tensor(X)\n","\n","\n","\n","print(PYTORCH_reshape(X,(2,3,5)))"]},{"cell_type":"markdown","metadata":{"id":"tdvGcavRm-gY"},"source":["**Expected Output**:\n","\n"," \n"," \n","
PYTORCH_reshape(X,new_shape) \n"," [[[5, 0, 3, 3, 7 ],
\n","   [9, 3, 5, 2, 4 ],
\n","   [7, 6, 8, 8, 1 ]],
\n","
\n","  [[6, 7, 7, 8, 1 ],
\n","   [5, 9, 8, 9, 4 ],
\n","   [3, 0, 3, 5, 0 ]]]\n"," \n","
"]},{"cell_type":"markdown","metadata":{"id":"d314jTpPm-gY"},"source":["#### 3.5 Transpose\n","\n","In this task, you will implement the transpose function for torch tensors.\n","\n","\n","\n","\n","**Your Task:** Implement the function `PYTORCH_transpose`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"rGwokl5Hm-gY"},"outputs":[],"source":["def PYTORCH_transpose(X,dim0,dim1):\n"," \"\"\"\n"," Reorganizes the tensor's elements and shape effectively by swapping along given direction.\n","\n"," Parameters:\n"," x (torch.Tensor): 3-dimensional torch tensor.\n","\n"," dim0 (int): the first dimension to be transposed.\n","\n"," dim1 (int): the second dimension to be transposed.\n","\n"," Returns:\n"," torch.Tensor: torch tensor of transposed version input.\n"," \"\"\"\n"," return NotImplemented #TODO"]},{"cell_type":"markdown","metadata":{"id":"DYLk7AHHm-gY"},"source":["#### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"0dQ_rDrbm-gY","outputId":"48b38fd0-5216-4e83-af7d-ec49df136974"},"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([[[5, 7, 2],\n"," [0, 9, 4],\n"," [3, 3, 7],\n"," [3, 5, 6]],\n","\n"," [[8, 7, 5],\n"," [8, 7, 9],\n"," [1, 8, 8],\n"," [6, 1, 9]]])\n"]}],"source":["np.random.seed(0)\n","X = np.random.randint(0, 10, size=(2,3,4))\n","\n","X = numpy2tensor(X)\n","\n","\n","print(PYTORCH_transpose(X,1,2))"]},{"cell_type":"markdown","metadata":{"id":"FxRUq3Tcm-gZ"},"source":["**Expected Output**:\n","\n"," \n"," \n","
PYTORCH_transpose(X,dim0,dim1) \n"," [[[5, 7, 2,],
\n","   [0, 9, 4,],
\n","   [3, 3, 7,],
\n","   [3, 5, 6,]],
\n","
\n","  [[8, 7, 5,],
\n","   [8, 7, 9,],
\n","   [1, 8, 8,],
\n","   [6, 1, 9,]]]
\n"," \n","
"]},{"cell_type":"markdown","metadata":{"id":"_ffbinT8m-gZ"},"source":["#### 3.6 Permute\n","\n","In this task, you will implement the permute function for torch tensors.\n","\n","\n","\n","\n","**Your Task:** Implement the function `PYTORCH_permute`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"StsE7aCWm-gZ"},"outputs":[],"source":["def PYTORCH_permute(X,dims):\n"," \"\"\"\n"," Reorganizes the the dimensions of a tensor according to a specified permutation tuple while maintaining the data's order.\n","\n"," Parameters:\n"," x (torch.Tensor): 3-dimensional torch tensor.\n","\n"," dims (tuple): tuple of integers that represents axis to be permuted\n","\n"," Returns:\n"," torch.Tensor: torch tensor of permuted version input.\n"," \"\"\"\n"," return NotImplemented #TODO"]},{"cell_type":"markdown","metadata":{"id":"AlFXHKwem-gZ"},"source":["#### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"PZnT8njwm-gZ","outputId":"707f1f27-79b0-4d3e-f007-ec0eff516c7c"},"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([[[5, 7, 2],\n"," [8, 7, 5]],\n","\n"," [[0, 9, 4],\n"," [8, 7, 9]],\n","\n"," [[3, 3, 7],\n"," [1, 8, 8]],\n","\n"," [[3, 5, 6],\n"," [6, 1, 9]]])\n"]}],"source":["np.random.seed(0)\n","X = np.random.randint(0, 10, size=(2,3,4))\n","\n","X = numpy2tensor(X)\n","\n","\n","print(PYTORCH_permute(X,(2,0,1)))"]},{"cell_type":"markdown","metadata":{"id":"SdB3tN1dm-gZ"},"source":["**Expected Output**:\n","\n"," \n"," \n","
PYTORCH_permute(X,dims) \n"," [[[5, 7, 2,],
\n","   [8, 7, 5,]],
\n","
\n","  [[0, 9, 4,],
\n","   [8, 7, 9,]],
\n","
\n","  [[3, 3, 7,],
\n","   [1, 8, 8,]],
\n","
\n","  [[3, 5, 6,],
\n","   [6, 1, 9,]]]
\n"," \n","
"]},{"cell_type":"markdown","metadata":{"id":"ieeiAncym-gZ"},"source":["#### 3.7 Concatenate\n","\n","In this task, you will implement the Concatenate function for torch tensors.\n","\n","\n","\n","\n","**Your Task:** Implement the function `PYTORCH_concatenate`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"3VHT1QZTm-gZ"},"outputs":[],"source":["def PYTORCH_concatenate(tensors,dim):\n"," \"\"\"\n"," Reorganizes the the dimensions of a tensor according to a specified permutation tuple while maintaining the data's order.\n","\n"," Parameters:\n"," tensors (tuple): tuple of Tensors with same shape except in concatenate dimension.\n","\n"," dim (int): concatenate dimension\n","\n"," Returns:\n"," torch.Tensor: concatenated tensor.\n"," \"\"\"\n"," return NotImplemented #TODO"]},{"cell_type":"markdown","metadata":{"id":"6b9JuLVim-gZ"},"source":["#### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"LEJXzoXRm-ga","outputId":"e89a3532-2c65-41ae-c039-a3cbaba84c2d"},"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([[[5, 0, 3, 3, 0, 0, 1],\n"," [7, 9, 3, 5, 2, 0, 2],\n"," [2, 4, 7, 6, 0, 1, 1]],\n","\n"," [[8, 8, 1, 6, 2, 0, 1],\n"," [7, 7, 8, 1, 1, 1, 0],\n"," [5, 9, 8, 9, 2, 0, 2]]])\n"]}],"source":["np.random.seed(0)\n","X = np.random.randint(0, 10, size=(2,3,4))\n","Y = np.random.randint(0, 3, size=(2,3,3))\n","X = numpy2tensor(X)\n","Y = numpy2tensor(Y)\n","\n","\n","print(PYTORCH_concatenate((X,Y),2))"]},{"cell_type":"markdown","metadata":{"id":"Nui-chnJm-ga"},"source":["**Expected Output**:\n","\n"," \n"," \n","
PYTORCH_concatenate(tensors,dim) \n"," [[[5, 0, 3, 3, 0, 0, 1],
\n","   [7, 9, 3, 5, 2, 0, 2],
\n","   [2, 4, 7, 6, 0, 1, 1]],
\n","
\n","  [[8, 8, 1, 6, 2, 0, 1],
\n","   [7, 7, 8, 1, 1, 1, 0],
\n","   [5, 9, 8, 9, 2, 0, 2]]],
\n"," \n"," \n","
"]},{"cell_type":"markdown","metadata":{"id":"KAnK2um8m-ga"},"source":["#### 3.8 Stack\n","\n","In this task, you will implement the stack function for torch tensors.\n","\n","\n","\n","\n","**Your Task:** Implement the function `PYTORCH_stack`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"nOYp8Mr0m-ga"},"outputs":[],"source":["def PYTORCH_stack(tensors,dim):\n","\n"," \"\"\"\n"," In contrast to Concatenation, which merges two tensors along an existing dimension,\n"," Stack creates a new dimension to combine tensors.\n","\n"," Parameters:\n"," tensors (tuple): tuple of tensors to be stacked\n"," dim (int): dimension to insert.\n","\n"," Returns:\n"," torch.Tensor: stacked tensor.\n"," \"\"\"\n","\n"," return NotImplemented #TODO\n"]},{"cell_type":"markdown","metadata":{"id":"w6Q152R9m-ga"},"source":["#### Test Example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"3hZe54j-m-ga","outputId":"14704ecd-56ed-49a2-fcb7-054135a25c9b"},"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([[[5, 0, 3],\n"," [1, 2, 0]],\n","\n"," [[3, 7, 9],\n"," [2, 0, 0]]])\n"]}],"source":["np.random.seed(0)\n","X = np.random.randint(0, 10, size=(2,3))\n","Y = np.random.randint(0, 3, size=(2,3))\n","X = numpy2tensor(X)\n","Y = numpy2tensor(Y)\n","\n","\n","print(PYTORCH_stack((X,Y),1))"]},{"cell_type":"markdown","metadata":{"id":"vHG_DL7ym-ga"},"source":["**Expected Output**:\n","\n"," \n"," \n","
PYTORCH_stack(tensors,dims) \n"," [[[5, 0, 3,],
\n","   [1, 2, 0,]],
\n","
\n","  [[3, 7, 9,],
\n","   [2, 0, 0,]]],
\n","
\n"," \n","
"]},{"cell_type":"markdown","metadata":{"id":"fexVGEz_m-ga"},"source":["#### 3.9 Padding\n","\n","In this task, you will implement the padding function for torch tensors.\n","\n","\n","\n","\n","**Your Task:** Implement the function `PYTORCH_padding`."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"DAyLlSBom-ga"},"outputs":[],"source":["def PYTORCH_padding(X,pad_widths):\n","\n"," \"\"\"\n"," This involves adding extra elements (usually zeros) around the edges of a tensor.\n"," We will encounter this during convolutional operations to control the spatial dimensions of the output.\n","\n"," Parameters:\n"," X (torch.Tensor): input tensor which is to be padded.\n"," pad_widths (tuple): even number of elements in tuple that represents padding widths in the order columns,rows and channels.\n","\n"," Returns:\n"," torch.Tensor: stacked tensor.\n"," \"\"\"\n","\n"," return NotImplemented #TODO"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"h5AlNAnim-gb","outputId":"3d66123f-c95b-46f0-ec17-2d9322dc7312"},"outputs":[{"name":"stdout","output_type":"stream","text":["tensor([[[0, 0, 0, 0, 0, 0],\n"," [0, 0, 6, 1, 0, 0],\n"," [0, 0, 4, 4, 0, 0],\n"," [0, 0, 0, 0, 0, 0]],\n","\n"," [[0, 0, 0, 0, 0, 0],\n"," [0, 0, 8, 4, 0, 0],\n"," [0, 0, 6, 3, 0, 0],\n"," [0, 0, 0, 0, 0, 0]]])\n"]}],"source":["np.random.seed(0)\n","X = np.random.randint(1, 10, size=(2,2,2))\n","\n","X = numpy2tensor(X)\n","\n","\n","print(PYTORCH_padding(X,(2,2,1,1,0,0)))"]},{"cell_type":"markdown","metadata":{"id":"hna8R6vdm-gb"},"source":["**Expected Output**:\n","\n"," \n"," \n"," \n"," \n","
PYTORCH_padding(X,pad_widths) \n"," [[[0, 0, 0, 0, 0, 0],
\n","  [0, 0, 6, 1, 0, 0],
\n","  [0, 0, 4, 4, 0, 0],
\n","  [0, 0, 0, 0, 0, 0]]
\n","
\n"," [[0, 0, 0, 0, 0, 0],
\n","  [0, 0, 8, 4, 0, 0],
\n","  [0, 0, 6, 3, 0, 0],
\n","  [0, 0, 0, 0, 0, 0]]]\n","
"]},{"cell_type":"markdown","source":[],"metadata":{"id":"CVR4lkxlQx2a"}},{"cell_type":"markdown","source":["### **4. Build a Neural Network**\n","\n","Now we go through the basic concepts and functionalities of PyTorch to build a neural network. By the end of this notebook, you'll understand some basics on dataloaders/datasets, build simple neural networks, and train a model."],"metadata":{"id":"B76sGTAKRbzg"}},{"cell_type":"markdown","source":["#### 4.1. Setting Up the Environment\n","\n","First, we need to install PyTorch. You can install it using pip. Run the following command in your terminal or in a code cell:"],"metadata":{"id":"1gX1CJmOSwRd"}},{"cell_type":"code","source":["!pip install torch torchvision"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"collapsed":true,"id":"df8fPMfDTAMU","outputId":"1b75b6fa-db52-4c13-91f2-b179d55cf770"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (2.5.1+cu121)\n","Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (0.20.1+cu121)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch) (3.16.1)\n","Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch) (4.12.2)\n","Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch) (3.4.2)\n","Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch) (3.1.4)\n","Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch) (2024.10.0)\n","Requirement already satisfied: sympy==1.13.1 in /usr/local/lib/python3.10/dist-packages (from torch) (1.13.1)\n","Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy==1.13.1->torch) (1.3.0)\n","Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from torchvision) (1.26.4)\n","Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.10/dist-packages (from torchvision) (11.0.0)\n","Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch) (3.0.2)\n"]}]},{"cell_type":"markdown","source":["#### 4.2. Importing PyTorch\n","\n","Now, let's import necessary PyTorch lybraies and check for CUDA"],"metadata":{"id":"ZQmljioHTR0X"}},{"cell_type":"code","source":["import torch\n","import torch.nn as nn\n","import torch.optim as optim\n","import torch.nn.functional as F\n","import torchvision\n","import torchvision.transforms as transforms"],"metadata":{"id":"iY_kpQ1CTqnH"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n","device"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"0diQ98STTuSA","outputId":"53995f8f-6942-448d-f864-da2b2ae0743c"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["device(type='cpu')"]},"metadata":{},"execution_count":5}]},{"cell_type":"markdown","source":["#### 4.3. Building a Neural Network\n","\n","Now, let's build a simple neural network using PyTorch. We'll create a basic feedforward network to classify handwritten digits from the MNIST dataset."],"metadata":{"id":"mES0vjJqTwG4"}},{"cell_type":"code","source":["class SimpleNN(nn.Module):\n"," def __init__(self):\n"," super(SimpleNN, self).__init__()\n"," self.fc1=nn.Linear(28*28, 128)\n"," self.fc2=nn.Linear(128,64)\n"," self.fc3=nn.Linear(64,10)\n","\n"," def forward(self, x):\n"," x=x.view(-1, 28*28)\n"," x=F.relu(self.fc1(x))\n"," x=F.relu(self.fc2(x))\n"," x=self.fc3(x)\n"," return x\n","\n","net=SimpleNN().to(device)"],"metadata":{"id":"RQJkH0GDT6Ui"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["#### 4.4. Loading the Dataset"],"metadata":{"id":"m0DrXL1WUCff"}},{"cell_type":"code","source":["transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))])\n","\n","trainset = torchvision.datasets.MNIST(root='./data', train=True, download=True, transform=transform)\n","trainloader = torch.utils.data.DataLoader(trainset, batch_size=32, shuffle=True)\n","\n","testset = torchvision.datasets.MNIST(root='./data', train=False, download=True, transform=transform)\n","testloader = torch.utils.data.DataLoader(testset, batch_size=32, shuffle=False)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"collapsed":true,"id":"2rIn2H20URNn","outputId":"1c5bdac1-e2b3-4d6e-c4d7-f85f103aa842"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n","Failed to download (trying next):\n","HTTP Error 403: Forbidden\n","\n","Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz\n","Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz to ./data/MNIST/raw/train-images-idx3-ubyte.gz\n"]},{"output_type":"stream","name":"stderr","text":["100%|██████████| 9.91M/9.91M [00:00<00:00, 15.8MB/s]\n"]},{"output_type":"stream","name":"stdout","text":["Extracting ./data/MNIST/raw/train-images-idx3-ubyte.gz to ./data/MNIST/raw\n","\n","Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n","Failed to download (trying next):\n","HTTP Error 403: Forbidden\n","\n","Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-labels-idx1-ubyte.gz\n","Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-labels-idx1-ubyte.gz to ./data/MNIST/raw/train-labels-idx1-ubyte.gz\n"]},{"output_type":"stream","name":"stderr","text":["100%|██████████| 28.9k/28.9k [00:00<00:00, 525kB/s]\n"]},{"output_type":"stream","name":"stdout","text":["Extracting ./data/MNIST/raw/train-labels-idx1-ubyte.gz to ./data/MNIST/raw\n","\n","Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n","Failed to download (trying next):\n","HTTP Error 403: Forbidden\n","\n","Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-images-idx3-ubyte.gz\n","Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw/t10k-images-idx3-ubyte.gz\n"]},{"output_type":"stream","name":"stderr","text":["100%|██████████| 1.65M/1.65M [00:00<00:00, 3.78MB/s]\n"]},{"output_type":"stream","name":"stdout","text":["Extracting ./data/MNIST/raw/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw\n","\n","Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n","Failed to download (trying next):\n","HTTP Error 403: Forbidden\n","\n","Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-labels-idx1-ubyte.gz\n","Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"]},{"output_type":"stream","name":"stderr","text":["100%|██████████| 4.54k/4.54k [00:00<00:00, 1.40MB/s]\n"]},{"output_type":"stream","name":"stdout","text":["Extracting ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw\n","\n"]}]},{"cell_type":"markdown","source":["#### 4.5. Defining the Loss Function and Optimizer"],"metadata":{"id":"9rFEVYOeTCU8"}},{"cell_type":"code","source":["criterion=nn.CrossEntropyLoss()\n","optimizer=optim.SGD(net.parameters(), lr=0.01, momentum=0.9)"],"metadata":{"id":"0Q-6cXj0Ui4_"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["#### 4.6. Training the network"],"metadata":{"id":"z7SslhphUmgp"}},{"cell_type":"code","source":["for epoch in range(5): #Loop over the dataset multiple time\n"," running_loss=0.0\n"," for i, data in enumerate(trainloader, 0):\n"," inputs, lables=data\n"," inputs, labels=inputs.to(device), lables.to(device)\n"," optimizer.zero_grad()\n"," outputs=net(inputs)\n"," loss=criterion(outputs, labels)\n"," loss.backward()\n"," optimizer.step()\n","\n"," running_loss +=loss.item()\n"," if i%100 ==99: #print every 100 miini_batches\n"," print(f'[Epoch {epoch+1}, Batch {i+1}] loss: {running_loss/100:.3f}')\n"," running_loss=0.0\n","print('Finished Training')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"collapsed":true,"id":"DytxDnj5UpvM","outputId":"c4b700f3-d18a-41d1-f73e-0d237e815c7b"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["[Epoch 1, Batch 100] loss: 1.447\n","[Epoch 1, Batch 200] loss: 0.534\n","[Epoch 1, Batch 300] loss: 0.497\n","[Epoch 1, Batch 400] loss: 0.407\n","[Epoch 1, Batch 500] loss: 0.354\n","[Epoch 1, Batch 600] loss: 0.311\n","[Epoch 1, Batch 700] loss: 0.319\n","[Epoch 1, Batch 800] loss: 0.300\n","[Epoch 1, Batch 900] loss: 0.274\n","[Epoch 1, Batch 1000] loss: 0.272\n","[Epoch 1, Batch 1100] loss: 0.231\n","[Epoch 1, Batch 1200] loss: 0.236\n","[Epoch 1, Batch 1300] loss: 0.214\n","[Epoch 1, Batch 1400] loss: 0.222\n","[Epoch 1, Batch 1500] loss: 0.204\n","[Epoch 1, Batch 1600] loss: 0.203\n","[Epoch 1, Batch 1700] loss: 0.233\n","[Epoch 1, Batch 1800] loss: 0.219\n","[Epoch 2, Batch 100] loss: 0.169\n","[Epoch 2, Batch 200] loss: 0.186\n","[Epoch 2, Batch 300] loss: 0.181\n","[Epoch 2, Batch 400] loss: 0.149\n","[Epoch 2, Batch 500] loss: 0.174\n","[Epoch 2, Batch 600] loss: 0.160\n","[Epoch 2, Batch 700] loss: 0.184\n","[Epoch 2, Batch 800] loss: 0.148\n","[Epoch 2, Batch 900] loss: 0.158\n","[Epoch 2, Batch 1000] loss: 0.148\n","[Epoch 2, Batch 1100] loss: 0.131\n","[Epoch 2, Batch 1200] loss: 0.144\n","[Epoch 2, Batch 1300] loss: 0.166\n","[Epoch 2, Batch 1400] loss: 0.164\n","[Epoch 2, Batch 1500] loss: 0.155\n","[Epoch 2, Batch 1600] loss: 0.176\n","[Epoch 2, Batch 1700] loss: 0.135\n","[Epoch 2, Batch 1800] loss: 0.129\n","[Epoch 3, Batch 100] loss: 0.138\n","[Epoch 3, Batch 200] loss: 0.105\n","[Epoch 3, Batch 300] loss: 0.116\n","[Epoch 3, Batch 400] loss: 0.131\n","[Epoch 3, Batch 500] loss: 0.123\n","[Epoch 3, Batch 600] loss: 0.125\n","[Epoch 3, Batch 700] loss: 0.100\n","[Epoch 3, Batch 800] loss: 0.111\n","[Epoch 3, Batch 900] loss: 0.124\n","[Epoch 3, Batch 1000] loss: 0.114\n","[Epoch 3, Batch 1100] loss: 0.107\n","[Epoch 3, Batch 1200] loss: 0.128\n","[Epoch 3, Batch 1300] loss: 0.131\n","[Epoch 3, Batch 1400] loss: 0.112\n","[Epoch 3, Batch 1500] loss: 0.114\n","[Epoch 3, Batch 1600] loss: 0.134\n","[Epoch 3, Batch 1700] loss: 0.095\n","[Epoch 3, Batch 1800] loss: 0.123\n","[Epoch 4, Batch 100] loss: 0.110\n","[Epoch 4, Batch 200] loss: 0.082\n","[Epoch 4, Batch 300] loss: 0.094\n","[Epoch 4, Batch 400] loss: 0.098\n","[Epoch 4, Batch 500] loss: 0.093\n","[Epoch 4, Batch 600] loss: 0.091\n","[Epoch 4, Batch 700] loss: 0.105\n","[Epoch 4, Batch 800] loss: 0.098\n","[Epoch 4, Batch 900] loss: 0.099\n","[Epoch 4, Batch 1000] loss: 0.092\n","[Epoch 4, Batch 1100] loss: 0.110\n","[Epoch 4, Batch 1200] loss: 0.105\n","[Epoch 4, Batch 1300] loss: 0.096\n","[Epoch 4, Batch 1400] loss: 0.099\n","[Epoch 4, Batch 1500] loss: 0.096\n","[Epoch 4, Batch 1600] loss: 0.118\n","[Epoch 4, Batch 1700] loss: 0.097\n","[Epoch 4, Batch 1800] loss: 0.119\n","[Epoch 5, Batch 100] loss: 0.077\n","[Epoch 5, Batch 200] loss: 0.078\n","[Epoch 5, Batch 300] loss: 0.085\n","[Epoch 5, Batch 400] loss: 0.073\n","[Epoch 5, Batch 500] loss: 0.085\n","[Epoch 5, Batch 600] loss: 0.086\n","[Epoch 5, Batch 700] loss: 0.082\n","[Epoch 5, Batch 800] loss: 0.070\n","[Epoch 5, Batch 900] loss: 0.075\n","[Epoch 5, Batch 1000] loss: 0.075\n","[Epoch 5, Batch 1100] loss: 0.095\n","[Epoch 5, Batch 1200] loss: 0.072\n","[Epoch 5, Batch 1300] loss: 0.095\n","[Epoch 5, Batch 1400] loss: 0.088\n","[Epoch 5, Batch 1500] loss: 0.080\n","[Epoch 5, Batch 1600] loss: 0.074\n","[Epoch 5, Batch 1700] loss: 0.079\n","[Epoch 5, Batch 1800] loss: 0.084\n","Finished Training\n"]}]},{"cell_type":"code","source":["correct = 0\n","total = 0\n","with torch.no_grad():\n"," for data in testloader:\n"," images, labels = data\n"," images, labels = images.to(device), labels.to(device)\n"," outputs = net(images)\n"," _, predicted = torch.max(outputs.data, 1)\n"," total += labels.size(0)\n"," correct += (predicted == labels).sum().item()\n","\n","print(f'Accuracy of the network on the 10000 test images: {100 * correct / total:.2f}%')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"LEY8tRn7Uu97","outputId":"4422bc95-4a5e-430b-8304-f6e92d00386b"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Accuracy of the network on the 10000 test images: 96.88%\n"]}]},{"cell_type":"markdown","source":["#### 4.6. Conclusion\n","\n","Bravo! Just now, using PyTorch, you constructed and trained a basic neural network. We discussed:\n","\n","\n","Build and work with tensors\n","\n","Constructing a neural network\n","\n","Testing and educating the network\n","\n","Try changing the training parameters and network architecture to observe how it affects performance. Cheers to learning!"],"metadata":{"id":"5NN6vUzQU04u"}}],"metadata":{"kernelspec":{"display_name":"cve","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.15"},"orig_nbformat":4,"colab":{"provenance":[]}},"nbformat":4,"nbformat_minor":0}