{"cells":[{"attachments":{},"cell_type":"markdown","metadata":{},"source":["# Specific Test V. Exploring Transformers\n","\n","> This is a copy of the training notebook used for Specific Test 5 of the DeepLense tests. The original notebook, datasets and results can be found [here](https://github.com/gauthamk02/deeplense-test-2023)\n","\n","**Task:** Use a vision transformer method of your choice to build a robust and efficient model for binary classification or unsupervised anomaly detection on the provided dataset. In the case of unsupervised anomaly detection, train your model to learn the distribution of the provided strong lensing images with no substructure. Please implement your approach in PyTorch or Keras and discuss your strategy.\n","\n","**Dataset Description:** A set of simulated strong gravitational lensing images with and without substructure. \n","\n","**Evaluation Metrics:** ROC curve (Receiver Operating Characteristic curve) and AUC score (Area Under the ROC Curve)"]},{"cell_type":"code","execution_count":27,"metadata":{"execution":{"iopub.execute_input":"2023-03-11T09:43:14.076073Z","iopub.status.busy":"2023-03-11T09:43:14.075565Z","iopub.status.idle":"2023-03-11T09:43:24.878632Z","shell.execute_reply":"2023-03-11T09:43:24.877344Z","shell.execute_reply.started":"2023-03-11T09:43:14.076037Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Requirement already satisfied: einops in /opt/conda/lib/python3.7/site-packages (0.6.0)\n","\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n","\u001b[0m"]}],"source":["!pip install einops"]},{"cell_type":"code","execution_count":28,"metadata":{"execution":{"iopub.execute_input":"2023-03-11T09:43:24.883148Z","iopub.status.busy":"2023-03-11T09:43:24.882826Z","iopub.status.idle":"2023-03-11T09:43:24.890557Z","shell.execute_reply":"2023-03-11T09:43:24.889077Z","shell.execute_reply.started":"2023-03-11T09:43:24.883115Z"},"trusted":true},"outputs":[],"source":["import torch\n","import torchvision.transforms as transforms\n","from torch.utils.data import DataLoader, Dataset\n","import matplotlib.pyplot as plt\n","from sklearn.model_selection import train_test_split\n","import numpy as np\n","import pandas as pd\n","import os\n","from PIL import Image\n","from tqdm import tqdm\n","import seaborn as sns\n","\n","device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"]},{"cell_type":"markdown","metadata":{},"source":["## Loading Data"]},{"cell_type":"code","execution_count":29,"metadata":{"execution":{"iopub.execute_input":"2023-03-11T09:43:24.892682Z","iopub.status.busy":"2023-03-11T09:43:24.892290Z","iopub.status.idle":"2023-03-11T09:43:24.942546Z","shell.execute_reply":"2023-03-11T09:43:24.941484Z","shell.execute_reply.started":"2023-03-11T09:43:24.892644Z"},"trusted":true},"outputs":[{"data":{"text/html":["
\n"," | path | \n","label | \n","
---|---|---|
0 | \n","/kaggle/input/ml4sci-test5/lenses/no_sub/image... | \n","no_sub | \n","
1 | \n","/kaggle/input/ml4sci-test5/lenses/no_sub/image... | \n","no_sub | \n","
2 | \n","/kaggle/input/ml4sci-test5/lenses/no_sub/image... | \n","no_sub | \n","
3 | \n","/kaggle/input/ml4sci-test5/lenses/no_sub/image... | \n","no_sub | \n","
4 | \n","/kaggle/input/ml4sci-test5/lenses/no_sub/image... | \n","no_sub | \n","