I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
.requires_grad_( ) changes an existing Tensor’s requires_grad flag in-place. The input flag defaults to False if not given.
Tensor and Function are interconnected and build up an acyclic graph, that encodes a complete history of
computation. Each tensor has a .grad_fn attribute that references a Function that has created the Tensor
(except for Tensors created by the user - their grad_fn is None).
Can you write Python code for it?

a = torch.randn(2, 2)    # a is created by user, its .grad_fn is None
a = ((a * 3) / (a - 1))
print(a.requires_grad)
a.requires_grad_(True)   # change the attribute .grad_fn of a
print(a.requires_grad)
b = (a * a).sum()        # add all elements of a  to b
print(b.grad_fn)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
1.2. Do a tensor operation:
Can you write Python code for it?

y = x + 2
print(y)

y1 = x1 + 2
print(y1)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
3.1.4. Train function
Train a model and show running_loss curve ana show accuracy curve
Can you write Python code for it?

def train(classifier_net, trainloader, testloader, device, lr, optimizer):
    '''
    Args:
        classifier_net(nn.model): train model
        trainloader(torch.utils.data.DateLoader): train loader
        testloader(torch.utils.data.DateLoader): test loader
        device(torch.device): the evironment your model training
        LR(float): learning rate
    '''
    # loss function
    criterion = nn.CrossEntropyLoss().to(device)

    optimizer = optimizer

    # save the mean value of loss in an epoch
    running_loss = []

    running_accuracy = []

    # count loss in an epoch
    temp_loss = 0.0

    # count the iteration number in an epoch
    iteration = 0

    for epoch in range(epoches):

        '''
        adjust learning rate when you are training the model
        '''
        # adjust learning rate
        # if epoch % 100 == 0 and epoch != 0:
        #     LR = LR * 0.1
        #     for param_group in optimizer.param_groups:
        #         param_group['lr'] = LR

        for i, data in enumerate(trainloader):
            point, label = data['point'], data['label']
            point, label = point.to(device).to(torch.float32), label.to(device)
            outputs = classifier_net(point)
            '''# TODO'''

            # empty parameters in optimizer
            optimizer.zero_grad()

            # calcutate loss value
            loss = criterion(outputs, label)

            # back propagation
            loss.backward()

            # update parameters in optimizer(update weigtht)
            optimizer.step()

            '''# TODO END'''

            # save loss in a list
            temp_loss += loss.item()
            iteration +=1
            # print loss value
#             print('[{0:d},{1:5.0f}] loss {2:.5f}'.format(epoch + 1, i, loss.item()))
            #slow down speed of print function
            # time.sleep(0.5)
        running_loss.append(temp_loss / iteration)
        temp_loss = 0
       ...


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
3.1.5. Test function
Test the performance of your model
Can you write Python code for it?

def predict(classifier_net, testloader, device):
#     correct = [0 for i in range(3)]
#     total = [0 for i in range(3)]
    correct = 0
    total = 0

    with torch.no_grad():
        '''
        you can also stop autograd from tracking history on Tensors with .requires_grad=True
        by wrapping the code block in with torch.no_grad():
        '''
        for data in testloader:
            point, label = data['point'], data['label']
            point, label = point.to(device).to(torch.float32), label.to(device)
            outputs = classifier_net(point)
            '''
            if you want to get probability of the model prediction,
            you can use softmax function here to transform outputs to probability.
            '''
            # transform the prediction to one-hot form
            _, predicted = torch.max(outputs, 1)
            # print('model prediction: ', predicted)
            # print('ground truth:', label, '\n')
            correct += (predicted == label).sum()
            total += label.size(0)
            # print('current correct is:', correct.item())
            # print('current total is:', total)

        # print('the accuracy of the model is {0:5f}'.format(correct.item()/total))

    return correct.item() / total

# show accuracy curve, you can skip this cell.
def show_accuracy(running_accuracy):
    x = np.array([i for i in range(len(running_accuracy))])
    y = np.array(running_accuracy)
    plt.figure()
    plt.plot(x, y, c='b')
    plt.axis()
    plt.title('accuracy curve:')
    plt.xlabel('step')
    plt.ylabel('accuracy value')
    plt.show()


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
4.2. Let’s quickly read the CSV and get the annotations in an (N, 2) array where N is the number of landmarks.
Can you write Python code for it?

# read a csv file by pandas
landmarks_frame = pd.read_csv('data/faces/face_landmarks.csv')

n = 0
# read image name, image name was saved in column 1.
img_name = landmarks_frame.iloc[n, 0]
# points were saved in columns from 2 to the end
landmarks = landmarks_frame.iloc[n, 1:].as_matrix()
# reshape the formate of points
landmarks = landmarks.astype('float').reshape(-1, 2)

print('Image name: {}'.format(img_name))
print('Landmarks shape: {}'.format(landmarks.shape))
print('First 4 Landmarks: {}'.format(landmarks[:4]))

def show_landmarks(image, landmarks):
    """Show image with landmarks"""
    plt.imshow(image)
    plt.scatter(landmarks[:, 0], landmarks[:, 1], s=10, marker='.', c='r')
    plt.pause(0.001)  # pause a bit so that plots are updated

plt.figure()
show_landmarks(io.imread(os.path.join('data/faces/', img_name)),
               landmarks)
plt.show()

class FaceLandmarksDataset(Dataset):
    def __init__(self, csv_file, root_dir, transform=None):
        """
        Args:
            csv_file (string): Path to the csv file with annotations.
            root_dir (string): Directory with all the images.
            transform (callable, optional): Optional transform to be applied
                on a sample.
        """
        self.landmarks_frame = pd.read_csv(csv_file)
        self.root_dir = root_dir
        self.transform = transform

    def __len__(self):
        return len(self.landmarks_frame)

    def __getitem__(self, idx):
        # combine the relative path of images
        img_name = os.path.join(self.root_dir,
                                self.landmarks_frame.iloc[idx, 0])
        image = io.imread(img_name)
        landmarks = self.landmarks_frame.iloc[idx, 1:].as_matrix()
        landmarks = landmarks.astype('float').reshape(-1, 2)
        # save all data we may need during training a network in a dict
        sample = {'image': image, 'landmarks': landmarks}

        if self.transform:
            sample = self.transform(sample)

        return sample


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
Note(very important):
to define a dataset, first we must to inherit the class torch.utils.data.Dataset. when we write ourselves dataset, it's neccesarry for us to overload the ____init____ method, ____len____ method, and ____getitem____ method. Of course you can define other method as you like.
4.3. Let’s instantiate this class and iterate through the data samples. We will print the sizes of first 4 samples and show their landmarks.
Can you write Python code for it?

face_dataset = FaceLandmarksDataset(csv_file='data/faces/face_landmarks.csv',
                                    root_dir='data/faces/')

fig = plt.figure()

for i in range(len(face_dataset)):
    sample = face_dataset[i]

    print(i, sample['image'].shape, sample['landmarks'].shape)

    # create subgraph
    ax = plt.subplot(1, 4, i + 1)
    plt.tight_layout()
    ax.set_title('Sample #{}'.format(i))
    ax.axis('off')
    show_landmarks(**sample)

    if i == 3:
        plt.show()
        break


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
y was created as a result of an operation, so it has a grad_fn.But y1 not
Can you write Python code for it?


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
作业要求:
4. DATA LOADING AND PROCESSING TUTORIAL
（Further content, read it when you are free）
A lot of effort in solving any machine learning problem goes in to preparing the data. PyTorch provides many tools to make data loading easy and hopefully, to make your code more readable. In this tutorial, we will see how to load and preprocess/augment data from a non trivial dataset.
4.1. To run this tutorial, please make sure the following packages are installed:

scikit-image: For image io and transforms

sudo apt-get install python-numpy

sudo apt-get install python-scipy

sudo apt-get install python-matplotlib

sudo pip install  scikit-image

pandas: For easier csv parsing

sudo apt-get install python-pandas
Can you write Python code for it?

import os
import torch
import pandas as pd
from skimage import io, transform
import numpy as np
import matplotlib.pyplot as plt
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms, utils

plt.ion()   # interactive mode


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
当variables是张量的时候，.backward()传递的参数grad_variables不可省略，且张量大小必须与variables一致。该参数也是系数，表示x各分量的权重。
Can you write Python code for it?

x = torch.ones(2, requires_grad=True)
t = x + torch.tensor([1., 2.])
y = t * t + 2 * t + 4
y.backward(torch.tensor([1., 0.1]), retain_graph=True)
print(x.grad)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
问题1：
传入 .backward()里面的tensor是什么？请尝试不同的输入并回答。
当variables是标量的时候，.backward()传递的参数grad_variables可省略。该参数实际上可表示为下面代码中的z和x的函数关系的系数，默认为1.0。
下面代码对于$y=x^2+2x+4$进行求导，可以观察到指定权重为2时，梯度是原来的两倍，即对$y=2x^2+4x+8$求导。
Can you write Python code for it?

x = torch.ones(1, requires_grad=True)
y = x * x + 2 * x + 4
y.backward(retain_graph=True)
print(x.grad)

x.grad.data.zero_()

y.backward(torch.tensor([2.0]), retain_graph=True)
print(x.grad)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
问题2:
根据本节前面提到的训练一个网络的完整流程，将下面的代码按照正确的顺序填写到train()函数里面的 # TODO 里面
# update parameters in optimizer(update weigtht)
optimizer.step()
# calcutate loss value
loss = criterion(outputs, label)
# empty parameters in optimizer
optimizer.zero_grad()
# back propagation
loss.backward()
Can you write Python code for it?

# show running loss curve, you can skip this cell.
def show_running_loss(running_loss):
    # generate x value
    x = np.array([i for i in range(len(running_loss))])
    # generate y value
    y = np.array(running_loss)
    # define a graph
    plt.figure()
    # generate curve
    plt.plot(x, y, c='b')
    # show axis
    plt.axis()
    # define title
    plt.title('loss curve:')
    #define the name of x axis
    plt.xlabel('step')
    plt.ylabel('loss value')
    # show graph
    plt.show()


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
问题3:
请尝试调整不同大小的学习率，观察loss曲线和accuracy曲线，并阐述学习率对loss值以及accuracy值的影响和原因。
下面调节学习率至原来的一半。
可以观察到学习率越高，损失值下降越快，而准确率提高越快；而学习率下降，损失值下降变得更加缓慢，准确率提高也变得缓慢。当学习率越高时，权重修改的大小就越大，因此损失值和准确率变化就更加快，模型训练很容易就达到了最优解附近。
其次，学习率过高时，会导致损失值和准确率存在波动。由于学习率过高，在训练的过程中权重修改很容易导致左右摆动，在最低点左右迭代，因此产生震荡。学习率过低，会导致模型训练完后还达不到想要的效果。
在本次数据中，上面代码给定的1e-3的学习率过低，导致最终的损失值在0.2~0.4左右，而将学习率调为0.01，不仅损失值下降快，而且最终效果要比前者好一点。
Can you write Python code for it?

epoches = 100
lr = 0.01
batch_size = 16

transform = torch.tensor
device = torch.device('cuda:0')
trainset = PointDataset('./labels/train.csv', transform=transform)
trainloader = DataLoader(dataset=trainset, batch_size=batch_size, shuffle=True)
testset = PointDataset('./labels/test.csv', transform=transform)
testloader = DataLoader(dataset=testset, batch_size=batch_size, shuffle=False)
show_original_points()

classifier_net = Network(2, 5, 3).to(device)
optimizer = optim.SGD(classifier_net.parameters(), lr=lr, momentum=0.9)
classifier_net = train(classifier_net, trainloader, testloader, device, lr, optimizer)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
问题6:
尝试使用Adam，Rprop优化器，观察两种曲线，阐述SGD,Adam,Rprop三种优化器对loss值以及accuracy值的影响.
Adam效率最差，SGD次之，而Rprop优化器的效率最高，可以更快得到更低的损失值和更高的准确率。
Can you write Python code for it?

epoches = 100
lr = 1e-3
batch_size = 16

transform = torch.tensor
device = torch.device('cuda:0')
trainset = PointDataset('./labels/train.csv', transform=transform)
trainloader = DataLoader(dataset=trainset, batch_size=batch_size, shuffle=True)
testset = PointDataset('./labels/test.csv', transform=transform)
testloader = DataLoader(dataset=testset, batch_size=batch_size, shuffle=False)
show_original_points()

classifier_net = Network(2, 5, 3).to(device)
optimizer = optim.Adam(classifier_net.parameters(), lr=lr)
classifier_net = train(classifier_net, trainloader, testloader, device, lr, optimizer)

epoches = 100
lr = 1e-3
batch_size = 16

transform = torch.tensor
device = torch.device('cuda:0')
trainset = PointDataset('./labels/train.csv', transform=transform)
trainloader = DataLoader(dataset=trainset, batch_size=batch_size, shuffle=True)
testset = PointDataset('./labels/test.csv', transform=transform)
testloader = DataLoader(dataset=testset, batch_size=batch_size, shuffle=False)
show_original_points()

classifier_net = Network(2, 5, 3).to(device)
optimizer = optim.Rprop(classifier_net.parameters(), lr=lr)
classifier_net = train(classifier_net, trainloader, testloader, device, lr, optimizer)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 3 Backpropagation/week_3_pytorch.ipynb".
The first task is:
问题7(自由发挥):
尝试同时调节以上几种参数，找出你认为最合适的参数（模型收敛得最快），并就此谈谈你的感想.
当学习率为0.01、batch大小为30时，采用Rprop优化器的模型收敛最快，且执行速度也不慢。
在训练过程中，调节好学习率可以加快模型收敛速度，而增大batch的大小可以让模型训练时间缩短，但是会讲低收敛速度，甚至可能影响最终准确率。在三种优化器中，Rprop效果最好，SGD次之，Adam效果最差。
Can you write Python code for it?

epoches = 100
lr = 0.01
batch_size = 30

transform = torch.tensor
device = torch.device('cuda:0')
trainset = PointDataset('./labels/train.csv', transform=transform)
trainloader = DataLoader(dataset=trainset, batch_size=batch_size, shuffle=True)
testset = PointDataset('./labels/test.csv', transform=transform)
testloader = DataLoader(dataset=testset, batch_size=batch_size, shuffle=False)
show_original_points()

classifier_net = Network(2, 5, 3).to(device)
optimizer = optim.Rprop(classifier_net.parameters(), lr=lr)
classifier_net = train(classifier_net, trainloader, testloader, device, lr, optimizer)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
1.1 Required Module
numpy: NumPy is the fundamental package for scientific computing in Python.
pytorch: End-to-end deep learning platform.
torchvision: This package consists of popular datasets, model architectures, and common image transformations for computer vision.
tensorflow: An open source machine learning framework.
tensorboard: A suite of visualization tools to make training easier to understand, debug, and optimize TensorFlow programs.
tensorboardX: Tensorboard for Pytorch.
matplotlib: It is a Python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms.
1.2 Common Setup
Can you write Python code for it?

# Load all necessary modules here, for clearness
import torch
import numpy as np
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
# from torchvision.datasets import MNIST
import torchvision
from torchvision import transforms
from torch.optim import lr_scheduler
# from tensorboardX import SummaryWriter
from collections import OrderedDict
import matplotlib.pyplot as plt
# from tqdm import tqdm

# Whether to put data in GPU according to GPU is available or not
# cuda = torch.cuda.is_available()
#  In case the default gpu does not have enough space, you can choose which device to use
# torch.cuda.set_device(device) # device: id

# Since gpu in lab is not enough for your guys, we prefer to cpu computation
cuda = torch.device('cpu')


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
3.3.2 & 3.3.3 compute gradient of loss over parameters & update parameters with gradient descent
Can you write Python code for it?

def train(train_loader, model, loss_fn, optimizer, get_grad=False):
    """train model using loss_fn and optimizer. When thid function is called, model trains for one epoch.
    Args:
        train_loader: train data
        model: prediction model
        loss_fn: loss function to judge the distance between target and outputs
        optimizer: optimize the loss function
        get_grad: True, False
    Returns:
        total_loss: loss
        average_grad2: average grad for hidden 2 in this epoch
        average_grad3: average grad for hidden 3 in this epoch
    """

    # set the module in training model, affecting module e.g., Dropout, BatchNorm, etc.
    model.train()

    total_loss = 0
    grad_2 = 0.0 # store sum(grad) for hidden 3 layer
    grad_3 = 0.0 # store sum(grad) for hidden 3 layer

    for batch_idx, (data, target) in enumerate(train_loader):
        optimizer.zero_grad() # clear gradients of all optimized torch.Tensors'
        outputs = model(data) # make predictions
        loss = loss_fn(outputs, target) # compute loss
        total_loss += loss.item() # accumulate every batch loss in a epoch
        loss.backward() # compute gradient of loss over parameters

        if get_grad == True:
            g2, g3 = model.get_grad() # get grad for hiddern 2 and 3 layer in this batch
            grad_2 += g2 # accumulate grad for hidden 2
            grad_3 += g3 # accumulate grad for hidden 2

        optimizer.step() # update parameters with gradient descent

    average_loss = total_loss / batch_idx # average loss in this epoch
    average_grad2 = grad_2 / batch_idx # average grad for hidden 2 in this epoch
    average_grad3 = grad_3 / batch_idx # average grad for hidden 3 in this epoch

    return average_loss, average_grad2, average_grad3

def evaluate(loader, model, loss_fn):
    """test model's prediction performance on loader.
    When thid function is called, model is evaluated.
    Args:
        loader: data for evaluation
       ...


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
3.4 save model
Pytorch provide two kinds of method to save model. We recommmend the method which only saves parameters. Because it's more feasible and dont' rely on fixed model.
When saving parameters, we not only save learnable parameters in model, but also learnable parameters in optimizer.
A common PyTorch convention is to save models using either a .pt or .pth file extension.
Read more abount save load from this link
Can you write Python code for it?

# show parameters in model

# Print model's state_dict
print("Model's state_dict:")
for param_tensor in model.state_dict():
    print(param_tensor, "\t", model.state_dict()[param_tensor].size())

# Print optimizer's state_dict
print("\nOptimizer's state_dict:")
for var_name in optimizer.state_dict():
    print(var_name, "\t", optimizer.state_dict()[var_name])

# save model

save_path = './model.pt'
torch.save(model.state_dict(), save_path)

# load parameters from files
saved_parametes = torch.load(save_path)
print(saved_parametes)

# initailze model by saved parameters
new_model = FeedForwardNeuralNetwork(input_size, hidden_size, output_size)
new_model.load_state_dict(saved_parametes)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
4. Training Advanced
4.1 l2_norm
we could minimize the regularization term below by use $weight_decay$ in SGD optimizer
\begin{equation}
    L_norm =  {\sum_{i=1}^{m}{\theta_{i}^{2}}}
\end{equation}
set l2_norm=0.01, let's train and see
Can you write Python code for it?

### Hyper parameters
batch_size = 128
n_epochs = 5
learning_rate = 0.01
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0.01 # use l2 penalty
get_grad = False

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)

train_accs, train_losses = fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
4.2 dropout
During training, randomly zeroes some of the elements of the input tensor with probability p using samples from a Bernoulli distribution.
Each channel will be zeroed out independently on every forward call.
Hints: 因为jupyter对变量有上下文关系，模型，优化器需要重新声明。可以使用以下代码进行重新定义模型和优化器。注意到此处用的是默认初始化。
Can you write Python code for it?

### Hyper parameters
batch_size = 128
n_epochs = 5
learning_rate = 0.01
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # without using l2 penalty
get_grad = False

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)

# Set dropout to True and probability = 0.5
model.set_use_dropout(True)

train_accs, train_losses = fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
4.4 data augmentation
data augmentation can be more complicated to gain a better generalization on test dataset
Can you write Python code for it?

# only add random horizontal flip
train_transform_1 = transforms.Compose([
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(), # Convert a PIL Image or numpy.ndarray to tensor.
    # Normalize a tensor image with mean and standard deviation
    transforms.Normalize((0.1307,), (0.3081,))
])

# only add random crop
train_transform_2 = transforms.Compose([
    transforms.RandomCrop(size=[28,28], padding=4),
    transforms.ToTensor(), # Convert a PIL Image or numpy.ndarray to tensor.
    # Normalize a tensor image with mean and standard deviation
    transforms.Normalize((0.1307,), (0.3081,))
])

# add random horizontal flip and random crop
train_transform_3 = transforms.Compose([
    transforms.RandomHorizontalFlip(),
    transforms.RandomCrop(size=[28,28], padding=4),
    transforms.ToTensor(), # Convert a PIL Image or numpy.ndarray to tensor.
    # Normalize a tensor image with mean and standard deviation
    transforms.Normalize((0.1307,), (0.3081,))
])

# reload train_loader using trans

train_dataset_1 = torchvision.datasets.MNIST(root='./data',
                            train=True,
                            transform=train_transform_1,
                            download=False)

train_loader_1 = torch.utils.data.DataLoader(dataset=train_dataset_1,
                                           batch_size=batch_size,
                                           shuffle=True)

print(train_dataset_1)

### Hyper parameters
batch_size = 128
n_epochs = 5
learning_rate = 0.01
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # without using l2 penalty
get_grad = False

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)

train_accs, train_losses = fit(train_loader_1, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
5. Visualizatio of training and validation phase
We could use tensorboard to visualize our training and test phase.
You could find example here
6. Gradient explosion and vanishing
We have embedded code which shows grad for hidden2 and hidden3 layer. By observing their grad changes, we can
see whether gradient is normal or not.
For plot grad changes, you need to set get_grad=True in fit function
Can you write Python code for it?

### Hyper parameters
batch_size = 128
n_epochs = 15
learning_rate = 0.01
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # use l2 penalty
get_grad = True

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)

fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
6.1.1 Gradient Vanishing
Set learning=e-10
Can you write Python code for it?

### Hyper parameters
batch_size = 128
n_epochs = 15
learning_rate = 1e-10
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # use l2 penalty
get_grad = True

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)

fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad=get_grad)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
6.1.2.2 normalization for input data
6.1.2.3 unsuitable weight initialization
Can you write Python code for it?

### Hyper parameters
batch_size = 128
n_epochs = 15
learning_rate = 1
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 0 # not to use l2 penalty
get_grad = True

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)

# reset parameters as 10
def wrong_weight_bias_reset(model):
    """Using normalization with mean=0, std=1 to initialize model's parameter
    """
    for m in model.modules():
        if isinstance(m, nn.Linear):
            # initialize linear layer with mean and std
            mean, std = 0, 1

            # Initialization method
            torch.nn.init.normal_(m.weight, mean, std)
            torch.nn.init.normal_(m.bias, mean, std)

wrong_weight_bias_reset(model)
show_weight_bias(model)

fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad=True)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
Define function fit and use train_epoch and test_epoch
Can you write Python code for it?

def fit(train_loader, val_loader, model, loss_fn, optimizer, n_epochs, get_grad=False):
    """train and val model here, we use train_epoch to train model and
    val_epoch to val model prediction performance
    Args:
        train_loader: train data
        val_loader: validation data
        model: prediction model
        loss_fn: loss function to judge the distance between target and outputs
        optimizer: optimize the loss function
        n_epochs: training epochs
        get_grad: Whether to get grad of hidden2 layer and hidden3 layer
    Returns:
        train_accs: accuracy of train n_epochs, a list
        train_losses: loss of n_epochs, a list
    """

    grad_2 = [] # save grad for hidden 2 every epoch
    grad_3 = [] # save grad for hidden 3 every epoch

    train_accs = [] # save train accuracy every epoch
    train_losses = [] # save train loss every epoch

    for epoch in range(n_epochs): # train for n_epochs
        # train model on training datasets, optimize loss function and update model parameters
        train_loss, average_grad2, average_grad3 = train(train_loader, model, loss_fn, optimizer, get_grad)

        # evaluate model performance on train dataset
        _, train_accuracy = evaluate(train_loader, model, loss_fn)
        message = 'Epoch: {}/{}. Train set: Average loss: {:.4f}, Accuracy: {:.4f}'.format(epoch+1, \
                                                                n_epochs, train_loss, train_accuracy)
        print(message)

        # save loss, accuracy, grad
        train_accs.append(train_accuracy)
        train_losses.append(train_loss)
        grad_2.append(average_grad2)
        grad_3.append(average_grad3)

        # evaluate model performance on val dataset
        val_loss, val_accuracy = evaluate(val_loader, model, loss_fn)
        message = 'Epoch: {}/{}. Validation set: Average loss: {:.4f}, Accuracy: {:.4f}'.format(epoch+1, \
                                                                n_epochs, val_loss,...


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
作业 2

运行一下fit函数，根据结束时候训练集的accuracy，回答：模型是否训练到过拟合。
使用提供的show_curve函数，画出训练的时候loss和accuracy的变化

Hints: 因为jupyter对变量有上下文关系，模型，优化器需要重新声明。可以使用以下代码进行重新定义模型和优化器。注意到此处用的是默认初始化。
Can you write Python code for it?

### Hyper parameters

batch_size = 128 # batch size is 128
n_epochs = 5 # train for 5 epochs
learning_rate = 0.01 # learning rate is 0.01
input_size = 28*28 # input image has size 28x28
hidden_size = 100 # hidden neurons is 100 for each layer
output_size = 10 # classes of prediction
l2_norm = 0 # not to use l2 penalty
dropout = False # not to use
get_grad = False # not to obtain grad

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)

train_accs, train_losses = fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
作业 3

将n_epochs设为10，观察模型是否能在训练集上达到过拟合， 使用show_curve作图。
当希望模型在5个epoch内在训练集上达到过拟合，可以通过适当调整learning rate来实现。选择一个合适的learing rate，训练模型，并且使用show_curve作图， 验证你的learning rate

Hints: 因为jupyter对变量有上下文关系，模型，优化器需要重新声明。可以使用以下代码进行重新定义模型和优化器。注意到此处用的是默认初始化。
Can you write Python code for it?

### Hyper parameters

batch_size = 128 # batch size is 128
n_epochs = 5 # train for 5 epochs
learning_rate = 0.01 # learning rate is 0.01
input_size = 28*28 # input image has size 28x28
hidden_size = 100 # hidden neurons is 100 for each layer
output_size = 10 # classes of prediction
l2_norm = 0 # not to use l2 penalty
dropout = False # not to use
get_grad = False # not to obtain grad

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)

# 3.1 Train
n_epochs = 10
train_accs, train_losses = fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
作业 5
思考正则项在loss中占比的影响。使用 l2_norm = 1, 训练模型
Hints: 因为jupyter对变量有上下文关系，模型，优化器需要重新声明。可以使用以下代码进行重新定义模型和优化器。注意到此处用的是默认初始化。
Can you write Python code for it?

# Hyper parameters
batch_size = 128
n_epochs = 5
learning_rate = 0.01
input_size = 28*28
hidden_size = 100
output_size = 10
l2_norm = 1 # use l2 penalty
get_grad = False

# declare a model
model = FeedForwardNeuralNetwork(input_size=input_size, hidden_size=hidden_size, output_size=output_size)
# Cross entropy
loss_fn = torch.nn.CrossEntropyLoss()
# l2_norm can be done in SGD
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=l2_norm)

train_accs, train_losses = fit(train_loader, test_loader, model, loss_fn, optimizer, n_epochs, get_grad)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 4 Training Issues 1/TrainingIssues.ipynb".
The first task is:
模型没有训练到过拟合，观察上面训练数据，随着代数增多，测试集的正确率并没有下降。
Can you write Python code for it?

show_curve(train_accs, 'accuracy')
show_curve(train_losses, 'loss')


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 5 CNN 1/Week5.ipynb".
The first task is:
GPU
我们可以选择在cpu或gpu上来训练我们的模型.
实验室提供了4卡的gpu服务器,要查看各个gpu设备的使用情况,可以在服务器上的jupyter主页点击new->terminal,在terminal中输入nvidia-smi即可查看每张卡的使用情况.如下图.

上图左边一栏显示了他们的设备id(0,1,2,3),风扇转速,温度,性能状态,能耗等信息,中间一栏显示他们的bus-id和显存使用量,右边一栏是GPU使用率等信息.注意到中间一栏的显存使用量,在训练模型前我们可以根据空余的显存来选择我们使用的gpu设备.
在本次实验中我们将代码中的torch.device('cuda:0')的0更换成所需的设备id即可选择在相应的gpu设备上运行程序.
CNN(卷积神经网络)
一个简单的CNN
接下来,让我们建立一个简单的CNN分类器.
这个CNN的整体流程是
卷积(Conv2d) -> BN(batch normalization) -> 激励函数(ReLU) -> 池化(MaxPooling) ->
卷积(Conv2d) -> BN(batch normalization) -> 激励函数(ReLU) -> 池化(MaxPooling) ->
全连接层(Linear) -> 输出.
Can you write Python code for it?

import torch
import torch.nn as nn
import torch.utils.data as Data
import torchvision

class MyCNN(nn.Module):

    def __init__(self, image_size, num_classes):
        super(MyCNN, self).__init__()
        # conv1: Conv2d -> BN -> ReLU -> MaxPool
        self.conv1 = nn.Sequential(
            nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3, stride=1, padding=1),
            nn.BatchNorm2d(16),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2),
        )
        # conv2: Conv2d -> BN -> ReLU -> MaxPool
        self.conv2 = nn.Sequential(
            nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3, stride=1, padding=1),
            nn.BatchNorm2d(32),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2),
        )
        # fully connected layer
        self.fc = nn.Linear(32 * (image_size // 4) * (image_size // 4), num_classes)

    def forward(self, x):
        """
        input: N * 3 * image_size * image_size
        output: N * num_classes
        """
        x = self.conv1(x)
        x = self.conv2(x)
        # view(x.size(0), -1): change tensor size from (N ,H , W) to (N, H*W)
        x = x.view(x.size(0), -1)
        output = self.fc(x)
        return output


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 5 CNN 1/Week5.ipynb".
The first task is:
ResNet
接下来,让我们完成更复杂的CNN的实现.
ResNet又叫做残差网络.在ResNet网络结构中会用到两种残差模块，一种是以两个3*3的卷积网络串接在一起作为一个残差模块，另外一种是1*1、3*3、1*1的3个卷积网络串接在一起作为一个残差模块。他们如下图所示。

我们以左边的模块为例实现一个ResidualBlock.注意到由于我们在两次卷积中可能会使输入的tensor的size与输出的tensor的size不相等,为了使它们能够相加,所以输出的tensor与输入的tensor size不同时,我们使用downsample(由外部传入)来使保持size相同
现在,试在To-Do补充代码完成下面的forward函数来完成ResidualBlock的实现,并运行它.
Can you write Python code for it?

# 3x3 convolution
def conv3x3(in_channels, out_channels, stride=1):
    return nn.Conv2d(in_channels, out_channels, kernel_size=3,
                     stride=stride, padding=1, bias=False)

# Residual block
class ResidualBlock(nn.Module):
    def __init__(self, in_channels, out_channels, stride=1, downsample=None):
        super(ResidualBlock, self).__init__()
        self.conv1 = conv3x3(in_channels, out_channels, stride)
        self.bn1 = nn.BatchNorm2d(out_channels)
        self.relu = nn.ReLU(inplace=True)
        self.conv2 = conv3x3(out_channels, out_channels)
        self.bn2 = nn.BatchNorm2d(out_channels)
        self.downsample = downsample

    def forward(self, x):
        """
        Defines the computation performed at every call.
        x: N * C * H * W
        """
        residual = x
        # if the size of input x changes, using downsample to change the size of residual
        if self.downsample:
            residual = self.downsample(x)
        out = self.conv1(x)
        out = self.bn1(out)

        out = self.relu(out)
        out = self.conv2(out)
        out = self.bn2(out)

        out += residual
        out = self.relu(out)
        return out


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 5 CNN 1/Week5.ipynb".
The first task is:
Vgg
接下来让我们阅读vgg网络的实现代码.VGGNet全部使用3*3的卷积核和2*2的池化核，通过不断加深网络结构来提升性能。Vgg表明了卷积神经网络的深度增加和小卷积核的使用对网络的最终分类识别效果有很大的作用.

下面是一份用于训练cifar10的简化版的vgg代码.
有时间的同学可以阅读并训练它.
Can you write Python code for it?

import math

class VGG(nn.Module):
    def __init__(self, cfg):
        super(VGG, self).__init__()
        self.features = self._make_layers(cfg)
        # linear layer
        self.classifier = nn.Linear(512, 10)

    def forward(self, x):
        out = self.features(x)
        out = out.view(out.size(0), -1)
        out = self.classifier(out)
        return out

    def _make_layers(self, cfg):
        """
        cfg: a list define layers this layer contains
            'M': MaxPool, number: Conv2d(out_channels=number) -> BN -> ReLU
        """
        layers = []
        in_channels = 3
        for x in cfg:
            if x == 'M':
                layers += [nn.MaxPool2d(kernel_size=2, stride=2)]
            else:
                layers += [nn.Conv2d(in_channels, x, kernel_size=3, padding=1),
                           nn.BatchNorm2d(x),
                           nn.ReLU(inplace=True)]
                in_channels = x
        layers += [nn.AvgPool2d(kernel_size=1, stride=1)]
        return nn.Sequential(*layers)

cfg = {
    'VGG11': [64, 'M', 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'],
    'VGG13': [64, 64, 'M', 128, 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'],
    'VGG16': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 'M', 512, 512, 512, 'M', 512, 512, 512, 'M'],
    'VGG19': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 256, 'M', 512, 512, 512, 512, 'M', 512, 512, 512, 512, 'M'],
}
vggnet = VGG(cfg['VGG11'])
print(vggnet)

# Hyper-parameters
num_epochs = 10
lr = 1e-3
# Device configuration
device = torch.device('cuda:0')

# optimizer
optimizer = torch.optim.Adam(vggnet.parameters(), lr=lr)

fit(vggnet, num_epochs, optimizer, device)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 5 CNN 1/Week5.ipynb".
The first task is:
下面在图像上简单一下测试我们的conv函数,这里使用3*3的高斯核对下面的图像进行滤波.
Can you write Python code for it?

from PIL import Image
import matplotlib.pyplot as plt
img = Image.open('pics/lena.jpg').convert('L')
plt.imshow(img, cmap='gray')

#  a Laplace kernel
laplace_kernel = np.array([[-1, -1, -1],
                           [-1, 8, -1],
                           [-1, -1, -1]])

# Gauss kernel with kernel_size=3
gauss_kernel3 = (1/ 16) * np.array([[1, 2, 1],
                                   [2, 4, 2],
                                   [1, 2, 1]])

# Gauss kernel with kernel_size=5
gauss_kernel5 = (1/ 84) * np.array([[1, 2, 3, 2, 1],
                                    [2, 5, 6, 5, 2],
                                    [3, 6, 8, 6, 3],
                                    [2, 5, 6, 5, 2],
                                    [1, 2, 3, 2, 1]])

fig, ax = plt.subplots(1, 3, figsize=(12, 8))

laplace_img = convolution(np.array(img), laplace_kernel, padding=1, stride=1)
ax[0].imshow(Image.fromarray(laplace_img), cmap='gray')
ax[0].set_title('laplace')

gauss3_img = convolution(np.array(img), gauss_kernel3, padding=1, stride=1)
ax[1].imshow(Image.fromarray(gauss3_img), cmap='gray')
ax[1].set_title('gauss kernel_size=3')

gauss5_img = convolution(np.array(img), gauss_kernel5, padding=2, stride=1)
ax[2].imshow(Image.fromarray(gauss5_img), cmap='gray')
ax[2].set_title('gauss kernel_size=5')


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 5 CNN 1/Week5.ipynb".
The first task is:
作业
尝试改变学习率lr,使用SGD或Adam优化器,训练10个epoch,提高ResNet在测试集上的accuracy.
Can you write Python code for it?

resnet = ResNet(ResidualBlock, [2, 2, 2])
num_epochs = 10
lr = 0.0009
device = torch.device('cuda:0')
optimizer = torch.optim.Adam(resnet.parameters(), lr=lr)
fit(resnet, num_epochs, optimizer, device)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 5 CNN 1/Week5.ipynb".
The first task is:
作业:
上述代码中convolution的实现只考虑卷积核以及padding和stride长宽一致的情况,若输入的卷积核可能长宽不一致,padding与stride的输入可能为两个元素的元祖(代表两个维度上的padding与stride)并使用下面test input对你的convolutionV2进行测试.
Can you write Python code for it?

def convolutionV2(img, kernel, padding=(0,0), stride=(1,1)):
    h, w = img.shape
    kh, kw = kernel.shape

    # height and width of image with padding
    ph, pw = h + 2 * padding[0], w + 2 * padding[1]
    padding_img = np.zeros((ph, pw))
    padding_img[padding[0]:h + padding[0], padding[1]:w + padding[1]] = img

    # height and width of output image
    result_h = (h + 2 * padding[0] - kh) // stride[0] + 1
    result_w = (w + 2 * padding[1] - kw) // stride[1] + 1

    result = np.zeros((result_h, result_w))

    # convolution
    x, y = 0, 0
    for i in range(0, ph - kh + 1, stride[0]):
        for j in range(0, pw - kw + 1, stride[1]):
            roi = padding_img[i:i+kh, j:j+kw]
            result[x, y] = np.sum(roi * kernel)
            y += 1
        y = 0
        x += 1
    return result

# test input
test_input = np.array([[1, 1, 2, 1],
                       [0, 1, 0, 2],
                       [2, 2, 0, 2],
                       [2, 2, 2, 1],
                       [2, 3, 2, 3]])

test_kernel = np.array([[1, 0], [0, 1], [0, 0]])

# output
print(convolutionV2(test_input, test_kernel, padding=(1, 0), stride=(1, 1)))

print(convolutionV2(test_input, test_kernel, padding=(2, 1), stride=(1, 2)))


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 5 CNN 1/Week5.ipynb".
The first task is:
使用fit函数训练实现的ResNet,观察结果变化.
Can you write Python code for it?

# Hyper-parameters
num_epochs = 10
lr = 0.001
# Device configuration
device = torch.device('cuda:0')
# optimizer
optimizer = torch.optim.Adam(resnet.parameters(), lr=lr)
fit(resnet, num_epochs, optimizer, device)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 5 CNN 1/Week5.ipynb".
The first task is:
卷积层
Pytorch提供了卷积层和池化层供我们使用.
卷积层与上面相似, 而池化层与卷积层相似,Pooling layer的主要目的是缩小features的size.常用的有MaxPool(滑动窗口取最大值)与AvgPool(滑动窗口取均值)
Can you write Python code for it?

import torch
import torch.nn as nn

x = torch.randn(1, 1, 32, 32)

conv_layer = nn.Conv2d(in_channels=1, out_channels=3, kernel_size=3, stride=1, padding=0)
y = conv_layer(x)
print(x.shape)
print(y.shape)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 5 CNN 1/Week5.ipynb".
The first task is:
接下来, 让我们测试我们写好的myconv2d函数.
Can you write Python code for it?


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
★★★ Question 7
请根据你的理解，完成一下反卷积函数，并在接下来的程序中，验证函数实现的正确性。
Can you write Python code for it?

#
def deconv2d( input, weights ):
    # input.shape is ( channel, height, weight )
    _, input_h, input_w = input.shape
    out_h = input_h*2
    out_w = input_w*2

    ############################################################
    # write your code here
    # fill zeros
    input = fill_zeros(input)
    # conv2d
    conv = conv2d(input, weights)
    # cut the padding
    c, _, k, _ = weights.shape
    p = int(k / 2)
    h, w = input_h * 2 + 1 - 2 * p, input_w * 2 + 1 - 2 * p
    result = np.zeros( [c, h, w] )
    for i in range(c):
        result[i] = conv[i, p: p + h, p: p + w]
    #############################################################

    return result


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
1.1 数据集的产生
从image_utils文件夹中引入数据生成器。我们不太需要理解数据生成的过程，只需要观察生成好的图片数据即可。观察和分析数据集通常是我们的第一步，通过观察数据集的特点，可以进一步决定我们训练数据所需要选择的模型和训练的方式。请依次执行一下部分的代码：
Can you write Python code for it?

# get the image generator
from image_utils import image_gen


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
2. 反卷积的实现
在深度学习上，细节的恢复本来是依靠简单的Upsample，为了使得细节恢复更加温和，就使用反卷积这种操作。这种操作具体来说指的是如下图所示的一个操作，即：先对输入的矩阵进行插空补零，然后在进行简单的普通的卷积操作，使得输出的矩阵比输入的矩阵更大。一句话来说就是，卷积的过程使得输出的Feature map比输入的Feature map在尺寸上更小，在特征表示上更加抽象；而反卷积的过程则是使得输出的Feature map比输入的Feature map在尺寸上更大，细节更加丰富，特征表示上更加具象，两者的参数均可训练。详细的介绍可参考这篇文章（点击这里）,这篇文章总结得很全。
而在反卷积的名字之争的话，则要另开一话了。感兴趣的可以点击这里阅读原文

逆卷积相对于卷积在神经网络结构的正向和反向传播中做相反的运算。逆卷积(Deconvolution)比较容易引起误会，转置卷积(Transposed Convolution)是一个更为合适的叫法.
Deconv本身其实有标准定义指conv的逆运算，一般通过Fourier Transform求解。深度学习里的deconv其实用错了，不是指逆运算，而是指transposed convolution, backward convolution, fractally strided convolution, upsampling convolution.

在实现上，反卷积操作思路如下：先对输入的矩阵填充0，然后进行卷积，卷积过程跟上述的加Padding的卷积过程一致，最后进行裁剪，使得输出的图像大小是输入的两倍。注意观察，填充0的操作会使得经过卷积之后，原来input的数值在output中不发生改变（红色数字）。如下图所示：

根据上述思路，反卷积的实现过程主要分为一下两个函数，一个是补零，一个是反卷积，实现起来非常简单。
Can you write Python code for it?

def fill_zeros(input):
    # c : channel
    # h : height
    # w : weight
    c, h, w  = input.shape
    result   = np.zeros( [c, h*2+1, w*2+1] )

    for i in range(c):
        for j in range(h):
            for k in range(w):
                result[ i, 2*j+1, 2*k+1 ] = input[i, j, k]
    return result


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
2.4 模型的评估方式
常用的分割效果评估方式有两种，一种是计算Jaccard值，一种是计算Dice值。两者比较类似，单调性相同，可以相互转换，但是数值性质有细微的不一样。下面会对此进行简单的回顾和说明。
值得注意的是（细心的你可能自己已经注意到了），我们在训练神经网络的时候，使用了Binary Cross Entropy，而在评估模型好坏的时候，却不是使用每一个像素点的分类的准确度百分比（之前你们使用的Cross Entropy训练分类网络的时候，评估模型的结果是时候，使用的是每一类的分类的准确度，而这里，对应的做法应该是计算每一个像素点的分类的准确度），这是为什么呢？ 这种损失函数与评估函数之间的差别，会给我们的训练带来什么不好的影响呢？
2.4.1 Jaccard值的理解
现在你只需要意识到这样的问题存在，然后我们不妨先来回顾下Jaccard值。

雅卡尔指数（英语：Jaccard index），又称为并交比（Intersection over Union）、雅卡尔相似系数（Jaccard similarity coefficient），是用于比较样本集的相似性与多样性的统计量。雅卡尔系数能够量度有限样本集合的相似度，其定义为两个集合交集大小与并集大小之间的比例：
$$J(A,B) = {{|A \cap B|}\over{|A \cup B|}} = {{|A \cap B|}\over{|A| + |B| - |A \cap B|}}.$$
如果A与B完全重合，则定义J(A,B) = 1。于是有:
$$0\le J(A,B)\le 1.$$

直观理解如下两图所示：

Jaccard值等于上子图的紫色部分和下子图的紫色部分的比值。在离散的像素图像中表现为：上下两图的紫色部分的像素点个数的比值（这也是我们在计算的时候使用的方法）。
2.4.2 Dice值的理解
有了上面Jaccard值的铺垫，Dice值就非常容易理解了:

Dice系数, 根据 Lee Raymond Dice 命名，是一种集合相似度度量函数，通常用于计算两个样本的相似度：
$$s = \frac{2 | X \cap Y |}{| X | + | Y |}$$
它和Sørensen 相似度指数相同, 也称作Sørensen-Dice系数。 它在形式上和Jaccard指数没多大区别，但是有些不同的性质。
和Jaccard类似，它的范围为0到1。

★★★ Question 3：
请结合对Jaccard值的理解，根据Jaccard值的计算公式，补充好./unet/dice_loss.py中JaccardCoeff类的内容，使其能够正确计算出预测结果的Jaccard值。然后执行以下代码进行测试和比较，观察你的输出结果是否正确。
【tips】参考和借鉴同文件中DiceCoeff类的实现方法
【tips】只需要按指示完成forward函数即可，不需要写backward函数
Can you write Python code for it?

#from unet import dice_coeff as dice
from unet import JaccardCoeff as jaccard

# get three 2D matrix as inputs
eye   = torch.tensor(np.eye(3)       )
ones  = torch.tensor(np.ones([3, 3]) )
zeros = torch.tensor(np.zeros([3,3]) )

# show the input
print( eye  )
print( ones )
print( zeros)

# get your result
print( jaccard().forward(eye,   eye) )
print( jaccard().forward(eye,   ones) )
print( jaccard().forward(zeros, ones) )
print( jaccard().forward(zeros, zeros))


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
Note 1
由于分割网络的代码参数量比较大，为了尽量降低实验过程对电脑设备和GPU显存大小的要求，在本次本次实验过程中，我们并没有使用语义分割最常用的数据集如VOC， CoCo等多分类语义分割数据集。而是选择了我们人工构造的几何形状作为数据集。这个数据集我们可以通过手动调节参数的方式，任意指定训练输入图片的大小，可以轻易得手动降低实验数据的数据量和对电脑内存（显存）资源的要求。这样子的__好处__在于在没有GPU资源的个人PC中，我们也可以训练好整个网络，让大家对语义分割模型有直观地感受。__不好的地方__在于这个数据集非常简单，很多的Tricks和模型的改进对性能的提升在这里可能体现得不明显。
1.2 观察数据集
我们在训练数据之前必须要对其进行充分的观察和分析，了解这个数据集的具体内容和相应的特点，清楚其难点在哪里，哪些部分的数据很具有挑战性，这些都可以增强我们对这个任务的直观的、感性的认识。
所以，现在我们现在可以来探索一下generator在做了些什么，为我们提供了怎么样的数据。查看生成数据的维度是一个非常直观且重要的观察方式，在我们自己编写模型的时候，模型input的维度要和generator的，请执行以下代码：
Can you write Python code for it?

# 2 mean how many image you want to get
x_test, y_test = generator(2)
print(x_test.shape)
print(y_test.shape)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
上图中左上角的子图是输入到UNet中的图片；右上角的子图是输入图片的标签，分割的（Ground Truth）；左下角的子图是每个像素点的概率图，颜色越亮，概率越大，这个子图的图片也就是上面的框框第19行的输出值；右下角的子图是网络的预测值，也即是将左下角的子图以0.5为临界值，上下取整后输出的图像。

★★★ Question 5
反复执行上述代码，找出预测结果不好的图片，并且分析为什么该图片的预测结果会比较糟糕？
Answer 5：
上面代码执行所得到的图片即为预测结果不好的图片，可以观察到右上图和右下图有很明显的区别。观察左上图可以发现，输入到UNet中的原图的右下角数据一点也不明显，而右上图的标签却将它们视为与明显的数据是同一类。由于原图的数据不明显，导致最终的预测结果较差，与实际标签相差较大。

3. 修改网络模型
简单修改网络结构，如修改网络的深度，使得网络更加深或者更加浅，体会网络的深度对分割模型的影响；又比如将__upsample过程中的浅层特征的concate修改成add__，体会不同操作对实验结果的影响。
3.1 降低网络的层数
网络的深度对网络的性能产生着至关重要的影响。一般来说，网络越深，描述能力越强，性能越高。但是，也会带来梯度消失、梯度爆炸等一系列的障碍，使得网络训练更加困难。在本次实验中，是要求大家练习降低网络的层数，目的是让大家更加熟悉UNet的结构。
原始的Unet结构是共有5层（根据上述提供的模型结构从上往下数），downsample操作执行了4次，对应部分的代码位于./unet/unet_model.py文件中。现根据已提供的代码，按照以下操作提示，将网络模型层数，使得Unet模型更加浅层，并且对该模型进行训练，观察实验结果，

★★★ Question 6
1、在__./unet/unet_model.py文件中创建一个新的类，命名为UNet_shallow,该类只downsample__两次，共三层，请参考Unet类的写法
2、将写好的模型重新进行训练，并观察训练好的实验结果。

以下部分代码与前面部分的代码高度类似，可从上面的函数中直接复制粘贴下执行，获取新的模型的结果。注意，上面几个部分的代码比较松散，中间穿插的内容比较多。在这里的话，但实际上训练模型的代码集中起来就以下几步，可以在这里集中理解一下下。
Can you write Python code for it?

# the only difference of the three cells with above is the model
from unet import UNet_shallow

unet_shallow = UNet_shallow(n_channels=1, n_classes=2)
criterion = nn.BCELoss()
optimizer = optim.SGD( unet_shallow.parameters(),
                       lr           = 0.001,
                       momentum     = 0.9,
                       weight_decay = 0.0005
                     )

if is_cuda:
    unet_shallow = unet_shallow.cuda()

epochs     = 5         # usual value is 50, 100, 150
steps      = 50        # usual value is ？ No usual value, just go through the whole training set
batch_size = 4         # usual value is 16， 32， 64, usually as big as possible
valid_size = 50        # No usual value, just 20% of the whole dataset

epoch_loss = 0

# use to plot the loss during training
curr_step  = np.zeros( epochs*steps )
step_loss  = np.zeros( epochs*steps )
counter    = 0
for epoch in range(epochs):
    for step in range(steps):
        batch_x, batch_y = generator(batch_size)

        # Remember the different between B*H*W*C and B*C*H*W
        batch_x = np.transpose( batch_x, axes=[0, 3, 1, 2] ).astype(np.float32)
        batch_y = np.transpose( batch_y, axes=[0, 3, 1, 2] ).astype(np.float32)

        batch_x = torch.from_numpy( batch_x )
        batch_y = torch.from_numpy( batch_y )

        if is_cuda:
            batch_x = batch_x.cuda()
            batch_y = batch_y.cuda()

        masks_pred       = unet_shallow(batch_x)
        masks_probs      = F.sigmoid(masks_pred)
        masks_probs_flat = masks_probs.reshape(-1)

        batch_y_flat     = batch_y.reshape(-1)
        loss             = criterion( masks_probs_flat, batch_y_flat)

        curr_step[ counter ] = epoch*steps + step
        step_loss[ counter ] = loss.item()
        counter += 1

        if (step+1) % 10==0:
            print("epoch {0:} step {1:} loss is {2:0.4f}".format(epoch+1, step+1, loss.item()) )

        optimizer.zero_grad()
       ...


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
以下代码作为验证程序正确性使用，不作理解要求。其实现思路同上，即将Pytorch中的一个的反卷积操作当做一个迷你的网络来使用即可。
请使用一下Pytorch反卷积代码验证你的结果。
Can you write Python code for it?

input         = torch.tensor(input_data).unsqueeze(0).float()

torch_deconv2d  = torch.nn.ConvTranspose2d( in_channels=4, \
            out_channels=2, kernel_size=3, padding = 1, stride=2)

state = torch_deconv2d.state_dict()
weights = np.rot90(np.array(weights_data), 2, axes=(2,3)).copy()

#weights = np.array(weights_data)
state["weight"] = torch.from_numpy(weights).float().transpose(1, 0)
state["bias"]   = torch.zeros(torch.Size([2]))
torch_deconv2d.load_state_dict(state)
print(torch_deconv2d(input))


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
其结果如下图所示：
Python
tensor([[ 1.,  0.,  0.],
        [ 0.,  1.,  0.],
        [ 0.,  0.,  1.]], dtype=torch.float64)
tensor([[ 1.,  1.,  1.],
        [ 1.,  1.,  1.],
        [ 1.,  1.,  1.]], dtype=torch.float64)
tensor([[ 0.,  0.,  0.],
        [ 0.,  0.,  0.],
        [ 0.,  0.,  0.]], dtype=torch.float64)
tensor(1.0000)
tensor(0.3333)
tensor(1.00000e-05 *
       1.1111)
tensor(1.)
【Tips】回顾Dice值的计算公式，考虑两张图像中每一个像素点均为0的情况，此时分母为0，计算没有意义。因此，我们需要在计算的时候给分子和分母加上平滑项eps，使得计算具有意义，eps取极小值，避免对非零值计算的时候产生太大的误差。

2.4.3 拓展理解——为什么要使用Jaccard和Dice来评估模型
也就是回答上面提到的问题，为什么我们在训练神经网络的时候，使用了Binary Cross Entropy，而在评估模型好坏的时候使用Jaccard和Dice值。这个问题的理解我们
在此引用了国外的一篇优秀的博文进行理解，感兴趣的同学可以点击这里阅读原文。
【tips】本问题仅作思考，不作任何要求，不加分，欢迎交流.
Answer：
对于二分类问题获取其它分类问题的模型输出可直接通过对比标签来计算正确率，来评估模型的好坏。而对于物体检测的问题，通常情况是需要得到预测边框或者其它结果，这类模型输出不能简单的对比标签计算正确率。通过Jaccard和Dice值可以很好地评估模型的好坏，因为这两个值涉及到了预测结果和标签的交集和并集，可以通过判断两者是否很好的重叠来计算正确率。
2.5 观测和分析实验结果

★★★ Question 4
在分割模型中，如何通过卷积神经网络的结果得到预测结果？请阅读和执行以下代码后进行简要的分析。
【Tips】网络输出的是一个很大数值，然后使用sigmod函数可以网络输出归一化为0到1之间的数值，使得各个像素点的概率可比较。语义分割网络输出的channel数为前景数目加上背景数目，一般第0个channel输出的是每个像素点对应被判别为背景的概率，第1个channel输出的每个像素点对应的判别第一类物体的概率。VOC数据集需要对20类物体进行分割，加上背景类，网络输出的channel数为21，本次实验中仅对1类物体进行分割，网络输出的channel数为2.
【Tips】可以先阅读和执行下面的代码再回来回到这个问题
Answer 4：
卷积神经网络的输出结果是一个较大的数值，通过激励函数的归一化处理可以将该大数值变成0至1的数值，也就是视为该像素点的概率。对于网络输出的多个通道的结果，我们将其数据归一化之后，可将通道中概率大的像素点，视为该通道所代表的物体类型。
比如下面的代码中，将概率大于等于0.5的像素点视为就是该类物体，而该类小于0.5的像素点视为不是该类物体。这样，就可以将神经网络的输出结果转换成预测结果，从而得到每一类物体对应的像素点。

根据自己的思考，理解以下代码，并且运行该代码，观察实验结果。
Can you write Python code for it?

def predict(network, image, grounth_truth ):

    # By default all the modules are initialized to
    # train mode (self.training = True). Also be aware
    # that some layers have different behavior during training
    # and evaluation (like BatchNorm, Dropout)
    # so we should change the mode
    network.eval()

    # be aware of the B×H×W×C and the B×C×H×W
    image = np.transpose( image, axes=[0, 3, 1, 2] ).astype(np.float32)

    # prepare the input datatype
    image_input = torch.from_numpy(image)
    image_input = image_input.cuda() if is_cuda else image_input

    # get the network output, and then regular each pixel between 0 and 1
    masks_pred = network(   image_input )
    masks_prob = F.sigmoid( masks_pred  )

    # get numpy object result from torch.tensor
    if is_cuda:
        masks_prob = masks_prob.cpu().data.numpy()
    else:
        masks_prob = masks_prob.data.numpy()

    masks_binary = masks_prob.copy()

    # if the prossibility is greater than 0.5, we mask it as 1
    # if the prossibility is less then 0.5, we mask it as 0
    masks_binary[ masks_binary>=0.5 ] = 1
    masks_binary[ masks_binary< 0.5 ] = 0
    # predication is finished！

    # show me the result
    fig, axs = plt.subplots(2,2, sharey=True, figsize=(10,8))
    axs[0][0].set_title("Original data show in color")
    axs[0][1].set_title("Ground Truth")
    axs[1][0].set_title("Probility heatmap")
    axs[1][1].set_title("Predition")

    ax00 = axs[0][0].imshow( image[0,0,...],        aspect="auto" )
    ax01 = axs[0][1].imshow( ground_truth[0,...,1], aspect="auto" )
    ax10 = axs[1][0].imshow( masks_prob[0,1,...],   aspect="auto", cmap="hot" )
    ax11 = axs[1][1].imshow( masks_binary[0,1,...], aspect="auto" )

    fig.colorbar(ax00, ax=axs[0][0])
    fig.colorbar(ax01, ax=axs[0][1])
    fig.colorbar(ax10, ax=axs[1][0])
    fig.colorbar(ax11, ax=axs[1][1])


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
如果我们是在使用GPU的话，我们还可以通过以下代码来查看我们当前使用的设备, 当然了，没有GPU的时候执行以下代码就会报错，可以不必理会。
Can you write Python code for it?

print( torch.cuda.current_device()  ) # which GPU you're using
print( torch.cuda.device_count()    ) # how many GPU you have
print( torch.cuda.get_device_name(0)) # device name


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
定义新的模型、损失函数和优化函数
Can you write Python code for it?

# define model criterion and optimizer
unet_add = UNet_add(n_channels=1, n_classes=2)
criterion = nn.BCELoss()
optimizer = optim.SGD( unet_add.parameters(),
                       lr           = 0.001,
                       momentum     = 0.9,
                       weight_decay = 0.0005
                     )

if is_cuda:
    unet_add = unet_add.cuda()


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
引入必要的通用的Python包，设置Jupyter的交互环境，为接下来的实验做准备，在执行新的代码之前。请运行下面代码（表面上无事发生，内心已有所不同）
Can you write Python code for it?

# import necessary package for the following exercises
from __future__ import division, print_function
#%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib
import numpy as np


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
查看训练结果
Can you write Python code for it?

# visiual UNet_add's result
input, ground_truth = generator(1)
predict(unet_add, input, ground_truth)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
由于Pytorch版本更替的原因，在比较新的Pytorch版本环境中运营该代码可能会产生一些弃用某些函数的warning，在此我们不必要关注这部分的信息，可以将其屏蔽掉：
Can you write Python code for it?


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
自行手动更改当前使用的GPU的命令如下所示（学会这项技能学会自己指定空闲的GPU进行使用）：
Can you write Python code for it?

id = 2
torch.cuda.set_device(id)
print( torch.cuda.current_device() )


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
言归正传，我们需要新建一个网络的对象，定义loss函数和优化器，这里使用的是二分类的binary_cross_entropy，这与前面分类网络部分代码没有区别。请执行以下代码：
Can you write Python code for it?

# Initialize a basic UNet
unet = UNet(n_channels=1, n_classes=2)
# Define a binary Corss entropy loss, the most common loss
criterion = nn.BCELoss()
# Define a optimizer, specific way to update the parameters in your network
optimizer = optim.SGD( unet. parameters(),
                       lr           = 0.001,
                       momentum     = 0.9,
                       weight_decay = 0.0005
                     )

# whether you PC has a GPU? If yes, use it
if is_cuda:
    unet = unet.cuda()


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 7 Segmentation/Assignment.ipynb".
The first task is:
设置的输入函数如下所示：
Can you write Python code for it?


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
WGAN-GP(improved wgan)
在WGAN中，需要进行截断, 在实验中发现： 对于比较深的WAGN，它不容易收敛。
大致原因如下：
1. 实验发现最后大多数的权重都在-c 和c上,这就意味了大部分权重只有两个可能数,这太简单了,作为一个深度神经网络来说,这实在是对它强大的拟合能力的浪费.
2. 实验发现容易导致梯度消失或梯度爆炸。判别器是一个多层网络，如果把clip的值设得稍微小了一点,每经过一层网络,梯度就变小一点点,多层之后就会指数衰减；反之,则容易导致梯度爆炸.
所以WGAN-GP使用了Gradient penalty(梯度惩罚）来代替clip.
因为Lipschitz限制是要求判别器的梯度不超过K,所以可以直接使用一个loss term来实现这一点,所以改进后D的优化目标改进为如下:

下面是WGAN-GP的具体代码实现,同WGAN,我们也只实现了他的训练代码,而模型我们直接使用DCGAN的模型.
Can you write Python code for it?

import torch.autograd as autograd

def wgan_gp_train(trainloader, G, D, G_optimizer, D_optimizer, device, z_dim, lambda_=10, n_d=2):

    D.train()
    G.train()

    D_total_loss = 0
    G_total_loss = 0

    for i, (x, _) in enumerate(trainloader):
        x = x.to(device)

        # update D network
        # D optimizer zero grads
        D_optimizer.zero_grad()

        # D real loss from real images
        d_real = D(x)
        d_real_loss = - d_real.mean()

        # D fake loss from fake images generated by G
        z = torch.rand(x.size(0), z_dim).to(device)
        g_z = G(z)
        d_fake = D(g_z)
        d_fake_loss = d_fake.mean()

        # D gradient penalty

        #   a random number epsilon
        epsilon = torch.rand(x.size(0), 1, 1, 1).cuda()
        x_hat = epsilon * x + (1 - epsilon) * g_z
        x_hat.requires_grad_(True)

        y_hat = D(x_hat)
        #   computes the sum of gradients of y_hat with regard to x_hat
        gradients = autograd.grad(outputs=y_hat, inputs=x_hat, grad_outputs=torch.ones(y_hat.size()).cuda(),
                                  create_graph=True, retain_graph=True, only_inputs=True)[0]
        #   computes gradientpenalty
        gradient_penalty =  torch.mean((gradients.view(gradients.size()[0], -1).norm(p=2, dim=1) - 1) ** 2)

        # D backward and step
        d_loss = d_real_loss + d_fake_loss + lambda_ * gradient_penalty
        d_loss.backward()
        D_optimizer.step()

        D_total_loss += d_loss.item()

        # update G network
        # G optimizer zero grads
        if (i + 1) % n_d == 0:
            G_optimizer.zero_grad()

            # G loss
            g_z = G(z)
            d_fake = D(g_z)
            g_loss = - d_fake.mean()

            # G backward and step
            g_loss.backward()
            G_optimizer.step()

            G_total_loss += g_loss.item()

    return D_total_loss / len(trainloader), G_total_loss * n_d /...


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
万事具备,接下来让我们来尝试这训练一个基本的GAN网络吧.这里实现run_gan函数来调用train以及visualize_results来训练我们的GAN.
Can you write Python code for it?

def run_gan(trainloader, G, D, G_optimizer, D_optimizer, loss_func, n_epochs, device, latent_dim):
    d_loss_hist = []
    g_loss_hist = []

    for epoch in range(n_epochs):
        d_loss, g_loss = train(trainloader, G, D, G_optimizer, D_optimizer, loss_func, device,
                               z_dim=latent_dim)
        print('Epoch {}: Train D loss: {:.4f}, G loss: {:.4f}'.format(epoch, d_loss, g_loss))

        d_loss_hist.append(d_loss)
        g_loss_hist.append(g_loss)

        if epoch == 0 or (epoch + 1) % 10 == 0:
            visualize_results(G, device, latent_dim)

    return d_loss_hist, g_loss_hist


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
可以看到,DCGAN的生成图片质量比起只有线性层的GAN要好不少.接下来,让我们尝试使用家具数据集来训练DCGAN.
Can you write Python code for it?

# RGB image channel = 3
image_channel=3

# epochs
n_epochs = 300
batch_size = 32
image_size=32
latent_dim = 100

device = torch.device('cuda:2')

learning_rate = 0.0002
betas = (0.5, 0.999)

bceloss = nn.BCELoss().to(device)

# mnist dataset and dataloader
train_dataset = load_furniture_data()
trainloader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

# G and D model, use DCGAN
G = DCGenerator(image_size=image_size, latent_dim=latent_dim, output_channel=image_channel).to(device)
D = DCDiscriminator(image_size=image_size, input_channel=image_channel).to(device)

# G and D optimizer, use Adam or SGD
G_optimizer = optim.Adam(G.parameters(), lr=learning_rate, betas=betas)
D_optimizer = optim.Adam(D.parameters(), lr=learning_rate, betas=betas)

d_loss_hist, g_loss_hist = run_gan(trainloader, G, D, G_optimizer, D_optimizer, bceloss,
                                   n_epochs, device, latent_dim)

loss_plot(d_loss_hist, g_loss_hist)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
可以看到,参数都被截断在[-c, c]之间,大部分参数集中在-c和c附近.
作业:
尝试使用n_d设置为5, 3, 1等,再次训练wGAN,n_d为多少时的结果最好?
答：
n_d为1时结果最好，虽然说n_d代表的是每训练一次G就训练多少次D，但在网络中是先训练D的，也就是每n_d批数据，才训练一次G。n_d为1时，G的训练次数是最多的。
Can you write Python code for it?

n_d = 1

# G and D model, use DCGAN, note that sigmoid is removed in D
G = DCGenerator(image_size=image_size, latent_dim=latent_dim, output_channel=image_channel).to(device)
D = DCDiscriminator(image_size=image_size, input_channel=image_channel, sigmoid=False).to(device)

# G and D optimizer, use Adam or SGD
G_optimizer = optim.Adam(G.parameters(), lr=learning_rate, betas=betas)
D_optimizer = optim.Adam(D.parameters(), lr=learning_rate, betas=betas)

d_loss_hist, g_loss_hist = run_wgan(trainloader, G, D, G_optimizer, D_optimizer, n_epochs, device,
                                    latent_dim, n_d, weight_clip)

loss_plot(d_loss_hist, g_loss_hist)

n_d = 3

# G and D model, use DCGAN, note that sigmoid is removed in D
G = DCGenerator(image_size=image_size, latent_dim=latent_dim, output_channel=image_channel).to(device)
D = DCDiscriminator(image_size=image_size, input_channel=image_channel, sigmoid=False).to(device)

# G and D optimizer, use Adam or SGD
G_optimizer = optim.Adam(G.parameters(), lr=learning_rate, betas=betas)
D_optimizer = optim.Adam(D.parameters(), lr=learning_rate, betas=betas)

d_loss_hist, g_loss_hist = run_wgan(trainloader, G, D, G_optimizer, D_optimizer, n_epochs, device,
                                    latent_dim, n_d, weight_clip)

loss_plot(d_loss_hist, g_loss_hist)

n_d = 5

# G and D model, use DCGAN, note that sigmoid is removed in D
G = DCGenerator(image_size=image_size, latent_dim=latent_dim, output_channel=image_channel).to(device)
D = DCDiscriminator(image_size=image_size, input_channel=image_channel, sigmoid=False).to(device)

# G and D optimizer, use Adam or SGD
G_optimizer = optim.Adam(G.parameters(), lr=learning_rate, betas=betas)
D_optimizer = optim.Adam(D.parameters(), lr=learning_rate, betas=betas)

d_loss_hist, g_loss_hist = run_wgan(trainloader, G, D, G_optimizer, D_optimizer, n_epochs, device,
                                    latent_dim, n_d, weight_clip)

loss_plot(d_loss_hist, g_loss_hist)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
同样的,我们使用同样的mnist数据集对DCGAN进行训练.
Can you write Python code for it?

# hyper params

# z dim
latent_dim = 100

# image size and channel
image_size=32
image_channel=1

# Adam lr and betas
learning_rate = 0.0002
betas = (0.5, 0.999)

# epochs and batch size
n_epochs = 100
batch_size = 32

# device : cpu or cuda:0/1/2/3
device = torch.device('cuda:2')

# mnist dataset and dataloader
train_dataset = load_mnist_data()
trainloader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

# use BCELoss as loss function
bceloss = nn.BCELoss().to(device)

# G and D model, use DCGAN
G = DCGenerator(image_size=image_size, latent_dim=latent_dim, output_channel=image_channel).to(device)
D = DCDiscriminator(image_size=image_size, input_channel=image_channel).to(device)

# G and D optimizer, use Adam or SGD
G_optimizer = optim.Adam(G.parameters(), lr=learning_rate, betas=betas)
D_optimizer = optim.Adam(D.parameters(), lr=learning_rate, betas=betas)

d_loss_hist, g_loss_hist = run_gan(trainloader, G, D, G_optimizer, D_optimizer, bceloss,
                                   n_epochs, device, latent_dim)

loss_plot(d_loss_hist, g_loss_hist)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
同理,观察loss曲线和D上的参数分布.
Can you write Python code for it?

loss_plot(d_loss_hist, g_loss_hist)

show_d_params(D)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
完成上方代码后,使用所写的L2Loss在mnist数据集上训练DCGAN.
Can you write Python code for it?

# hyper params

# z dim
latent_dim = 100

# image size and channel
image_size=32
image_channel=1

# Adam lr and betas
learning_rate = 0.0002
betas = (0.5, 0.999)

# epochs and batch size
n_epochs = 100
batch_size = 32

# device : cpu or cuda:0/1/2/3
device = torch.device('cuda:2')

# mnist dataset and dataloader
train_dataset = load_mnist_data()
trainloader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

# use L2Loss as loss function
l2loss = L2Loss().to(device)

# G and D model, use DCGAN
G = DCGenerator(image_size=image_size, latent_dim=latent_dim, output_channel=image_channel).to(device)
D = DCDiscriminator(image_size=image_size, input_channel=image_channel).to(device)

# G and D optimizer, use Adam or SGD
G_optimizer = optim.Adam(G.parameters(), lr=learning_rate, betas=betas)
D_optimizer = optim.Adam(D.parameters(), lr=learning_rate, betas=betas)

d_loss_hist, g_loss_hist = run_gan(trainloader, G, D, G_optimizer, D_optimizer, l2loss, n_epochs, device,
                                   latent_dim)
loss_plot(d_loss_hist, g_loss_hist)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
当模型训练后,我们需要查看此时G生成的图片效果,下面的visualize_results代码便实现了这块内容.注意,我们生成的图片都在[-1,1],因此,我们需要将图片反向归一化(denorm)到[0,1].
Can you write Python code for it?

def visualize_results(G, device, z_dim, result_size=20):
    G.eval()

    z = torch.rand(result_size, z_dim).to(device)
    g_z = G(z)

    show(torchvision.utils.make_grid(denorm(g_z.detach().cpu()), nrow=5))


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
接下来运行下面两个cell的代码,集中展示wgan的参数分布.
Can you write Python code for it?

from utils import show_weights_hist
def show_d_params(D):
    plist = []
    for params in D.parameters():
        plist.extend(params.cpu().data.view(-1).numpy())
    show_weights_hist(plist)

show_d_params(D)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
数据集
在训练我们的GAN网络之前, 先介绍一下本次实验可训练GAN的数据集,我们提供了两个数据集来供大家进行尝试数据集.
- MNIST手写体3类数据集,这里为了加快我们的训练速度,我们提供了一个简化版本的只包含数字0,2的2类MNIST数据集,每类各1000张.图片为28*28的单通道灰度图(我们将其resize到32*32),对于GAN而言,我们不需要测试集.我们本次实验主要使用该数据集作为主要的训练数据集.

室内家具数据集.为了加快我们的训练速度,我们将其做了删减处理,仅包含chair等一个类,共500张.图片为32*32的3通道彩色图片.

下面是两个加载数据集的函数.注意我们将所有图片normalize到了[-1,1]之间.
Can you write Python code for it?

def load_mnist_data():
    """
    load mnist(0,1,2) dataset
    """

    transform = torchvision.transforms.Compose([
        # transform to 1-channel gray image since we reading image in RGB mode
        transforms.Grayscale(1),
        # resize image from 28 * 28 to 32 * 32
        transforms.Resize(32),
        transforms.ToTensor(),
        # normalize with mean=0.5 std=0.5
        transforms.Normalize(mean=(0.5, ),
                             std=(0.5, ))
        ])

    train_dataset = torchvision.datasets.ImageFolder(root='./data/mnist', transform=transform)

    return train_dataset

def load_furniture_data():
    """
    load furniture dataset
    """
    transform = torchvision.transforms.Compose([
        transforms.ToTensor(),
        # normalize with mean=0.5 std=0.5
        transforms.Normalize(mean=(0.5, 0.5, 0.5),
                             std=(0.5, 0.5, 0.5))
        ])
    train_dataset = torchvision.datasets.ImageFolder(root='./data/household_furniture', transform=transform)
    return train_dataset


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
由WGAN的原理我们知道,D_loss的相反数可以表示生成数据分布与真实分布的Wasserstein距离,其数值越小,表明两个分布越相似,GAN训练得越好.它的值给我们训练GAN提供了一个指标.
运行下方代码观察wgan的loss曲线,可以看到,总体上,D_loss的相反数随着epoch数增加逐渐下降,同时生成的数据也越来越逼近真实数据,这与wgan的原理是相符合的.
Can you write Python code for it?


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
训练完后,让我们来看一下G生成的图片效果,可以看到即使是一个简单的GAN在这种简单的数据集上的生成效果还是不错的,虽然仍然存在不少瑕疵,比如说我们可以看到生成的图片上的数字有很多奇怪的雪花等等.
让我们看一下G和D的loss变化曲线(运行下方语句.)
Can you write Python code for it?

from utils import loss_plot

loss_plot(d_loss_hist, g_loss_hist)


I am working on the file "Homework/Principles of Artificial Neural Networks/Week 9 GAN 1/GAN-1.ipynb".
The first task is:
设置好超参数就可以开始训练!让我们尝试用它来训练2类的mnist数据集
Can you write Python code for it?

# hyper params

# z dim
latent_dim = 100

# image size and channel
image_size=32
image_channel=1

# Adam lr and betas
learning_rate = 0.0002
betas = (0.5, 0.999)

# epochs and batch size
n_epochs = 100
batch_size = 32

# device : cpu or cuda:0/1/2/3
device = torch.device('cuda:2')

# mnist dataset and dataloader
train_dataset = load_mnist_data()
trainloader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

# use BCELoss as loss function
bceloss = nn.BCELoss().to(device)

# G and D model
G = Generator(image_size=image_size, latent_dim=latent_dim, output_channel=image_channel).to(device)
D = Discriminator(image_size=image_size, input_channel=image_channel).to(device)

# G and D optimizer, use Adam or SGD
G_optimizer = optim.Adam(G.parameters(), lr=learning_rate, betas=betas)
D_optimizer = optim.Adam(D.parameters(), lr=learning_rate, betas=betas)

d_loss_hist, g_loss_hist = run_gan(trainloader, G, D, G_optimizer, D_optimizer, bceloss,
                                   n_epochs, device, latent_dim)


I am working on the file "Homework01/01 - First approach.ipynb".
The first task is:
2) Use the funtion to load the data on memory
Can you write Python code for it?

input_data = read_dir(path,'.in')
ans_data = read_dir(path,'.ans')


I am working on the file "Homework01/01 - First approach.ipynb".
The first task is:
3) Create a set of functions that implements a model to treat our data.
Can you write Python code for it?

def price(l):
    p = int(l[0])
    a = int(l[1])
    b = int(l[2])
    c = int(l[3])
    d = int(l[4])
    n = int(l[5])
    P = []
    for k in range(int(n)):
        P.append(p*(np.sin(a*(k+1)+b)+np.cos(c*(k+1)+d)+2))
    return P

def max_decline(prices):
    max_dif = 0.0
    dif = max(prices)-min(prices)
    if dif >  0.0:
        max_dif = dif
    return max_dif

def max_dec_list(data):
    declines = []
    for d in data:
        key = d.keys()[0]
        value = d.values()[0]
        stock_prices = price(value)
        declines.append({key:[max_decline(stock_prices)]})
    return declines
comp_data = max_dec_list(input_data)


I am working on the file "Homework01/01 - First approach.ipynb".
The first task is:
4) Compute the error between the computed data and the teorical.
Can you write Python code for it?

for l1 in ans_data:
    for l2 in comp_data:
        if l1.keys()[0][0:2] == l2.keys()[0][0:2]:
            print l1.keys()[0], l2.keys()[0]
            print 'error in file', l1.keys()[0][0:2], 'is ', '%.6f' % abs(float(l1.get(l1.keys()[0])[0])- float(l2.get(l2.keys()[0])[0]))


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
4. Here, we plot the proportion of people that survived in the first class.
Can you write Python code for it?

titanic[titanic.pclass == "First Class"].survived.value_counts(sort=False).plot(kind='pie')


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
As we can see, the lower the class, the higher the probability of death.
5. Here we add new columns that will help us to calculate proportions of survived people in the last part.
Can you write Python code for it?

titanic.insert(0, 'alive', 0)
titanic.insert(0, 'dead', 0)
titanic.insert(0, 'ratio', 0)


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
Finally, to have some final general idea for the data, we average the bounds.
Can you write Python code for it?

s1 = final_df[['NewCasesLower', 'NewCasesUpper']].mean(axis=1)
s2 = final_df[['NewDeathsLower', 'NewDeathsUpper']].mean(axis=1)
final = pd.concat([s1, s2], axis=1)
final.columns = ['NewCasesAverage', 'NewDeathsAverage']
final


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
Finally, we do a merge with the metadata. We join on the BARCODE column. This column will be the index of the metadata when we import it in this case. Finally we set the index for the three columns BARCODE, GROUP and SAMPLE which are all the columns of the metada and are unique.
The only NaN value we found was the NA value on the metadata, which may indicate that there is no sample for the first group. We replaced it anyway by unknown.
Can you write Python code for it?

metadata = pd.read_excel(DATA_FOLDER + '/microbiome/metadata.xls', sheetname='Sheet1', index_col=0)
metadata.fillna('unknown', inplace=True)
merged = pd.merge(mid, metadata, right_index=True, left_on='BARCODE')
merged = merged.set_index(keys=['BARCODE', 'Taxon'])
merged


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
For each of the following questions state clearly your assumptions and discuss your findings:
1. Describe the type and the value range of each attribute. Indicate and transform the attributes that can be Categorical.
2. Plot histograms for the travel class, embarkation port, sex and age attributes. For the latter one, use discrete decade intervals.
3. Calculate the proportion of passengers by cabin floor. Present your results in a pie chart.
4. For each travel class, calculate the proportion of the passengers that survived. Present your results in pie charts.
5. Calculate the proportion of the passengers that survived by travel class and sex. Present your results in a single histogram.
6. Create 2 equally populated age categories and calculate survival proportions by age category, travel class and sex. Present your results in a DataFrame with unique index.
1. We start by importing the data from the file.
Can you write Python code for it?

titanic = pd.read_excel(DATA_FOLDER + '/titanic.xls', sheetname='titanic')
titanic


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
Here we set these new columns to appropriate values. We essentialy separate the survived columns for easier summing later on. Finnaly we slice the data to take only the columns of interest.
Can you write Python code for it?

titanic.loc[titanic['survived'] == "Survived", 'alive'] = 1
titanic.loc[titanic['survived'] == "Deceased", 'dead'] = 1
df = titanic[['pclass', 'sex', 'alive', 'dead', 'ratio']]


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
Next we can list the data types of each field.
Can you write Python code for it?


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
Next we plot the histogram of the sex.
Can you write Python code for it?


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
Next, we plot the proportion of people that survived in the second class.
Can you write Python code for it?

titanic[titanic.pclass == "Second Class"].survived.value_counts(sort=False).plot(kind='pie')


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
Now we repeat this operation for every other spreadsheet except the metadata. At each iteration we simply concatenate the data at the end of the previous data, this accumulating all the files' data into a single dataframe. We don't care about any index right now since we will use a random one later.
Can you write Python code for it?

for i in range(2, 10):
    midi = pd.read_excel(DATA_FOLDER + '/microbiome/MID' + str(i) + '.xls', sheetname='Sheet 1', header=None)
    midi.fillna('unknown', inplace=True)
    midi['BARCODE'] = 'MID' + str(i)
    midi.columns = ['Taxon', 'Count', 'BARCODE']
    mid = pd.concat([mid, midi])


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
Task 3. Class War in Titanic
Use pandas to import the data file Data/titanic.xls. It contains data on all the passengers that travelled on the Titanic.
Can you write Python code for it?

from IPython.core.display import HTML
HTML(filename=DATA_FOLDER+'/titanic.html')


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
We can see that there is a huge difference of survival between the classes and sexes : for instance, the third class males have 7 less times chance of survival than the first class females.
6. Next we insert a new column that will be the age category of each person. Since we wan't to split the people in two equal groups based on age, we compute the median age of passangers. We also drop the passengers with an unknown age value, to avoid bad results for the median computation.
Can you write Python code for it?

titanic.dropna(axis=0, subset=['age'], inplace=True)
titanic.insert(0, 'age_category', 0)
median = titanic['age'].median()


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
We group the data by the sec and class of the passangers and we sum it. Then we have the sum of alive and dead people groupped as we wish and we can easily calculate the proportion of them that survived, which we plot as a histogram.
Can you write Python code for it?

aggregated = df.groupby(['sex', 'pclass']).sum()
(aggregated['alive'] / (aggregated['alive'] + aggregated['dead'])).plot(kind='bar')


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
We now transform the data for the Sierra Leone :
- we first create a new dictionary for which the keys are date, and the values are the previously extracted values from the .csv files.
- then for each value in this dictionary, we try to get the value of the day before, and perform the difference to get the new deaths of this day.
Can you write Python code for it?

rows_sl_total_deaths = [get_row_sl(file) for file in get_files("sl")]
dic_sl_total_deaths = {}
for row in rows_sl_total_deaths:
    dic_sl_total_deaths[row['Date']] = row

rows_sl = []
for date, entry in dic_sl_total_deaths.items():
    date_before = date - datetime.timedelta(days=1)
    if date_before in dic_sl_total_deaths:

        if entry['TotalDeathUpper'] != 0 and dic_sl_total_deaths[date_before]['TotalDeathUpper'] != 0 and entry['TotalDeathLower'] != 0 and dic_sl_total_deaths[date_before]['TotalDeathLower'] != 0:
            copy = dict(entry)
            del copy['TotalDeathUpper']
            del copy['TotalDeathLower']

            copy['NewDeathsUpper'] = entry['TotalDeathUpper'] - dic_sl_total_deaths[date_before]['TotalDeathUpper']
            copy['NewDeathsLower'] = entry['TotalDeathLower'] - dic_sl_total_deaths[date_before]['TotalDeathLower']

            rows_sl.append(copy)


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
When it comes to the object fields, we can be a bit more precise. name, sec, ticket, cabin, embarked, boat and home.dex are all strings.
Next, we call the describe method to list some statistics on the data. We thus obtain the range of all of the numeric fields of our data.
Can you write Python code for it?


I am working on the file "Homework01/Homework 1.ipynb".
The first task is:
sum_row: for a given row, returns the total value for the new cases / deaths. We first defined this function as the sum of all new cases / deaths in all provinces, but we discovered some strange data for some provinces, so we decided to only take into account the 'total' column
sum_rows: sum all the rows given in argument
Can you write Python code for it?

def sum_row(row, total_col):
    return float(row[total_col].values[0])

def sum_rows(rows, total_col):
    tot = 0
    for row in rows:
        tot += sum_row(row, total_col)
    return tot


I am working on the file "Homework01/Homework-1-final.ipynb".
The first task is:
3. Cleaning and reindexing

At first we get rid of the NaN value, we must replace them by "unknown". In order to have a more meaningful and single index, we will reset it to be the name of the RNA sequence.
Can you write Python code for it?

#Replacing the NaN values with unknwown
Combined_data = Combined_data.fillna('unknown')

#Reseting the index
Combined_data = Combined_data.set_index('Name')

#Showing the result
Combined_data


I am working on the file "Homework01/Homework-1-final.ipynb".
The first task is:
Creating and filling the DataFrame
In order to iterate only once over the data folder, we will attach the metadata to each excel spreadsheet right after creating a DataFrame with it. This will allow the code to be shorter and clearer, but also to iterate only once on every line and therefore be more efficient.
Can you write Python code for it?

#Creating an empty DataFrame to store our data and initializing a counter.
Combined_data = pd.DataFrame()
K = 0
while (K < int(len(allFiles))):

    #Creating a DataFrame and filling it with the excel's data
    df = pd.read_excel(allFiles[K], header=None)

    #Getting the metadata of the corresponding spreadsheet
    df['BARCODE'] = Sheet10_Meta.at[int(K), 'BARCODE']
    df['GROUP'] = Sheet10_Meta.at[int(K), 'GROUP']
    df['SAMPLE'] = Sheet10_Meta.at[int(K),'SAMPLE']

    #Append the recently created DataFrame to our combined one
    Combined_data = Combined_data.append(df)

    K = K + 1

#Renaming the columns with meaningfull names
Combined_data.columns = ['Name', 'Value','BARCODE','GROUP','SAMPLE']
Combined_data.head()


I am working on the file "Homework01/Homework-1-final.ipynb".
The first task is:
Question 3.3
Calculate the proportion of passengers by cabin floor. Present your results in a pie chart.
assumptions:
- Because we are tasked with categorizing persons by the floor of their cabin it was problematic that you had cabin input: "F E57" and "F G63". There were only 7 of these instances with conflicting cabinfloors. We also presumed that the was a floor "T". Even though there was only one instance, so it might have been a typo.
- We assume that you don't want to include people without cabinfloor
Can you write Python code for it?

'''
Parsing the cabinfloor, into floors A, B, C, D, E, F, G, T and display in a pie chart

'''
#Dropping NaN (People without cabin)
cabin_floors = titanic.cabin.dropna()

# removes digits and spaces
cabin_floors = cabin_floors.str.replace(r'[\d ]+', '')
# removes duplicate letters and leave unique (CC -> C) (FG -> G)
cabin_floors = cabin_floors.str.replace(r'(.)(?=.*\1)', '')
# removes ambigous data from the dataset (FE -> NaN)(FG -> NaN)
cabin_floors = cabin_floors.str.replace(r'([A-Z]{1})\w+', 'NaN' )

# Recategorizing (Since we altered the entries, we messed with the categories)
cabin_floors = cabin_floors.astype('category')
# Removing NaN (uin this case ambigous data)
cabin_floors = cabin_floors.cat.remove_categories('NaN')
cabin_floors = cabin_floors.dropna()

# Preparing data for plt.pie
numberOfCabinPlaces = cabin_floors.count()
grouped = cabin_floors.groupby(cabin_floors).count()
sizes = np.array(grouped)
labels = np.array(grouped.index)

# Plotting the pie chart
plt.pie(sizes, labels=labels, autopct='%1.1f%%', pctdistance=0.75, labeldistance=1.1)
print("There are {cabin} passengers that have cabins and {nocabin} passengers without a cabin"
      .format(cabin=numberOfCabinPlaces, nocabin=(len(titanic) - numberOfCabinPlaces)))


I am working on the file "Homework01/Homework-1-final.ipynb".
The first task is:
Question 3.4
For each travel class, calculate the proportion of the passengers that survived. Present your results in pie charts.
assumptions:
Can you write Python code for it?

# function that returns the number of people that survived and died given a specific travelclass
def survivedPerClass(pclass):
    survived = len(titanic.survived[titanic.survived == 1][titanic.pclass == pclass])
    died = len(titanic.survived[titanic.survived == 0][titanic.pclass == pclass])
    return [survived, died]

# Fixing the layout horizontal
the_grid = plt.GridSpec(1, 3)
labels = ["Survived", "Died"]

# Each iteration plots a pie chart
for p in titanic.pclass.unique():
    sizes = survivedPerClass(p)
    plt.subplot(the_grid[0, p-1], aspect=1 )
    plt.pie(sizes, labels=labels, autopct='%1.1f%%')

plt.show()


I am working on the file "Homework01/Homework-1-final.ipynb".
The first task is:
Question 3.5
"Calculate the proportion of the passengers that survived by travel class and sex. Present your results in a single histogram."
assumptions:
    1. By "proportions" We assume it is a likelyhood-percentage of surviving
Can you write Python code for it?

# group by selected data and get a count for each category
survivalrate = titanic.groupby(['pclass', 'sex', 'survived']).size()

# calculate percentage
survivalpercentage = survivalrate.groupby(level=['pclass', 'sex']).apply(lambda x: x / x.sum() * 100)

# plotting in a histogram
histogram = survivalpercentage.filter(like='1', axis=0).plot(kind='bar')
histogram.set_title('Proportion of the passengers that survived by travel class and sex')
histogram.set_ylabel('Percent likelyhood of surviving titanic')
histogram.set_xlabel('class/gender group')
plt.show(histogram)


I am working on the file "Homework01/Homework-1-final.ipynb".
The first task is:
Question 3.6
"Create 2 equally populated age categories and calculate survival proportions by age category, travel class and sex. Present your results in a DataFrame with unique index."
assumptions:
1. By "proportions" we assume it is a likelyhood-percentage of surviving
2. To create 2 equally populated age categories; we will find the median and round up from the median to nearest whole year difference before splitting.
Can you write Python code for it?

#drop NaN rows
age_without_nan = titanic.age.dropna()

#categorizing
age_categories = pd.qcut(age_without_nan, 2, labels=["Younger", "Older"])

#Numbers to explain difference
median = int(np.float64(age_without_nan.median()))
amount = int(age_without_nan[median])
print("The Median age is {median} years old".format(median = median))
print("and there are {amount} passengers that are {median} year old \n".format(amount=amount, median=median))

print(age_categories.groupby(age_categories).count())
print("\nAs you can see the pd.qcut does not cut into entirely equal sized bins, because the age is of a discreet nature")

# imported for the sake of surpressing some warnings
import warnings
warnings.filterwarnings('ignore')

# extract relevant attributes
csas = titanic[['pclass', 'sex', 'age', 'survived']]
csas.dropna(subset=['age'], inplace=True)

# Defining the categories
csas['age_group'] = csas.age > csas.age.median()
csas['age_group'] = csas['age_group'].map(lambda age_category: 'older' if age_category else "younger")

# Converting to int to make it able to aggregate and give percentage
csas.survived = csas.survived.astype(int)

g_categories = csas.groupby(['pclass', 'age_group', 'sex'])
result = pd.DataFrame(g_categories.survived.mean()).rename(columns={'survived': 'survived proportion'})

# reset current index and spesify the unique index
result.reset_index(inplace=True)
unique_index = result.pclass.astype(str) + ': ' + result.age_group.astype(str) + ' ' + result.sex.astype(str)

# Finalize the unique index dataframe
result_w_unique = result[['survived proportion']]
result_w_unique.set_index(unique_index, inplace=True)
print(result_w_unique)


I am working on the file "Homework01/Homework-1-final.ipynb".
The first task is:
Task 1. Compiling Ebola Data
The DATA_FOLDER/ebola folder contains summarized reports of Ebola cases from three countries (Guinea, Liberia and Sierra Leone) during the recent outbreak of the disease in West Africa. For each country, there are daily reports that contain various information about the outbreak in several cities in each country.
Use pandas to import these data files into a single Dataframe.
Using this DataFrame, calculate for each country, the daily average per month of new cases and deaths.
Make sure you handle all the different expressions for new cases and deaths that are used in the reports.
Can you write Python code for it?

'''
Functions needed to solve task 1
'''

#function to import excel file into a dataframe
def importdata(path,date):
    allpathFiles = glob.glob(DATA_FOLDER+path+'/*.csv')
    list_data = []
    for file in allpathFiles:
        excel = pd.read_csv(file,parse_dates=[date])
        list_data.append(excel)
    return pd.concat(list_data)

#function to add the month on a new column of a DataFrame
def add_month(df):
    copy_df = df.copy()
    months = [calendar.month_name[x.month] for x in copy_df.Date]
    copy_df['Month'] = months
    return copy_df

#founction which loc only the column within a country and a specified month
#return a dataframe
def chooseCountry_month(dataframe,country,descr,month):
    df = dataframe.loc[(dataframe['Country']==country) & (dataframe['Description']==descr)]
    #df = add_month(df)
    df_month = df.loc[(df['Month']==month)]
    return df_month

# Create a dataframe with the number of death, the new cases and the daily infos for a country and a specified month
def getmonthresults(dataframe,country,month):
    if country =='Liberia':
        descr_kill ='Total death/s in confirmed cases'
        descr_cases ='Total confirmed cases'
    if country =='Guinea':
        descr_kill ='Total deaths of confirmed'
        descr_cases ='Total cases of confirmed'
    if country == 'Sierra Leone':
        descr_kill ='death_confirmed'
        descr_cases ='cum_confirmed'

    df_kill = chooseCountry_month(dataframe,country,descr_kill,month)
    df_cases = chooseCountry_month(dataframe,country,descr_cases,month)

    #calculate the number of new cases and of new deaths for the all month
    res_kill = int(df_kill.iloc[len(df_kill)-1].Totals)-int(df_kill.iloc[0].Totals)
    res_cases = int(df_cases.iloc[len(df_cases)-1].Totals)-int(df_cases.iloc[0].Totals)
    #calculate the number of days counted which is last day of register - first day of register
    nb_day = df_kill.iloc[len(df_kill)-1].Date.day-df_kill.iloc[0].Date.day

    # Sometimes the values in the dataframe are...


I am working on the file "Homework02/Homework 2.ipynb".
The first task is:
1. Top-ranking universities
We noticed that the actual data from topuniversities is not directly on the webpage, but on a separate text file, in JSON format.
Thus, we first get this JSON, parse it, and take the first 200 entries in it.
We noticed that the univertsity with rank 199 is actually the 198th entry, and thus the last 3 universities needs to have their rank corrected.
Can you write Python code for it?

r = requests.get('https://www.topuniversities.com/sites/default/files/qs-rankings-data/357051.txt?_=1508259845358')
raw_data = json.loads(r.text)['data'][:200]


I am working on the file "Homework02/Homework 2.ipynb".
The first task is:
4. Find useful insights in the data
We begin by cleaning and adapting our dataset to better see meaningful correlations. For example, we put the ranking of each unversity in the reverse order, since we want to have a higher value for higher ranked universities to get meaningful correlations. Moreover we drop some rows where we miss the data we need. We also add another ratio that we thought to be usefull for the last part of the homework, the international faculy ratio for each unversity.
Can you write Python code for it?

raw_merge_cleaned = raw_merge.copy()
raw_merge_cleaned['rank_time'] = 200 - raw_merge_cleaned['Rank timeshighereducation'].astype(int)
raw_merge_cleaned['rank_top'] = 200 - raw_merge_cleaned['Rank topuniversities'].astype(int)
raw_merge_cleaned[pd.notnull(raw_merge['International students ratio topuniversities'])]
raw_merge_cleaned['International faculty ratio'] = raw_merge_cleaned['International faculty'] / raw_merge_cleaned['Total faculty']

raw_merge_cleaned.corr()


I am working on the file "Homework02/Homework 2.ipynb".
The first task is:
As there can be ties in rank, the displayed rank is not always a integer. Furthermore, as said above, the last 3 universities have incorrect ranks and need to be fixed.
Can you write Python code for it?

def get_rank(rank_display):
    rank = int(rank_display.replace("=", ""))
    if rank >= 199:
        rank -= 1
    return rank


I am working on the file "Homework02/Homework 2.ipynb".
The first task is:
For clarity, we only show the universities with the 10 highest ratio.
Can you write Python code for it?

df2.sort_values('staff_student_ratio', ascending=False)[['name', 'staff_student_ratio']].head(10).plot(title='Rank according to the ratio between faculty members and students', figsize=(13,6),kind='bar', x = 'name')


df.sort_values('international_student_ratio', ascending=False)[['name', 'international_student_ratio']].head(10).plot(title='Rank according to the ratio of international students',figsize=(13,6), kind='bar', x = 'name')


I am working on the file "Homework02/Homework 2.ipynb".
The first task is:
For this ranking, we see quite a difference with the previous ranking. Here, universities with a high ratio are not necessarily the universities with the highest ranking.
Rank according to the ratio of international students
Can you write Python code for it?


I am working on the file "Homework02/Homework 2.ipynb".
The first task is:
Here we can note some surprising results. A lot of countries that have high ratios do not seem to have a lot of universities that are highly ranked. In fact we believe that for these countries, only a few or even a single university is in the ranking. Given that the university would be the best university in that country, it is understandable that their ratio is high.
Rank according to the ratio between faculty members and students, grouped by region
Can you write Python code for it?

df_staff_region = df.groupby('region').mean().sort_values('staff_student_ratio', ascending=False)[['staff_student_ratio']]
df_staff_region

df_staff_region['staff_student_ratio'].plot(title = 'Rank according to the ratio between faculty members and students, grouped by region',figsize=(13,6),kind='bar')


I am working on the file "Homework02/Homework 2.ipynb".
The first task is:
Here we note that the highest ratios seem to be in regions that have the most of highly ranked universities.
Rank according to the ratio of international students, grouped by country
Can you write Python code for it?

df_int_country = df.groupby('country').mean().sort_values('international_student_ratio', ascending=False)[['international_student_ratio']]
df_int_country

df_int_country['international_student_ratio'].plot(title='Rank according to the ratio of international students, grouped by country', figsize=(16,6),kind='bar')


I am working on the file "Homework02/Homework 2.ipynb".
The first task is:
Rank according to the ratio between faculty members and students
Can you write Python code for it?