I am working on the file "DL0110EN/2.5.2_early_stopping_v2.ipynb".
The first task is:

Early Stopping and Saving the Mode

Run several epochs of gradient descent and save the model that performs best on the validation data.
Can you write Python code for it?

# Train the model

LOSS_TRAIN = []
LOSS_VAL = []
n=1;
min_loss = 1000

def train_model_early_stopping(epochs, min_loss):
    for epoch in range(epochs):
        for x, y in trainloader:
            yhat = model(x)
            loss = criterion(yhat, y)
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            loss_train = criterion(model(train_data.x), train_data.y).data
            loss_val = criterion(model(val_data.x), val_data.y).data
            LOSS_TRAIN.append(loss_train)
            LOSS_VAL.append(loss_val)
            if loss_val < min_loss:
                value = epoch
                min_loss = loss_val
                torch.save(model.state_dict(), 'best_model.pt')

train_model_early_stopping(20, min_loss)


I am working on the file "DL0110EN/2.5.2_early_stopping_v2.ipynb".
The first task is:
Let's compare the prediction  from the model obtained using early stopping and the model derived from  using the maximum number of iterations.
Can you write Python code for it?

plt.plot(model_best(val_data.x).data.numpy(), label = 'best model')
plt.plot(model(val_data.x).data.numpy(), label = 'maximum iterations')
plt.plot(val_data.y.numpy(), 'rx', label = 'true line')
plt.legend()
plt.show()


I am working on the file "DL0110EN/2.5.2_early_stopping_v2.ipynb".
The first task is:
Load the model parameters torch.load(), then assign them to the object model_best using the method load_state_dict.
Can you write Python code for it?

# Assign the best model to model_best

model_best.load_state_dict(torch.load('best_model.pt'))


I am working on the file "DL0110EN/2.5.2_early_stopping_v2.ipynb".
The first task is:
We overlay the training points in red over the function that generated the data. Notice the outliers are at x=-3 and around x=2
Can you write Python code for it?

# Plot the training data points

plt.plot(train_data.x.numpy(), train_data.y.numpy(), 'xr')
plt.plot(train_data.x.numpy(), train_data.f.numpy())
plt.show()


I am working on the file "DL0110EN/2.6.2_multiple_linear_regression_training.ipynb".
The first task is:

Make Some Data

Create a dataset class with two-dimensional features:
Can you write Python code for it?

from torch.utils.data import Dataset, DataLoader
class Data2D(Dataset):
    def __init__(self):
            self.x=torch.zeros(20,2)
            self.x[:,0]=torch.arange(-1,1,0.1)
            self.x[:,1]=torch.arange(-1,1,0.1)
            self.w=torch.tensor([ [1.0],[1.0]])
            self.b=1
            self.f=torch.mm(self.x,self.w)+self.b

            self.y=self.f+0.1*torch.randn((self.x.shape[0],1))
            self.len=self.x.shape[0]

    def __getitem__(self,index):
        return self.x[index],self.y[index]

    def __len__(self):
        return self.len


I am working on the file "DL0110EN/2.6.2_multiple_linear_regression_training.ipynb".
The first task is:

Create the Model, Optimizer, and Total Loss Function (cost)

Create a custom module:
Can you write Python code for it?

class linear_regression(nn.Module):
    def __init__(self,input_size,output_size):
        super(linear_regression,self).__init__()
        self.linear=nn.Linear(input_size,output_size)
    def forward(self,x):
        yhat=self.linear(x)
        return yhat


I am working on the file "DL0110EN/2.6.2_multiple_linear_regression_training.ipynb".
The first task is:
Create a data loader object. Set the batch_size equal to 2:
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.2_multiple_linear_regression_training.ipynb".
The first task is:
Create a dataset object:
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.2_multiple_linear_regression_training.ipynb".
The first task is:
Create a model. Use two features: make the input size 2 and the output size 1:
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.2_multiple_linear_regression_training.ipynb".
The first task is:
Create an optimizer  object. Set the learning rate to 0.1. Don't forget to enter the model parameters in the constructor.
<img src = "https://ibm.box.com/shared/static/f8hskuwrnctjg21agud69ddla0jkbef5.png" width = 100, align = "center">
Can you write Python code for it?

optimizer = optim.SGD(model.parameters(), lr = 0.1)


I am working on the file "DL0110EN/2.6.2_multiple_linear_regression_training.ipynb".
The first task is:
Create the criterion function that calculates the total loss or cost:
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.2_multiple_linear_regression_training.ipynb".
The first task is:
Set the random seed:
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.3.2.training_multiple_output_linear_regression.ipynb".
The first task is:

Make Some Data

Create a dataset class with two-dimensional features and two targets:
Can you write Python code for it?

from torch.utils.data import Dataset, DataLoader
class Data(Dataset):
    def __init__(self):
            self.x=torch.zeros(20,2)
            self.x[:,0]=torch.arange(-1,1,0.1)
            self.x[:,1]=torch.arange(-1,1,0.1)
            self.w=torch.tensor([ [1.0,-1.0],[1.0,3.0]])
            self.b=torch.tensor([[1.0,-1.0]])
            self.f=torch.mm(self.x,self.w)+self.b

            self.y=self.f+0.001*torch.randn((self.x.shape[0],1))
            self.len=self.x.shape[0]

    def __getitem__(self,index):

        return self.x[index],self.y[index]

    def __len__(self):
        return self.len


I am working on the file "DL0110EN/2.6.3.2.training_multiple_output_linear_regression.ipynb".
The first task is:

Create the Model, Optimizer, and Total Loss Function (cost)

Create a custom module:
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.3.2.training_multiple_output_linear_regression.ipynb".
The first task is:
Create a data loader object and set the batch_size to 5:
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.3.2.training_multiple_output_linear_regression.ipynb".
The first task is:
Create the criterion function that calculates the total loss or cost:
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.3.2.training_multiple_output_linear_regression.ipynb".
The first task is:
create a dataset object
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.3.multi-target_linear_regression.ipynb".
The first task is:
we can create a tensor with two rows representing one sample of data
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.3.multi-target_linear_regression.ipynb".
The first task is:
we can make a prediction
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.3.multi-target_linear_regression.ipynb".
The first task is:
we can make a prediction using multiple samples
Can you write Python code for it?


I am working on the file "DL0110EN/2.6.3.multi-target_linear_regression.ipynb".
The first task is:
we can use the diagram to represent the model or object
<img src = "https://ibm.box.com/shared/static/icmwnxru7nytlhnq5x486rffea9ncpk7.png" width = 600, align = "center">
we can see the parameters
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:

 Logistic Function

Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:

Build a Logistic Regression Using nn.Sequential

Create a 1x1 tensor where x represents one data sample with one dimension, and 2x1 tensor X represents two data samples of one dimension:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Apply the element-wise Sigmoid from the function module and plot the results:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Apply the element-wise function Sigmoid with the object:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Calling the object performed the following operation:
Create a 1x2 tensor where x represents one data sample with one dimension, and 2x3 tensor X represents one data sample of two dimension:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Create a 1x1 tensor where x represents one data sample with one dimension, and 3x1 tensor X represents one data sample of one dimension:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Create a logistic regression object with a function with two inputs:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Create a logistic regression object with the nn.Sequential model with a two-dimensional input:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Create a logistic regression object with the nn.Sequential model with a one-dimensional input:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Create a model to predict one dimension:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Create a sigmoid object:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Create a tensor ranging from -100 to 100:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
In this case, the parameters are randomly insulated. You can view them the following ways:
Can you write Python code for it?

print("list(model.parameters()):\n ", list(model.parameters()))
print( "\nmodel.state_dict():\n ",model.state_dict())


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Make a prediction with multiple samples:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Make a prediction with one sample:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
Plot the results:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
The object is represented in the following diagram:
<img src = "https://ibm.box.com/shared/static/8yjc6pyix9ga1x9eeog49u38i59uctog.png" width = 800, align = "center">
In this case, the parameters are randomly insulated. You can view them the following ways:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
The operation is represented in the following diagram:
<img src = "https://ibm.box.com/shared/static/pnarlh41gl7epmy2f8ob3l58pd0zh9wq.png" width = 500, align = "center">
Make a prediction with multiple samples:
Can you write Python code for it?


I am working on the file "DL0110EN/3.1.1logistic_regression_prediction.ipynb".
The first task is:
The operation is represented in the following diagram:

 Build Custom Modules

In this section, you will build a custom model. The model or object function is identical to  using nn.Sequential.
Create a logistic regression custom module:
Can you write Python code for it?

class logistic_regression(nn.Module):
    def __init__(self,n_inputs):
        super(logistic_regression,self).__init__()
        self.linear=nn.Linear(n_inputs,1)
    def forward(self,x):
        yhat=torch.sigmoid(self.linear(x))
        return yhat


I am working on the file "DL0110EN/3.2.1.logistic_regression_with_mean_square_error_v2.ipynb".
The first task is:

Create the Model and Total Loss Function (Cost)

Create a custom module for logistic regression:
Can you write Python code for it?

# Create logistic_regression class

class logistic_regression(nn.Module):

    # Construstor
    def __init__(self,n_inputs):
        super(logistic_regression, self).__init__()
        self.linear = nn.Linear(n_inputs, 1)

    # Prediction
    def forward(self, x):
        yhat = torch.sigmoid(self.linear(x))
        return yhat


I am working on the file "DL0110EN/3.2.1.logistic_regression_with_mean_square_error_v2.ipynb".
The first task is:
Create DataLoader object, cost or criterion function and optimizer:
Can you write Python code for it?

# Create DataLoader, Cost Function, Optimizer

trainloader = DataLoader(dataset = data_set, batch_size = 3)
criterion_rms = nn.MSELoss()
learning_rate = 2
optimizer = torch.optim.SGD(model.parameters(), lr = learning_rate)


I am working on the file "DL0110EN/3.2.1.logistic_regression_with_mean_square_error_v2.ipynb".
The first task is:
Create a plot_error_surfaces object to visualize the data space and the parameter space during training:
Can you write Python code for it?

# Create plot_error_surfaces object

get_surface = plot_error_surfaces(15, 13, data_set[:][0], data_set[:][1], 30)


I am working on the file "DL0110EN/3.2.1.logistic_regression_with_mean_square_error_v2.ipynb".
The first task is:
Create a logistic regression object and print the parameters:
Can you write Python code for it?

# Create model object and print parameter

model = logistic_regression(1)
print("The parameters: ", model.state_dict())


I am working on the file "DL0110EN/3.2.1.logistic_regression_with_mean_square_error_v2.ipynb".
The first task is:
Get the actual class of each sample and calculate the accuracy on the test data.
Can you write Python code for it?

# Make the Prediction

yhat = model(data_set.x)
label = yhat > 0.5
print("The accuracy: ", torch.mean((label == data_set.y.type(torch.ByteTensor)).type(torch.float)))


I am working on the file "DL0110EN/3.2.1.logistic_regression_with_mean_square_error_v2.ipynb".
The first task is:
Set the random seed:
Can you write Python code for it?


I am working on the file "DL0110EN/3.2.1.logistic_regression_with_mean_square_error_v2.ipynb".
The first task is:
The class plot_error_surfaces is just to help you visualize the data space and the parameter space during training and has nothing to do with PyTorch.
Can you write Python code for it?

# Create class for plotting and the function for plotting

class plot_error_surfaces(object):

    # Construstor
    def __init__(self, w_range, b_range, X, Y, n_samples = 30, go = True):
        W = np.linspace(-w_range, w_range, n_samples)
        B = np.linspace(-b_range, b_range, n_samples)
        w, b = np.meshgrid(W, B)
        Z = np.zeros((30, 30))
        count1 = 0
        self.y = Y.numpy()
        self.x = X.numpy()
        for w1, b1 in zip(w, b):
            count2 = 0
            for w2, b2 in zip(w1, b1):
                Z[count1, count2] = np.mean((self.y - (1 / (1 + np.exp(-1 * (w2 * self.x + b2))))) ** 2)
                count2 += 1
            count1 += 1
        self.Z = Z
        self.w = w
        self.b = b
        self.W = []
        self.B = []
        self.LOSS = []
        self.n = 0
        if go == True:
            plt.figure()
            plt.figure(figsize = (7.5, 5))
            plt.axes(projection = '3d').plot_surface(self.w, self.b, self.Z, rstride = 1, cstride = 1, cmap = 'viridis', edgecolor = 'none')
            plt.title('Loss Surface')
            plt.xlabel('w')
            plt.ylabel('b')
            plt.show()
            plt.figure()
            plt.title('Loss Surface Contour')
            plt.xlabel('w')
            plt.ylabel('b')
            plt.contour(self.w, self.b, self.Z)
            plt.show()

     # Setter
    def set_para_loss(self, model, loss):
        self.n = self.n + 1
        self.W.append(list(model.parameters())[0].item())
        self.B.append(list(model.parameters())[1].item())
        self.LOSS.append(loss)

    # Plot diagram
    def final_plot(self):
        ax = plt.axes(projection = '3d')
        ax.plot_wireframe(self.w, self.b, self.Z)
        ax.scatter(self.W, self.B, self.LOSS, c = 'r', marker = 'x', s = 200, alpha = 1)
        plt.figure()
        plt.contour(self.w, self.b, self.Z)
        plt.scatter(self.W, self.B, c = 'r', marker = 'x')
        plt.xlabel('w')
        plt.ylabel('b')
       ...


I am working on the file "DL0110EN/3.2.2.bad_inshilization_logistic_regression_with_mean_square_error.ipynb".
The first task is:

Create the Model and Total Loss Function (cost)

Create a custom module for logistic regression:
Can you write Python code for it?


I am working on the file "DL0110EN/3.2.2.bad_inshilization_logistic_regression_with_mean_square_error.ipynb".
The first task is:

Train the Model via Batch Gradient Descent

Can you write Python code for it?

for epoch in range(100):

    for x,y in trainloader:
        #make a prediction
        yhat= model(x)
        #calculate the loss
        loss = criterion_rms(yhat, y)
        #clear gradient
        optimizer.zero_grad()
        #Backward pass: compute gradient of the loss with respect to all the learnable parameters
        loss.backward()
        #the step function on an Optimizer makes an update to its parameters
        optimizer.step()
        #for plotting
        get_surface.get_stuff(model,loss.tolist())
        #plot every 20 iterataions
    if epoch%20==0:
        get_surface.plot_ps()


I am working on the file "DL0110EN/3.2.2.bad_inshilization_logistic_regression_with_mean_square_error.ipynb".
The first task is:
Create a dataloader object:
Can you write Python code for it?

learning_rate=2

optimizer=torch.optim.SGD(model.parameters(), lr=learning_rate)


I am working on the file "DL0110EN/3.2.2.bad_inshilization_logistic_regression_with_mean_square_error.ipynb".
The first task is:
Helper functions
The class plot_error_surfaces is just to help you visualize the data space and the parameter space during training and has nothing to do with Pytorch.
Can you write Python code for it?

class plot_error_surfaces(object):
    def __init__(self,w_range, b_range,X,Y,n_samples=30,go=True):
        W = np.linspace(-w_range, w_range, n_samples)
        B = np.linspace(-b_range, b_range, n_samples)
        w, b = np.meshgrid(W, B)
        Z=np.zeros((30,30))
        count1=0
        self.y=Y.numpy()
        self.x=X.numpy()
        for w1,b1 in zip(w,b):
            count2=0
            for w2,b2 in zip(w1,b1):
                1 / (1 + np.exp(w2*self.x+b2))
                Z[count1,count2]=np.mean((self.y-(1 / (1 + np.exp(w2*self.x+b2))))**2)
                count2 +=1

            count1 +=1
        self.Z=Z
        self.w=w
        self.b=b
        self.W=[]
        self.B=[]
        self.LOSS=[]
        self.n=0
        if go==True:
            plt.figure()
            plt.figure(figsize=(7.5,5))
            plt.axes(projection='3d').plot_surface(self.w, self.b, self.Z, rstride=1, cstride=1,cmap='viridis', edgecolor='none')
            plt.title('Loss Surface')
            plt.xlabel('w')
            plt.ylabel('b')
            plt.show()
            plt.figure()
            plt.title('Loss Surface Contour')
            plt.xlabel('w')
            plt.ylabel('b')
            plt.contour(self.w, self.b, self.Z)
            plt.show()
    def get_stuff(self,model,loss):
        self.n=self.n+1
        self.W.append(list(model.parameters())[0].item())
        self.B.append(list(model.parameters())[1].item())
        self.LOSS.append(loss)

    def final_plot(self):
        ax = plt.axes(projection='3d')
        ax.plot_wireframe(self.w, self.b, self.Z)
        ax.scatter(self.W,self.B, self.LOSS, c='r', marker='x',s=200,alpha=1)
        plt.figure()
        plt.contour(self.w,self.b, self.Z)
        plt.scatter(self.W,self.B,c='r', marker='x')
        plt.xlabel('w')
        plt.ylabel('b')
        plt.show()
    def plot_ps(self):
        plt.subplot(121)
        plt.ylim
        plt.plot(self.x,self.y,'ro',label="training points")
       ...


I am working on the file "DL0110EN/3.2.2.bad_inshilization_logistic_regression_with_mean_square_error.ipynb".
The first task is:
Replace the random initialized variable values with some predetermined values that will not converge:
Can you write Python code for it?

model.state_dict() ['linear.weight'].data[0]=torch.tensor([[-5]])
model.state_dict() ['linear.bias'].data[0]=torch.tensor([[-10]])

model.state_dict()


I am working on the file "DL0110EN/3.2.3cross_entropy_logistic_regression.ipynb".
The first task is:

Create the Model and Total Loss Function (cost)

Create a custom module for logistic regression:
Can you write Python code for it?


I am working on the file "DL0110EN/3.2.3cross_entropy_logistic_regression.ipynb".
The first task is:

Train the Model via Batch Gradient Descent

Can you write Python code for it?

for epoch in range(100):

    for x,y in trainloader:
        #make a prediction
        yhat= model(x)
        #calculate the loss
        loss = criterion(yhat, y)
        #clear gradient
        optimizer.zero_grad()
        #Backward pass: compute gradient of the loss with respect to all the learnable parameters
        loss.backward()
        #the step function on an Optimizer makes an update to its parameters
        optimizer.step()
        #for plotting
        get_surface.get_stuff(model,loss.tolist())
        #plot every 20 iterataions
    if epoch%20==0:
        get_surface.plot_ps()


I am working on the file "DL0110EN/3.2.3cross_entropy_logistic_regression.ipynb".
The first task is:
Get the actual class of each sample and calculate the accuracy on the test data:
Can you write Python code for it?


I am working on the file "DL0110EN/3.2.3cross_entropy_logistic_regression.ipynb".
The first task is:
Helper functions
The class plot_error_surfaces is just to help you visualize the data space and the parameter space during training and has nothing to do with Pytorch.
Can you write Python code for it?

class plot_error_surfaces(object):
    def __init__(self,w_range, b_range,X,Y,n_samples=50,go=True):
        W = np.linspace(-w_range, w_range, n_samples)
        B = np.linspace(-b_range, b_range, n_samples)
        w, b = np.meshgrid(W, B)
        Z=np.zeros((30,30))
        count1=0
        self.y=Y.numpy()
        self.x=X.numpy()
        for w1,b1 in zip(w,b):
            count2=0
            for w2,b2 in zip(w1,b1):

                yhat= 1 / (1 + np.exp(-1*(w2*self.x+b2)))
                Z[count1,count2]=-1*np.mean(self.y*np.log(yhat+1e-16) +(1-self.y)*np.log(1-yhat+1e-16))
                count2 +=1

            count1 +=1
        self.Z=Z
        self.w=w
        self.b=b
        self.W=[]
        self.B=[]
        self.LOSS=[]
        self.n=0
        if go==True:
            plt.figure()
            plt.figure(figsize=(7.5,5))
            plt.axes(projection='3d').plot_surface(self.w, self.b, self.Z, rstride=1, cstride=1,cmap='viridis', edgecolor='none')
            plt.title('Loss Surface')
            plt.xlabel('w')
            plt.ylabel('b')
            plt.show()
            plt.figure()
            plt.title('Loss Surface Contour')
            plt.xlabel('w')
            plt.ylabel('b')
            plt.contour(self.w, self.b, self.Z)
            plt.show()
    def get_stuff(self,model,loss):
        self.n=self.n+1
        self.W.append(list(model.parameters())[0].item())
        self.B.append(list(model.parameters())[1].item())
        self.LOSS.append(loss)

    def final_plot(self):
        ax = plt.axes(projection='3d')
        ax.plot_wireframe(self.w, self.b, self.Z)
        ax.scatter(self.W,self.B, self.LOSS, c='r', marker='x',s=200,alpha=1)
        plt.figure()
        plt.contour(self.w,self.b, self.Z)
        plt.scatter(self.W,self.B,c='r', marker='x')
        plt.xlabel('w')
        plt.ylabel('b')
        plt.show()
    def plot_ps(self):
        plt.subplot(121)
        plt.ylim
        plt.plot(self.x,self.y,'ro',label="training points")
 ...


I am working on the file "DL0110EN/3.2.3cross_entropy_logistic_regression.ipynb".
The first task is:
Replace the random initialized variable values. Theses random initialized variable values did convergence for the RMS Loss but will converge for the Cross-Entropy Loss.
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.1_softmax_in_one_dimension_v2.ipynb".
The first task is:

Train the Model

Create the criterion function, the optimizer and the dataloader
Can you write Python code for it?

# Create criterion function, optimizer, and dataloader

criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr = 0.01)
trainloader = DataLoader(dataset = data_set, batch_size = 5)


I am working on the file "DL0110EN/3.3.1_softmax_in_one_dimension_v2.ipynb".
The first task is:

Analyze Results

Find the predicted class on the test data:
Can you write Python code for it?

# Make the prediction

z =  model(data_set.x)
_, yhat = z.max(1)
print("The prediction:", yhat)


I am working on the file "DL0110EN/3.3.1_softmax_in_one_dimension_v2.ipynb".
The first task is:
<!--Empty Space for separating topics-->

<h2 id="Softmax">Build a Softmax Classifier </h2>

Build a Softmax classifier by using the Sequential module:
Can you write Python code for it?

# Build Softmax Classifier technically you only need  linear

model = nn.Sequential(nn.Linear(1, 3))
model.state_dict()


I am working on the file "DL0110EN/3.3.1_softmax_in_one_dimension_v2.ipynb".
The first task is:
Calculate the accuracy on the test data:
Can you write Python code for it?

# Print the accuracy

correct = (data_set.y == yhat).sum().item()
accuracy = correct / len(data_set)
print("The accuracy: ", accuracy)


I am working on the file "DL0110EN/3.3.1_softmax_in_one_dimension_v2.ipynb".
The first task is:
Create the dataset object:
Can you write Python code for it?

# Create the dataset object and plot the dataset object

data_set = Data()
data_set.x
plot_data(data_set)


I am working on the file "DL0110EN/3.3.1_softmax_in_one_dimension_v2.ipynb".
The first task is:
Set the random seed:
Can you write Python code for it?

#Set the random seed

torch.manual_seed(0)


I am working on the file "DL0110EN/3.3.1_softmax_in_one_dimension_v2.ipynb".
The first task is:
Train the model for every 50 epochs plot, the line generated for each class.
Can you write Python code for it?

# Train the model

LOSS = []
def train_model(epochs):
    for epoch in range(epochs):
        if epoch % 50 == 0:
            pass
            plot_data(data_set, model)
        for x, y in trainloader:
            optimizer.zero_grad()
            yhat = model(x)
            loss = criterion(yhat, y)
            LOSS.append(loss)
            loss.backward()
            optimizer.step()
train_model(300)


I am working on the file "DL0110EN/3.3.1_softmax_in_one_dimension_v2.ipynb".
The first task is:
Use the helper function to plot labeled data points:
Can you write Python code for it?

# Create class for plotting

def plot_data(data_set, model = None, n = 1, color = False):
    X = data_set[:][0]
    Y = data_set[:][1]
    plt.plot(X[Y == 0, 0].numpy(), Y[Y == 0].numpy(), 'bo', label = 'y = 0')
    plt.plot(X[Y == 1, 0].numpy(), 0 * Y[Y == 1].numpy(), 'ro', label = 'y = 1')
    plt.plot(X[Y == 2, 0].numpy(), 0 * Y[Y == 2].numpy(), 'go', label = 'y = 2')
    plt.ylim((-0.1, 3))
    plt.legend()
    if model != None:
        w = list(model.parameters())[0][0].detach()
        b = list(model.parameters())[1][0].detach()
        y_label = ['yhat=0', 'yhat=1', 'yhat=2']
        y_color = ['b', 'r', 'g']
        Y = []
        for w, b, y_l, y_c in zip(model.state_dict()['0.weight'], model.state_dict()['0.bias'], y_label, y_color):
            Y.append((w * X + b).numpy())
            plt.plot(X.numpy(), (w * X + b).numpy(), y_c, label = y_l)
        if color == True:
            x = X.numpy()
            x = x.reshape(-1)
            top = np.ones(x.shape)
            y0 = Y[0].reshape(-1)
            y1 = Y[1].reshape(-1)
            y2 = Y[2].reshape(-1)
            plt.fill_between(x, y0, where = y1 > y1, interpolate = True, color = 'blue')
            plt.fill_between(x, y0, where = y1 > y2, interpolate = True, color = 'blue')
            plt.fill_between(x, y1, where = y1 > y0, interpolate = True, color = 'red')
            plt.fill_between(x, y1, where = ((y1 > y2) * (y1 > y0)),interpolate = True, color = 'red')
            plt.fill_between(x, y2, where = (y2 > y0) * (y0 > 0),interpolate = True, color = 'green')
            plt.fill_between(x, y2, where = (y2 > y1), interpolate = True, color = 'green')
    plt.legend()
    plt.show()


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:

Prepare Data

Load the training dataset by setting the parameters train to True and convert it to a tensor by placing a transform object in the argument transform.
Can you write Python code for it?

train_dataset=dsets.MNIST(root='./data', train=True, download=True, transform=transforms.ToTensor())
train_dataset


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:

 Define the Softmax Classifier, Criterion Function, Optimizer, and Train the Model

Can you write Python code for it?


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:

Build a Softmax Classifer
Build a Softmax classifier class:
Can you write Python code for it?

class SoftMax(nn.Module):
    def __init__(self,input_size,output_size):
        super(SoftMax,self).__init__()
        self.linear=nn.Linear(input_size,output_size)
    def forward(self,x):
        z=self.linear(x)
        return z


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
Define the dataset loader:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
Flatten the tensor as shown in this image:

The size of the tensor is now 784.

Set the input size and output size:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
Load the testing dataset by setting the parameters train False and convert it to a tensor by placing a transform object in the argument transform.
Can you write Python code for it?

validation_dataset=dsets.MNIST(root='./data', train=False, download=True, transform=transforms.ToTensor())
validation_dataset


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
Loss function:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
Optimizer class:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
Plot the 3rd sample:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
Plot the first five misclassified samples:
Can you write Python code for it?

count=0
for x,y in validation_dataset:

    z=model(x.reshape(-1,28*28))
    _,yhat=torch.max(z,1)
    if yhat!=y:
        show_data((x,y))

        plt.show()
        print("yhat:",yhat)
        count+=1
    if count>=5:
        break


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
The Softmax function requires vector inputs. Note that the vector shape is 28x28.
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
Train the model and determine validation accuracy (should take a few minutes):
Can you write Python code for it?

n_epochs=10
loss_list=[]
accuracy_list=[]
N_test=len(validation_dataset)
#n_epochs
for epoch in range(n_epochs):
    for x, y in train_loader:

        #clear gradient
        optimizer.zero_grad()
        #make a prediction
        z=model(x.view(-1,28*28))
        # calculate loss
        loss=criterion(z,y)
        # calculate gradients of parameters
        loss.backward()
        # update parameters
        optimizer.step()

    correct=0
    #perform a prediction on the validation  data
    for x_test, y_test in validation_loader:

        z=model(x_test.view(-1,28*28))
        _,yhat=torch.max(z.data,1)

        correct+=(yhat==y_test).sum().item()

    accuracy=correct/N_test

    accuracy_list.append(accuracy)

    loss_list.append(loss.data)
    accuracy_list.append(accuracy)


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
Use the following function to plot out the parameters of the Softmax function:
Can you write Python code for it?

def PlotParameters(model):
    W=model.state_dict() ['linear.weight'].data
    w_min=W.min().item()
    w_max=W.max().item()
    fig, axes = plt.subplots(2, 5)
    fig.subplots_adjust(hspace=0.01, wspace=0.1)
    for i,ax in enumerate(axes.flat):
        if i<10:
             # Set the label for the sub-plot.
            ax.set_xlabel( "class: {0}".format(i))

            # Plot the image.
            ax.imshow(W[i,:].view(28,28), vmin=w_min, vmax=w_max, cmap='seismic')

            ax.set_xticks([])
            ax.set_yticks([])

        # Ensure the plot is shown correctly with multiple plots
        # in a single Notebook cell.
    plt.show()


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
You can cover the model parameters for each class to a rectangular grid:

Plot the model parameters for each class:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.2lab_predicting _MNIST_using_Softmax.ipynb".
The first task is:
You see that it is a 1. Now, plot the second sample:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:

Prepare Data

Load the training dataset by setting the parameters train to True and convert it to a tensor by placing a transform object in the argument transform.
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:

Analyze Results

Plot the loss and accuracy on the validation data:
Can you write Python code for it?

fig, ax1 = plt.subplots()
color = 'tab:red'
ax1.plot(loss_list,color=color)
ax1.set_xlabel('epoch',color=color)
ax1.set_ylabel('total loss',color=color)
ax1.tick_params(axis='y', color=color)

ax2 = ax1.twinx()
color = 'tab:blue'
ax2.set_ylabel('accuracy', color=color)
ax2.plot( accuracy_list, color=color)
ax2.tick_params(axis='y', labelcolor=color)
fig.tight_layout()


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
Cover the model parameters for each class to a rectangular grid:

Plot the model parameters for each class:
Loss function:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
Define the dataset loader:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
Flatten the tensor as shown in this image:

The size of the tensor is now 784.

Set the input size and output size.

Create a Softmax Classifier by Using Sequential
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
Load the testing dataset by setting the parameters train False and convert it to a tensor by placing a transform object in the argument transform.
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
Note that the data type is long:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
Optimizer class:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
Plot the 3rd sample:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
Plot the first five misclassified samples:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
The Softmax function requires vector inputs. If you see the vector shape, you'll note it's 28x28.
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
Train the model and determine validation accuracy:
Can you write Python code for it?


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
Use the following function to visualize data:
Can you write Python code for it?

def show_data(data_sample):

    plt.imshow(data_sample[0].numpy().reshape(28,28),cmap='gray')
    #print(data_sample[1].item())
    plt.title('y= '+ str(data_sample[1].item()))


I am working on the file "DL0110EN/3.3.3practice_predicting_MNIST.ipynb".
The first task is:
You see its a 1. Now, plot the second sample:
Can you write Python code for it?


I am working on the file "DL0110EN/4.1.1simple1hiddenlayer .ipynb".
The first task is:

Helper Functions

Can you write Python code for it?

def PlotStuff(X,Y,model,epoch,leg=True):
    plt.plot(X.numpy(),model(X).detach().numpy(),label='epoch '+str(epoch))
    plt.plot(X.numpy(),Y.numpy(),'r')
    plt.xlabel('x')
    if leg==True:
        plt.legend()
    else:
        pass


I am working on the file "DL0110EN/4.1.1simple1hiddenlayer .ipynb".
The first task is:

Neural Network Module and Training Function

Define the activations and the output of the first linear layer as an attribute. Note that this is not good practice.
Can you write Python code for it?

class Net(nn.Module):
    def __init__(self,D_in,H,D_out):
        super(Net,self).__init__()
        self.linear1=nn.Linear(D_in,H)
        self.linear2=nn.Linear(H,D_out)
        #define the first linear layer as an attribute, this is not good practice
        self.a1=None
        self.l1=None

    def forward(self,x):
        self.l1=self.linear1(x)
        self.a1=F.sigmoid(self.l1)
        yhat=F.sigmoid(self.linear2(self.a1))
        return yhat


I am working on the file "DL0110EN/4.1.1simple1hiddenlayer .ipynb".
The first task is:

Prepare Data

Can you write Python code for it?


I am working on the file "DL0110EN/4.1.1simple1hiddenlayer .ipynb".
The first task is:

Define the Neural Network, Criterion Function, Optimizer. and Train the  Model

Create the Cross-Entropy loss function:
Can you write Python code for it?

def criterion_cross(outputs,labels):
    out=-1*torch.mean(labels*torch.log(outputs) +(1-labels)*torch.log(1-outputs))
    return out


I am working on the file "DL0110EN/4.1.1simple1hiddenlayer .ipynb".
The first task is:
By examining the output of the  activation, you see by the 600th epoch that the data has been mapped to a linearly separable space.

Practice Question

Repeat the previous steps above by using the MSE cost or total loss:
Can you write Python code for it?

learning_rate=0.1
criterion_mse=nn.MSELoss()
model=Net(D_in,H,D_out)
optimizer=torch.optim.SGD(model.parameters(), lr=learning_rate)
cost_mse=train(Y,X,model,optimizer,criterion_mse,epochs=1000)
plt.plot(cost_mse)
plt.xlabel('epoch')
plt.title('MSE loss ')


I am working on the file "DL0110EN/4.1.1simple1hiddenlayer .ipynb".
The first task is:
Define the Neural Network, Optimizer, and Train the Model:
Can you write Python code for it?

D_in=1
H=2
D_out=1

learning_rate=0.1
model=Net(D_in,H,D_out)
optimizer=torch.optim.SGD(model.parameters(), lr=learning_rate)
cost_cross=train(Y,X,model,optimizer,criterion_cross,epochs=1000)
plt.plot(cost_cross)
plt.xlabel('epoch')
plt.title('cross entropy loss ')


I am working on the file "DL0110EN/4.1.1simple1hiddenlayer .ipynb".
The first task is:
Define the training function:
Can you write Python code for it?

def train(Y,X,model,optimizer,criterion,epochs=1000):
    error=[]

    for epoch in range(epochs):

        for y,x in zip(Y,X):
            yhat=model(x)
            loss=criterion(yhat,y)
            loss.backward()
            optimizer.step()
           #clear gradient
            optimizer.zero_grad()
        error.append(loss.item())
        if epoch % 300==0:
            PlotStuff(X,Y,model,epoch,leg=True)
            plt.show()
            model(X)
            plt.scatter( model.a1.detach().numpy()[:,0], model.a1.detach().numpy()[:,1],c=Y.numpy().reshape(-1))
            plt.title('activations')
            plt.show()
    return  error


I am working on the file "DL0110EN/4.1.2xor.ipynb".
The first task is:

 One Neuron

Create a neural network model with one neuron. Then, use the following code to train it:
Double-click here for the solution.

Can you write Python code for it?

learning_rate=0.001
criterion=nn.BCELoss()
optimizer=torch.optim.SGD(model.parameters(), lr=learning_rate)
train_loader=DataLoader(dataset=data_set,batch_size=1)
LOSS12=train(data_set,model,criterion, train_loader, optimizer, epochs=500)
plot_decision_regions_2class(model,data_set)


I am working on the file "DL0110EN/4.1.2xor.ipynb".
The first task is:
Dataset object:
Can you write Python code for it?


I am working on the file "DL0110EN/4.1.2xor.ipynb".
The first task is:
Define a function to train the model:
Can you write Python code for it?

def train(data_set,model,criterion, train_loader, optimizer, epochs=5):
    LOSS=[]
    ACC=[]
    for epoch in range(epochs):
        for x,y in train_loader:
            optimizer.zero_grad()

            yhat=model(x)
            loss=criterion(yhat,y)
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()

        ACC.append(accuracy(model,data_set))
        LOSS.append(loss)

    fig, ax1 = plt.subplots()
    color = 'tab:red'
    ax1.plot(LOSS,color=color)
    ax1.set_xlabel('epoch',color=color)
    ax1.set_ylabel('total loss',color=color)
    ax1.tick_params(axis='y', color=color)

    ax2 = ax1.twinx()
    color = 'tab:blue'
    ax2.set_ylabel('accuracy', color=color)  # we already handled the x-label with ax1
    ax2.plot( ACC, color=color)
    ax2.tick_params(axis='y', labelcolor=color)
    fig.tight_layout()  # otherwise the right y-label is slightly clipped

    plt.show()

    return LOSS


I am working on the file "DL0110EN/4.1.2xor.ipynb".
The first task is:
Use the following function to calculate accuracy:
Can you write Python code for it?

def accuracy(model,data_set):

    return np.mean(data_set.y.view(-1).numpy()==(model(data_set.x)[:,0]>0.5).numpy())


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:

Helper Functions

Use the following helper functions for plotting the loss:
Can you write Python code for it?

def plot_accuracy_loss(training_results):

    plt.subplot(2, 1, 1)

    plt.plot(training_results['training_loss'],'r')
    plt.ylabel('loss')
    plt.title('training loss iterations')

    plt.subplot(2,1,2)
    plt.plot( training_results['validation_accuracy'])

    plt.ylabel('accuracy')
    plt.xlabel('epochs ')
    plt.show()


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:

Neural Network Module and Training Function

Define the neural network module or class:
Can you write Python code for it?

class Net(nn.Module):
    def __init__(self,D_in,H,D_out):
        super(Net,self).__init__()
        self.linear1=nn.Linear(D_in,H)
        self.linear2=nn.Linear(H,D_out)

    def forward(self,x):
        x=torch.sigmoid(self.linear1(x))
        x=self.linear2(x)
        return x


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:

Prepare Data

Load the training dataset by setting the parameters train to True and convert it to a tensor by placing a transform object in the argument transform.
Can you write Python code for it?

train_dataset=dsets.MNIST(root='./data', train=True, download=True, transform=transforms.ToTensor())


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:

Analyze Results

Plot the training total loss or cost for every iteration and plot the training accuracy for every epoch:
Can you write Python code for it?


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:

Define the Neural Network, Criterion Function, Optimizer, and Train the Model

Create the criterion function:
Can you write Python code for it?


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:
Create the criterion function:
Can you write Python code for it?


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:
Create the model with 100 hidden layers:
Can you write Python code for it?

input_dim=28*28
hidden_dim=100
output_dim=10

model=Net(input_dim,hidden_dim,output_dim)


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:
Create the training-data loader and the validation-data loader objects:
Can you write Python code for it?


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:
Define a function to train the model. In this case, the function returns a Python dictionary to store the training loss and accuracy on the validation data.
Can you write Python code for it?

def train(model,criterion, train_loader,validation_loader, optimizer, epochs=100):
    i=0
    useful_stuff={'training_loss':[],'validation_accuracy':[]}

    #n_epochs
    for epoch in range(epochs):
        for i,(x, y) in enumerate(train_loader):

            #clear gradient
            optimizer.zero_grad()
            #make a prediction logits
            z=model(x.view(-1,28*28))
            # calculate loss
            loss=criterion(z,y)

            # calculate gradients of parameters
            loss.backward()
            # update parameters
            optimizer.step()
            useful_stuff['training_loss'].append(loss.data.item())
        correct=0
        for x, y in validation_loader:
            #perform a prediction on the validation  data
            yhat=model(x.view(-1,28*28))

            _,lable=torch.max(yhat,1)
            correct+=(lable==y).sum().item()

        accuracy=100*(correct/len(validation_dataset))

        useful_stuff['validation_accuracy'].append(accuracy)

    return useful_stuff


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:
Define the neural network module or class:
Can you write Python code for it?

def show_data(data_sample):

    plt.imshow(data_sample.numpy().reshape(28,28),cmap='gray')
    plt.show()


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:
Define the optimizer object with a learning rate of 0.01:
Can you write Python code for it?


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:
Load the testing dataset by setting the parameters train to False and convert it to a tensor by placing a transform object in the argument transform:
Can you write Python code for it?

validation_dataset=dsets.MNIST(root='./data', train=False, download=True, transform=transforms.ToTensor())


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:
Plot the first five misclassified samples:
Can you write Python code for it?

count=0
for x,y in validation_dataset:

    z=model(x.reshape(-1,28*28))
    _,yhat=torch.max(z,1)
    if yhat!=y:
        show_data(x)
        count+=1
    if count>=5:
        break


I am working on the file "DL0110EN/4.1.3.one_layer_neural_network_MNIST.ipynb".
The first task is:
Use the following function for printing the model parameters:
Can you write Python code for it?

def pint_model_parameters(model):
    for i in range(len(list(model.parameters()))):
        if (i+1)%2==0:
            print("the number of bias parameters for layer",i)
        else:
            print("the number of parameters for layer",i+1)

        print( list(model.parameters())[i].size() )


I am working on the file "DL0110EN/4.3.1lactivationfuction.ipynb".
The first task is:

 Logistic Function

Can you write Python code for it?


I am working on the file "DL0110EN/4.3.1lactivationfuction.ipynb".
The first task is:

 Compare Activation Functions

Can you write Python code for it?


I am working on the file "DL0110EN/4.3.1lactivationfuction.ipynb".
The first task is:

 Relu

When you use sequential, you can create a Relu object:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.1lactivationfuction.ipynb".
The first task is:
Apply the element-wise function Sigmoid with the object:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.1lactivationfuction.ipynb".
The first task is:
Call the object and plot it:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.1lactivationfuction.ipynb".
The first task is:
For custom modules, call the relu object from the nn.functional, which applies the element-wise sigmoid from the function module and plots the results:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.1lactivationfuction.ipynb".
The first task is:
For custom modules, call the sigmoid from the torch (nn.functional for the old version), which applies the element-wise sigmoid from the function module and plots the results:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.1lactivationfuction.ipynb".
The first task is:
Plot the results:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.1lactivationfuction.ipynb".
The first task is:
When you use sequential, you can create a sigmoid object:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.1lactivationfuction.ipynb".
The first task is:
w=torch.te

 Tanh

When you use sequential, you can create a tanh object:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.2mist1layer.ipynb".
The first task is:

Neural Network Module and Training Function

Define the neural network module or class:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.2mist1layer.ipynb".
The first task is:

Prepare Data

Load the training dataset by setting the parameters train to True and convert it to a tensor by placing a transform object in the argument transform.
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.2mist1layer.ipynb".
The first task is:

Define the Neural Network, Criterion Function, Optimizer, and Train the Model

Create the criterion function:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.2mist1layer.ipynb".
The first task is:
Compare the validation loss for each model:
Can you write Python code for it?

plt.plot(training_results_tanch['validation_accuracy'],label='tanh')
plt.plot(training_results['validation_accuracy'],label='sigmoid')
plt.plot(training_results_relu['validation_accuracy'],label='relu')
plt.ylabel('validation accuracy')
plt.xlabel('epochs ')
plt.legend()


I am working on the file "DL0110EN/4.3.2mist1layer.ipynb".
The first task is:
Create the criterion function:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.2mist1layer.ipynb".
The first task is:
Create the model with 100 hidden layers:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.2mist1layer.ipynb".
The first task is:
Define a function to train the model. In this case, the function returns a Python dictionary to store the training loss and accuracy on the validation data.
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.2mist1layer.ipynb".
The first task is:
Define the class for the Relu activation functiona:
Can you write Python code for it?

class NetRelu(nn.Module):
    def __init__(self,D_in,H,D_out):
        super(NetRelu,self).__init__()
        self.linear1=nn.Linear(D_in,H)
        self.linear2=nn.Linear(H,D_out)

    def forward(self,x):
        x=F.relu(self.linear1(x))
        x=self.linear2(x)
        return x


I am working on the file "DL0110EN/4.3.2mist1layer.ipynb".
The first task is:
Print the model parameters:

Test Sigmoid, Tanh, and Relu

Train the network by using the sigmoid activations function:
Can you write Python code for it?

model=Net(input_dim,hidden_dim,output_dim)

learning_rate=0.01
optimizer=torch.optim.SGD(model.parameters(),lr=learning_rate)
training_results=train(model,criterion, train_loader,validation_loader, optimizer, epochs=30)


I am working on the file "DL0110EN/4.3.2mist1layer.ipynb".
The first task is:
Train the network by using the Tanh activations function:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.3mist1layerassignmnt.ipynb".
The first task is:

Prepare Data

Load the training dataset by setting the parameter train to True and convert it to a tensor by placing a transform object in the argument transform:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.3mist1layerassignmnt.ipynb".
The first task is:

Criterion Function

Create the criterion function:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.3mist1layerassignmnt.ipynb".
The first task is:
Compare the validation loss for each model:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.3mist1layerassignmnt.ipynb".
The first task is:
Create the criterion function:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.3mist1layerassignmnt.ipynb".
The first task is:
Double-click here for the solution.

Train the network by using the Tanh activations function:
Use nn.Sequential to build a one hidden layer neural  model_Tanh network with a Tanh activation to classify the 10 digits from the MNIST dataset.
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.3mist1layerassignmnt.ipynb".
The first task is:
Load the testing dataset by setting the parameter train to False and convert it to a tensor by placing a transform object in the argument transform:
Can you write Python code for it?


I am working on the file "DL0110EN/4.3.3mist1layerassignmnt.ipynb".
The first task is:
Use nn.Sequential to build a one hidden layer neural  model network with a sigmoid activation to classify the 10 digits from the MNIST dataset.
Can you write Python code for it?

learning_rate=0.01
optimizer=torch.optim.SGD(model.parameters(),lr=learning_rate)
training_results=train(model,criterion, train_loader,validation_loader, optimizer, epochs=30)


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:

Test Sigmoid ,Tanh and Relu

train the network using the Sigmoid activations function
Can you write Python code for it?

model=Net(input_dim,hidden_dim1,hidden_dim2,output_dim)

learning_rate=0.01
optimizer=torch.optim.SGD(model.parameters(),lr=learning_rate)
training_results=train(model,criterion, train_loader,validation_loader, optimizer, epochs=cust_epochs)


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:

Analyse Results

compare the training loss for each activation
Can you write Python code for it?

plt.plot(training_results_tanch['training_loss'],label='tanh')
plt.plot(training_results['training_loss'],label='sim')
plt.plot(training_results_relu['training_loss'],label='relu')
plt.ylabel('loss')
plt.title('training loss iterations')
plt.legend()


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
Load the testing dataset by setting the parameters train  False and convert it to a tensor  by placing a transform object int the argument transform
Can you write Python code for it?


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
compare the validation loss for each model
Can you write Python code for it?


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
create  the model with 100 hidden layers
Can you write Python code for it?

input_dim=28*28
hidden_dim1=50
hidden_dim2=50
output_dim=10


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
create the criterion function
Can you write Python code for it?


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
create the training-data loader and the validation-data loader object
Can you write Python code for it?


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
define a function to  train the model, in this case the function returns a Python dictionary to store the training loss and accuracy on the validation data
Can you write Python code for it?


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
define the class for the Relu activation function
Can you write Python code for it?

class NetRelu(nn.Module):
    def __init__(self,D_in,H1,H2,D_out):
        super(NetRelu,self).__init__()
        self.linear1=nn.Linear(D_in,H1)
        self.linear2=nn.Linear(H1,H2)
        self.linear3=nn.Linear(H2,D_out)

    def forward(self,x):
        x=F.relu(self.linear1(x))
        x=F.relu(self.linear2(x))
        x=self.linear3(x)
        return x


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
define the class with the Tanh activation function
Can you write Python code for it?

class NetTanh(nn.Module):
    def __init__(self,D_in,H1,H2,D_out):
        super(NetTanh,self).__init__()
        self.linear1=nn.Linear(D_in,H1)
        self.linear2=nn.Linear(H1,H2)
        self.linear3=nn.Linear(H2,D_out)

    def forward(self,x):
        x=torch.tanh(self.linear1(x))
        x=torch.tanh(self.linear2(x))
        x=self.linear3(x)
        return x


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
print the model parameters
The epoch number in the video is 35. You can try 10 for now. If you try 35, it may take a long time.
Can you write Python code for it?


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
train the network using the Relu activations function
Can you write Python code for it?


I am working on the file "DL0110EN/4.4.1mist2layer.ipynb".
The first task is:
train the network using the Tanh activations function
Can you write Python code for it?


I am working on the file "DL0110EN/4.4.2mulitclassspiralrulu.ipynb".
The first task is:

 Helper Functions

Functions used to plot:
Can you write Python code for it?

def plot_decision_regions_3class(model,data_set):
    cmap_light = ListedColormap(['#FFAAAA', '#AAFFAA','#00AAFF'])
    cmap_bold = ListedColormap(['#FF0000', '#00FF00','#00AAFF'])
    X=data_set.x.numpy()
    y=data_set.y.numpy()
    h = .02
    x_min, x_max = X[:, 0].min()-0.1 , X[:, 0].max()+0.1
    y_min, y_max = X[:, 1].min()-0.1 , X[:, 1].max() +0.1
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),np.arange(y_min, y_max, h))
    XX=torch.torch.Tensor(np.c_[xx.ravel(), yy.ravel()])
    _,yhat=torch.max(model(XX),1)
    yhat=yhat.numpy().reshape(xx.shape)
    plt.pcolormesh(xx, yy, yhat, cmap=cmap_light)
    plt.plot(X[y[:]==0,0],X[y[:]==0,1],'ro',label='y=0')
    plt.plot(X[y[:]==1,0],X[y[:]==1,1],'go',label='y=1')
    plt.plot(X[y[:]==2,0],X[y[:]==2,1],'o',label='y=2')
    plt.title("decision region")
    plt.legend()


I am working on the file "DL0110EN/4.4.2mulitclassspiralrulu.ipynb".
The first task is:

Train and Validate the Model

Crate a dataset object:
Can you write Python code for it?


I am working on the file "DL0110EN/4.4.2mulitclassspiralrulu.ipynb".
The first task is:

Neural  Network Module and Function for Training

Neural Network Module using   ModuleList()
Can you write Python code for it?

class Net(nn.Module):
    def __init__(self,Layers):
        super(Net,self).__init__()
        self.hidden = nn.ModuleList()

        for input_size,output_size in zip(Layers,Layers[1:]):
            self.hidden.append(nn.Linear(input_size,output_size))

    def forward(self,activation):
        L=len(self.hidden)
        for (l,linear_transform)  in zip(range(L),self.hidden):
            if l<L-1:
                activation =F.relu(linear_transform (activation))

            else:
                activation =linear_transform (activation)

        return activation


I am working on the file "DL0110EN/4.4.2mulitclassspiralrulu.ipynb".
The first task is:
A function used to calculate accuracy
Can you write Python code for it?

def accuracy(model,data_set):
    _,yhat=torch.max(model(data_set.x),1)
    return (yhat==data_set.y).numpy().mean()


I am working on the file "DL0110EN/4.4.2mulitclassspiralrulu.ipynb".
The first task is:
A function used to train.
Can you write Python code for it?

def train(data_set,model,criterion, train_loader, optimizer, epochs=100):
    LOSS=[]
    ACC=[]
    for epoch in range(epochs):
        for x,y in train_loader:
            optimizer.zero_grad()

            yhat=model(x)
            loss=criterion(yhat,y)
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()

        LOSS.append(loss.item())
        ACC.append(accuracy(model,data_set))

    fig, ax1 = plt.subplots()
    color = 'tab:red'
    ax1.plot(LOSS,color=color)
    ax1.set_xlabel('epoch',color=color)
    ax1.set_ylabel('total loss',color=color)
    ax1.tick_params(axis='y', color=color)

    ax2 = ax1.twinx()
    color = 'tab:blue'
    ax2.set_ylabel('accuracy', color=color)  # we already handled the x-label with ax1
    ax2.plot( ACC, color=color)
    ax2.tick_params(axis='y', labelcolor=color)
    fig.tight_layout()  # otherwise the right y-label is slightly clipped

    plt.show()
    return LOSS


I am working on the file "DL0110EN/4.4.2mulitclassspiralrulu.ipynb".
The first task is:
Create a  network to classify three classes with 1 hidden layer with 50 neurons
Can you write Python code for it?

Layers=[2,50,3]
model=Net(Layers)
learning_rate=0.10

optimizer=torch.optim.SGD(model.parameters(), lr=learning_rate)
train_loader=DataLoader(dataset=data_set,batch_size=20)

criterion=nn.CrossEntropyLoss()
LOSS=train(data_set,model,criterion, train_loader, optimizer, epochs=100)

plot_decision_regions_3class(model,data_set)


I am working on the file "DL0110EN/4.4.2mulitclassspiralrulu.ipynb".
The first task is:
Create a  network to classify three classes with 2 hidden layer with 20 neurons
Can you write Python code for it?

Layers=[2,10,10,3]
model=Net(Layers)
learning_rate=0.01

optimizer=torch.optim.SGD(model.parameters(), lr=learning_rate)
train_loader=DataLoader(dataset=data_set,batch_size=20)

criterion=nn.CrossEntropyLoss()
LOSS=train(data_set,model,criterion, train_loader, optimizer, epochs=1000)
plot_decision_regions_3class(model,data_set)


I am working on the file "DL0110EN/5.1.1Xaviermist1layer.ipynb".
The first task is:

Neural Network Module and Training Function

define the neural network module or class Xavier Initialization
Can you write Python code for it?

class Net_Xavier(nn.Module):
    def __init__(self,Layers):
        super(Net_Xavier,self).__init__()
        self.hidden = nn.ModuleList()

        for input_size,output_size in zip(Layers,Layers[1:]):
            linear=nn.Linear(input_size,output_size)
            torch.nn.init.xavier_uniform_(linear.weight)
            self.hidden.append(nn.Linear(input_size,output_size))

    def forward(self,x):
        L=len(self.hidden)
        for (l,linear_transform)  in zip(range(L),self.hidden):
            if l<L-1:
                x =F.tanh(linear_transform (x))

            else:
                x =linear_transform (x)

        return x


I am working on the file "DL0110EN/5.1.1Xaviermist1layer.ipynb".
The first task is:

Prepare Data

Load the training dataset by setting the parameters train  to True and convert it to a tensor  by placing a transform object int the argument transform
Can you write Python code for it?

train_dataset=dsets.MNIST(root='./data', train=True, download=True, transform=transforms.ToTensor())


I am working on the file "DL0110EN/5.1.1Xaviermist1layer.ipynb".
The first task is:

Analyse Results

compare the training loss for each initialization
Can you write Python code for it?

plt.plot(training_results_Xavier['training_loss'],label='Xavier')
plt.plot(training_results['training_loss'],label='default ')
plt.plot(training_results_Uniform['training_loss'],label='Uniform')
plt.ylabel('loss')
plt.title('training loss iterations')
plt.legend()


I am working on the file "DL0110EN/5.1.1Xaviermist1layer.ipynb".
The first task is:
Class or Neral Network with Pytroch Default Initialization
Can you write Python code for it?

class Net(nn.Module):
    def __init__(self,Layers):
        super(Net,self).__init__()
        self.hidden = nn.ModuleList()

        for input_size,output_size in zip(Layers,Layers[1:]):
            linear=nn.Linear(input_size,output_size)

            #inear.weight.data.
            self.hidden.append(linear)

    def forward(self,x):
        L=len(self.hidden)
        for (l,linear_transform)  in zip(range(L),self.hidden):
            if l<L-1:
                x =F.tanh(linear_transform (x))

            else:
                x =linear_transform (x)

        return x


I am working on the file "DL0110EN/5.1.1Xaviermist1layer.ipynb".
The first task is:
Class or Neral Network with Uniform Initialization
Can you write Python code for it?

class Net_Uniform(nn.Module):
    def __init__(self,Layers):
        super(Net_Uniform,self).__init__()
        self.hidden = nn.ModuleList()

        for input_size,output_size in zip(Layers,Layers[1:]):
            linear=nn.Linear(input_size,output_size)
            linear.weight.data.uniform_(0, 1)
            #inear.weight.data.
            self.hidden.append(linear)

    def forward(self,x):
        L=len(self.hidden)
        for (l,linear_transform)  in zip(range(L),self.hidden):
            if l<L-1:
                x =F.tanh(linear_transform (x))

            else:
                x =linear_transform (x)

        return x


I am working on the file "DL0110EN/5.1.1Xaviermist1layer.ipynb".
The first task is:
compare the validation loss for each model
Can you write Python code for it?

plt.plot(training_results_Xavier['validation_accuracy'],label='Xavier')
plt.plot(training_results['validation_accuracy'],label='default ')
plt.plot(training_results_Uniform['validation_accuracy'],label='Uniform')
plt.ylabel('validation accuracy')
plt.xlabel('epochs ')
plt.legend()


I am working on the file "DL0110EN/5.1.1Xaviermist1layer.ipynb".
The first task is:
create  the model with 100 hidden layers
Can you write Python code for it?

input_dim=28*28

output_dim=10

layers=[input_dim,100,10,100,10,100,output_dim]
epochs=15


I am working on the file "DL0110EN/5.1.1Xaviermist1layer.ipynb".
The first task is:
define a function to  train the model, in this case the function returns a Python dictionary to store the training loss and accuracy on the validation data
Can you write Python code for it?

def train(model,criterion, train_loader,validation_loader, optimizer, epochs=100):
    i=0
    useful_stuff={'training_loss':[],'validation_accuracy':[]}

    #n_epochs
    for epoch in range(epochs):
        for i,(x, y) in enumerate(train_loader):

            #clear gradient
            optimizer.zero_grad()
            #make a prediction logits
            z=model(x.view(-1,28*28))
            # calculate loss
            loss=criterion(z,y)

            # calculate gradients of parameters
            loss.backward()
            # update parameters
            optimizer.step()
            useful_stuff['training_loss'].append(loss.data.item())
        correct=0
        for x, y in validation_loader:
            #perform a prediction on the validation  data
            yhat=model(x.view(-1,28*28))

            _,lable=torch.max(yhat,1)
            correct+=(lable==y).sum().item()

        accuracy=100*(correct/len(validation_dataset))

        useful_stuff['validation_accuracy'].append(accuracy)

    return useful_stuff


I am working on the file "DL0110EN/5.1.1Xaviermist1layer.ipynb".
The first task is:
print the model parameters

Test Pytroch Default Initialization,Xavier Initialization,Uniform Initialization

train the network using Pytroch Default Initialization
Can you write Python code for it?

model=Net(layers)

learning_rate=0.01
optimizer=torch.optim.SGD(model.parameters(),lr=learning_rate)
training_results=train(model,criterion, train_loader,validation_loader, optimizer, epochs=epochs)


I am working on the file "DL0110EN/5.1.1Xaviermist1layer.ipynb".
The first task is:
train the network using Uniform Initialization
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:

Create the Model, Optimizer, and Total Loss Function (cost)

Create a custom module with three layers. in_size is the size of the input features, n_hidden is the size of the layers, and out_size is the size. p is the dropout probability. The default is 0, that is, no dropout.
Can you write Python code for it?

class Net(nn.Module):
    def __init__(self,in_size,n_hidden,out_size,p=0):

        super(Net,self).__init__()
        self.drop=nn.Dropout(p=p)
        self.linear1=nn.Linear(in_size,n_hidden)
        self.linear2=nn.Linear(n_hidden,n_hidden)
        self.linear3=nn.Linear(n_hidden,out_size)

    def forward(self,x):
        x=F.relu(self.linear1(x))
        x=self.drop(x)
        x=F.relu(self.linear2(x))
        x=self.drop(x)
        x=self.linear3(x)
        return x


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:

Train the Model via Mini-Batch Gradient Descent

Set the model using dropout to training mode; this is the default mode, but it's a good practice:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
Create a dataset object:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
Create two model objects: model had no dropout and model_drop has a dropout probability of 0.5:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
Get some validation data:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
Initialize a dictionary that stores the training and validation loss for each model:
Can you write Python code for it?

LOSS={}
LOSS['training data no dropout']=[]
LOSS['validation data no dropout']=[]
LOSS['training data dropout']=[]
LOSS['validation data dropout']=[]


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
Test the model with dropout on the validation data:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
Train the model by using the Adam optimizer. See the unit on other optimizers. Use the Cross Entropy Loss:
Can you write Python code for it?

optimizer_ofit = torch.optim.Adam(model.parameters(), lr=0.01)
optimizer_drop = torch.optim.Adam(model_drop.parameters(), lr=0.01)
criterion = torch.nn.CrossEntropyLoss()


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
Use this function only for plotting:
Can you write Python code for it?

def plot_decision_regions_3class(data_set,model=None):

    cmap_light = ListedColormap(['#FFAAAA', '#AAFFAA','#00AAFF'])
    cmap_bold = ListedColormap(['#FF0000', '#00FF00','#00AAFF'])
    X=data_set.x.numpy()
    y=data_set.y.numpy()
    h = .02
    x_min, x_max = X[:, 0].min()-0.1 , X[:, 0].max()+0.1
    y_min, y_max = X[:, 1].min()-0.1 , X[:, 1].max() +0.1
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),np.arange(y_min, y_max, h))
    newdata=np.c_[xx.ravel(), yy.ravel()]

    #XX=torch.torch.Tensor(newdata)
    #_,yhat=torch.max(model(XX),1)
    #yhat=yhat.numpy().reshape(xx.shape)

    Z=data_set.fun(newdata).flatten()
    f=np.zeros(Z.shape)
    f[Z>0]=1
    f=f.reshape(xx.shape)
    if model!=None:
        model.eval()
        XX=torch.torch.Tensor(newdata)
        _,yhat=torch.max(model(XX),1)
        yhat=yhat.numpy().reshape(xx.shape)
        plt.pcolormesh(xx, yy, yhat, cmap=cmap_light)
        plt.contour(xx, yy, f, cmap=plt.cm.Paired)
    else:
        plt.contour(xx, yy, f, cmap=plt.cm.Paired)
        plt.pcolormesh(xx, yy, f, cmap=cmap_light)

    plt.title("decision region vs True decision boundary")
    plt.legend()


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
Use this function to calculate accuracy:
Can you write Python code for it?

def accuracy(model,data_set):
    _,yhat=torch.max(model(data_set.x),1)
    return (yhat==data_set.y).numpy().mean()


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
You can see that the model using dropout does better at tracking the function that generated the data.
Plot out the loss for the training and validation data on both models:
Can you write Python code for it?

plt.figure(figsize=(6.1, 10))
for key, value in LOSS.items():
    plt.plot(np.log(np.array(value)),label=key)
    plt.legend()
    plt.xlabel("iterations")
    plt.ylabel("Log of cost or total loss")


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
You see that the model with dropout performs better on the validation data.
Plot the decision boundary and the prediction of the networks in different colors:
true function
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
model with dropout
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutPredictin.ipynb".
The first task is:
model without dropout
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutRegression.ipynb".
The first task is:

Train the Model via Mini-Batch Gradient Descent

Set the model using dropout to training mode; this is the default mode, but it's a good practice.
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutRegression.ipynb".
The first task is:
Create a dataset object:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutRegression.ipynb".
The first task is:
Create two model objects: model had no dropout, and model_drop has a dropout probability of 0.5:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutRegression.ipynb".
The first task is:
Get some validation data:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutRegression.ipynb".
The first task is:
Make a prediction by using both models:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutRegression.ipynb".
The first task is:
Set the model with dropout to evaluation mode:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.1dropoutRegression.ipynb".
The first task is:
Train the model by using the Adam optimizer. See the unit on other optimizers. Use the mean square loss:
Can you write Python code for it?

optimizer_ofit = torch.optim.Adam(model.parameters(), lr=0.01)
optimizer_drop = torch.optim.Adam(model_drop.parameters(), lr=0.01)
criterion = torch.nn.MSELoss()


I am working on the file "DL0110EN/5.1.1dropoutRegression.ipynb".
The first task is:
You can see that the model using dropout does better at tracking the function that generated the data.
Plot out the loss for training and validation data on both models:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2.He_Initialization.ipynb".
The first task is:

Neural Network Module and Training Function

define the neural network module or class He Initialization
Can you write Python code for it?

class Net_He(nn.Module):
    def __init__(self,Layers):
        super(Net_He,self).__init__()
        self.hidden = nn.ModuleList()

        for input_size,output_size in zip(Layers,Layers[1:]):
            linear=nn.Linear(input_size,output_size)
            torch.nn.init.kaiming_uniform_(linear.weight,nonlinearity='relu')
            self.hidden.append(nn.Linear(input_size,output_size))

    def forward(self,x):
        L=len(self.hidden)
        for (l,linear_transform)  in zip(range(L),self.hidden):
            if l<L-1:
                x =F.relu(linear_transform (x))

            else:
                x =linear_transform (x)

        return x


I am working on the file "DL0110EN/5.1.2.He_Initialization.ipynb".
The first task is:
Class or Neral Network with Pytroch Default Initialization
Can you write Python code for it?

class Net(nn.Module):
    def __init__(self,Layers):
        super(Net,self).__init__()
        self.hidden = nn.ModuleList()

        for input_size,output_size in zip(Layers,Layers[1:]):
            linear=nn.Linear(input_size,output_size)

            #inear.weight.data.
            self.hidden.append(linear)

    def forward(self,x):
        L=len(self.hidden)
        for (l,linear_transform)  in zip(range(L),self.hidden):
            if l<L-1:
                x =F.relu(linear_transform (x))

            else:
                x =linear_transform (x)

        return x


I am working on the file "DL0110EN/5.1.2.He_Initialization.ipynb".
The first task is:
Load the testing dataset by setting the parameters train  False and convert it to a tensor  by placing a transform object int the argument transform
Can you write Python code for it?

validation_dataset=dsets.MNIST(root='./data', train=False, download=True, transform=transforms.ToTensor())


I am working on the file "DL0110EN/5.1.2.He_Initialization.ipynb".
The first task is:
compare the validation loss for each model
Can you write Python code for it?

plt.plot(training_results_He['validation_accuracy'],label='He')
plt.plot(training_results['validation_accuracy'],label='default ')
plt.plot(training_results_Uniform['validation_accuracy'],label='Uniform')
plt.ylabel('validation accuracy')
plt.xlabel('epochs ')
plt.legend()


I am working on the file "DL0110EN/5.1.2.He_Initialization.ipynb".
The first task is:
create a list that contains layer  size
Can you write Python code for it?

input_dim=28*28

output_dim=10

layers=[input_dim,100,200,100,output_dim]


I am working on the file "DL0110EN/5.1.2.He_Initialization.ipynb".
The first task is:
create the criterion function
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2.He_Initialization.ipynb".
The first task is:
define a function to  train the model, in this case the function returns a Python dictionary to store the training loss and accuracy on the validation data
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2.He_Initialization.ipynb".
The first task is:
print the model parameters

Test Pytroch Default Initialization,Xavier Initialization,Uniform Initialization

train the network using Pytroch Default Initialization
Can you write Python code for it?

model=Net(layers)

learning_rate=0.01
optimizer=torch.optim.SGD(model.parameters(),lr=learning_rate)
training_results=train(model,criterion, train_loader,validation_loader, optimizer, epochs=30)


I am working on the file "DL0110EN/5.1.2.He_Initialization.ipynb".
The first task is:
train the network using the Relu activations function
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2.dropoutAssignment.ipynb".
The first task is:

Get Some Data

Create a nonlinearly separable dataset:
Can you write Python code for it?

from torch.utils.data import Dataset, DataLoader

class Data(Dataset):
    def __init__(self,N_SAMPLES = 1000,noise_std=0.1,train=True):

        a=np.matrix([-1,1,2,1,1,-3,1]).T

        self.x = np.matrix(np.random.rand(N_SAMPLES,2))

        self.f=np.array(a[0]+(self.x)*a[1:3]+np.multiply(self.x[:,0], self.x[:,1])*a[4]+np.multiply(self.x, self.x)*a[5:7]).flatten()
        self.a=a

        self.y=np.zeros(N_SAMPLES)
        self.y[self.f> 0]=1
        self.y=torch.from_numpy(self.y).type(torch.LongTensor)
        self.x=torch.from_numpy(self.x).type(torch.FloatTensor)
        self.x = self.x+noise_std*torch.randn(self.x.size())
        self.f=torch.from_numpy(self.f)
        self.a=a
        if train==True:
            torch.manual_seed(1)

            self.x = self.x+noise_std*torch.randn(self.x.size())
            torch.manual_seed(0)

    def __getitem__(self,index):
        return self.x[index],self.y[index]
    def __len__(self):
        return self.len
    def plot(self):
        X=data_set.x.numpy()
        y=data_set.y.numpy()
        h = .02
        x_min, x_max = X[:, 0].min() , X[:, 0].max()
        y_min, y_max = X[:, 1].min(), X[:, 1].max()
        xx, yy = np.meshgrid(np.arange(x_min, x_max, h),np.arange(y_min, y_max, h))
        Z=data_set.fun(np.c_[xx.ravel(), yy.ravel()]).flatten()
        f=np.zeros(Z.shape)
        f[Z>0]=1
        f=f.reshape(xx.shape)
        plt.title('True decision boundary  and sample points with noise ')

        plt.plot(self.x[self.y==0,0].numpy(),self.x[self.y==0,1].numpy(),'bo',label='y=0' )
        plt.plot(self.x[self.y==1,0].numpy(), self.x[self.y==1,1].numpy(),'ro',label='y=1' )
        plt.contour(xx, yy, f, cmap=plt.cm.Paired)
        plt.xlim(0,1)
        plt.ylim(0,1)
        plt.legend()
    def fun(self,x):

        x=np.matrix(x)

        out=np.array(self.a[0]+(x)*self.a[1:3]+np.multiply(x[:,0], x[:,1])*self.a[4]+np.multiply(x, x)*self.a[5:7])
        out=np.array(out)
       ...


I am working on the file "DL0110EN/5.1.2.dropoutAssignment.ipynb".
The first task is:

Create the Model, Optimizer, and Total Loss Function (cost)

Create a three-layer neural network model with a ReLU() activation function for regression. All the appropriate layers should be 300 units.
Double-click here for the solution.

Create a three-layer neural network model_drop with a ReLU() activation function for regression. All the appropriate layers should be 300 units. Apply dropout to all but the last layer and make the probability of dropout is 50%.
Double-click here for the solution.

Train the Model via Mini-Batch Gradient Descent

Set the model by using dropout to training mode; this is the default mode, but it's a good practice.
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2.dropoutAssignment.ipynb".
The first task is:
Get some validation data:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2.dropoutAssignment.ipynb".
The first task is:
You can see that the model using dropout does better at tracking the function that generated the data.
Plot out the loss for training and validation data on both models:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2.dropoutAssignment.ipynb".
The first task is:
model with dropout
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2dropoutRegressionAssignemnt.ipynb".
The first task is:

Get Some Data

Create polynomial dataset objects:
Can you write Python code for it?

from torch.utils.data import Dataset, DataLoader

class Data(Dataset):
    def __init__(self,N_SAMPLES = 40,noise_std=1,train=True):

        self.x = torch.linspace(-1, 1, N_SAMPLES).view(-1,1)
        self.f=self.x**2

        if train!=True:
            torch.manual_seed(1)

            self.y = self.f+noise_std*torch.randn(self.f.size())
            self.y=self.y.view(-1,1)
            torch.manual_seed(0)
        else:
            self.y = self.f+noise_std*torch.randn(self.f.size())
            self.y=self.y.view(-1,1)
    def __getitem__(self,index):
        return self.x[index],self.y[index]
    def __len__(self):
        return self.len
    def plot(self):
        plt.figure(figsize=(6.1, 10))
        plt.scatter(self.x.numpy(), self.y.numpy(), label="Samples")
        plt.plot(self.x.numpy(), self.f.numpy()  ,label="True function",color='orange')
        plt.xlabel("x")
        plt.ylabel("y")
        plt.xlim((-1, 1))
        plt.ylim((-2, 2.5))
        plt.legend(loc="best")
        plt.show()


I am working on the file "DL0110EN/5.1.2dropoutRegressionAssignemnt.ipynb".
The first task is:
Create a dataset object:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2dropoutRegressionAssignemnt.ipynb".
The first task is:
Create a three-layer neural network model with a ReLU() activation function for regression. All the appropriate layers should be 300 units.
Double-click here for the solution.

Create a three-layer neural network model_drop with a ReLU() activation function for regression. All the appropriate layers should be 300 units. Apply dropout to all but the last layer and make the probability of dropout is 50%.
Double-click here for the solution.

Train the Model via Mini-Batch Gradient Descent

Set the model using dropout to training mode; this is the default mode, but it's a good practice.
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2dropoutRegressionAssignemnt.ipynb".
The first task is:
Train the model by using the Adam optimizer. See the unit on other optimizers. Use the mean square loss:
Can you write Python code for it?


I am working on the file "DL0110EN/5.1.2dropoutRegressionAssignemnt.ipynb".
The first task is:
You can see that the model using dropout does better at tracking the function that generated the data.
Plot out the loss for training and validation data on both models:
Can you write Python code for it?


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:

Local Minima

In this section, we will create a fourth order polynomial with a local minimum at 4 and a global minimum a -2. We will then see how the momentum parameter affects convergence to a global minimum. The fourth order polynomial is given by:
Can you write Python code for it?

# Create a function to calculate the fourth order polynomial

def fourth_order(yhat):
    out = torch.mean(2 * (yhat ** 4) - 9 * (yhat ** 3) - 21 * (yhat ** 2) + 88 * yhat + 48)
    return out


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:

Noise

In this section, we will create a fourth order polynomial with a local minimum at 4 and a global minimum a -2, but we will add noise to the function when the Gradient is calculated. We will then see how the momentum parameter affects convergence to a global minimum.
with no momentum, we get stuck in a local minimum
Can you write Python code for it?

# Make the prediction without momentum when there is noise

optimizer = torch.optim.SGD(w.parameters(), lr = 0.001)
plot_fourth_order(w, optimizer, std = 10)


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:

Saddle Points

Let's create a cubic function with Saddle points
Can you write Python code for it?

# Define a function to output a cubic

def cubic(yhat):
    out = yhat ** 3
    return out


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:

Practice

Create two  SGD  objects with a learning rate of  0.001. Use the default momentum parameter value  for one and a value of  0.9 for the second. Use the function plot_fourth_order with an std=100, to plot the different steps of each. Make sure you run the function on two independent cells.
Can you write Python code for it?

# Practice: Create two SGD optimizer with lr = 0.001, and one without momentum and the other with momentum = 0.9. Plot the result out.

# Type your code here


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:
This function will plot a 4th order function and the parameter values obtained via Gradient Descent. You can also add Gaussian noise with a standard deviation determined by the parameter std.
Can you write Python code for it?

# Plot the fourth order function and the parameter values

def plot_fourth_order(w, optimizer, std = 0, color = 'r', paramlabel = 'parameter values', objfun = True):
    W = torch.arange(-4, 6, 0.1)
    LOSS = []
    for w.state_dict()['linear.weight'][0] in W:
        LOSS.append(fourth_order(w(torch.tensor([[1.0]]))).item())
    w.state_dict()['linear.weight'][0] = 6
    n_epochs = 100
    parameter = []
    loss_list = []

    #n_epochs
    for n in range(n_epochs):
        optimizer.zero_grad()
        loss = fourth_order(w(torch.tensor([[1.0]]))) + std * torch.randn(1, 1)
        loss_list.append(loss)
        parameter.append(w.state_dict()['linear.weight'][0].detach().data.item())
        loss.backward()
        optimizer.step()

    # Plotting
    if objfun:
        plt.plot(W.numpy(), LOSS, label = 'objective function')
    plt.plot(parameter, loss_list, 'ro',label = paramlabel, color = color)
    plt.xlabel('w')
    plt.ylabel('l(w)')
    plt.legend()


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:
This function will plot a cubic function and the parameter values obtained via Gradient Descent.
Can you write Python code for it?

# Plot the cubic

def plot_cubic(w, optimizer):
    LOSS = []
    W = torch.arange(-4, 4, 0.1)
    for w.state_dict()['linear.weight'][0] in W:
        LOSS.append(cubic(w(torch.tensor([[1.0]]))).item())
    w.state_dict()['linear.weight'][0] = 4.0
    n_epochs = 10
    parameter = []
    loss_list = []

    # n_epochs
    # Use PyTorch custom module to implement a ploynomial function
    for n in range(n_epochs):
        optimizer.zero_grad()
        loss = cubic(w(torch.tensor([[1.0]])))
        loss_list.append(loss)
        parameter.append(w.state_dict()['linear.weight'][0].detach().data.item())
        loss.backward()
        optimizer.step()
    plt.plot(parameter, loss_list, 'ro', label = 'parameter values')
    plt.plot(W.numpy(), LOSS, label = 'objective function')
    plt.xlabel('w')
    plt.ylabel('l(w)')
    plt.legend()


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:
We create an object w, when we call the object with an input of one, it will behave like an individual parameter value. i.e w(1) is analogous to $w$
Can you write Python code for it?

# Create a one_param object

w = one_param(1, 1)


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:
We create an optimizer with no momentum term
Can you write Python code for it?

# Create a optimizer without momentum

optimizer = torch.optim.SGD(w.parameters(), lr = 0.01, momentum = 0)


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:
We run several iterations of stochastic gradient descent and plot the results. We see the parameter values get stuck in the saddle point.
Can you write Python code for it?

# Plot the model

plot_cubic(w, optimizer)


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:
we create an optimizer with momentum term of 0.9
Can you write Python code for it?

# Create a optimizer with momentum

optimizer = torch.optim.SGD(w.parameters(), lr = 0.01, momentum = 0.90)


I am working on the file "DL0110EN/5.4.1_MomentumwithPolynomialFunctions.ipynb".
The first task is:
with  momentum, we get to the global  minimum
Can you write Python code for it?

# Make the prediction with momentum when there is noise

optimizer = torch.optim.SGD(w.parameters(), lr = 0.001,momentum = 0.9)
plot_fourth_order(w, optimizer, std = 10)


I am working on the file "DL0110EN/5.4.2High Condition Number.ipynb".
The first task is:

Train the Model via Stochastic Gradient Descent

Run 1 epochs of stochastic gradient descent and view parameter space.
Can you write Python code for it?

# Train the model

def train_model(epochs = 1):
    for epoch in range(epochs):
        for i, (x, y) in enumerate(trainloader):
            #no momentum
            yhat = model(x)
            loss = criterion(yhat, y)

            #momentum
            yhat_m = model_momentum(x)
            loss_m = criterion(yhat_m, y)

            #apply optimization to momentum term and term without momentum

            #for plotting
            #get_surface.get_stuff(model, loss.tolist())
            #get_surface.get_stuff1(model_momentum, loss_m.tolist())

            get_surface.set_para_loss(model = model_momentum, name = "momentum" ,loss = loss_m.tolist())
            get_surface.set_para_loss(model = model, name = "no momentum" , loss = loss.tolist())

            optimizer.zero_grad()
            optimizer_momentum.zero_grad()
            loss.backward()
            loss_m.backward()
            optimizer.step()
            optimizer_momentum.step()
        get_surface.plot_ps(iteration = i)
train_model()


I am working on the file "DL0110EN/5.4.2High Condition Number.ipynb".
The first task is:

Make Some Data

Generate values from -2 to 2 that create a line with a slope of 0.1 and a bias of 10000. This is the line that you need to estimate. Add some noise to the data:
Can you write Python code for it?

# Define a class to create the dataset

class Data(Dataset):

    # Constructor
    def __init__(self):
        self.x = torch.arange(-2, 2, 0.1).view(-1, 1)
        self.f = 1 * self.x + 10000
        self.y = self.f + 0.1 * torch.randn(self.x.size())
        self.len = self.x.shape[0]

    # Getter
    def __getitem__(self, index):
        return self.x[index], self.y[index]

    # Get Length
    def __len__(self):
        return self.len


I am working on the file "DL0110EN/5.4.2High Condition Number.ipynb".
The first task is:

Create the Model and Total Loss Function (Cost)

Create a linear regression class
Can you write Python code for it?

# Define linear regression class

class linear_regression(nn.Module):

    # Constructor
    def __init__(self, input_size, output_size):
        super(linear_regression, self).__init__()
        self.linear = nn.Linear(input_size, output_size)

    # Prediction
    def forward(self, x):
        yhat = self.linear(x)
        return yhat


I am working on the file "DL0110EN/5.4.2High Condition Number.ipynb".
The first task is:
Create a dataloader object:
Can you write Python code for it?

# Create a data loader

trainloader = DataLoader(dataset = dataset, batch_size = 1, shuffle = True)


I am working on the file "DL0110EN/5.4.2High Condition Number.ipynb".
The first task is:
Create a linear regression object, and an SGD optimiser object with momentum .
Can you write Python code for it?

# Create a linear regression object and the optimizer with momentum

model_momentum = linear_regression(1, 1)
optimizer_momentum = optim.SGD(model_momentum.parameters(), lr = 0.01, momentum = 0.2)


I am working on the file "DL0110EN/5.4.2High Condition Number.ipynb".
The first task is:
Create a linear regression object, and an SGD optimizer object with no momentum.
Can you write Python code for it?

# Create a linear regression object and the optimizer without momentum

model = linear_regression(1, 1)
optimizer = optim.SGD(model.parameters(), lr = 0.01)


I am working on the file "DL0110EN/5.4.2High Condition Number.ipynb".
The first task is:
The class plot_error_surfaces is just to help you visualize the data space and the parameter space during training and has nothing to do with Pytorch.
Can you write Python code for it?

# Define the class for plot out the surface

class plot_error_surfaces(object):

    # Constructor
    def __init__(self, w_range, b_range, X, Y, n_samples = 30, go = True):
        W = np.linspace(-w_range, w_range, n_samples)
        B = np.linspace(-b_range, b_range, n_samples)
        w, b = np.meshgrid(W, B)
        Z = np.zeros((n_samples, n_samples))
        count1 = 0
        self.y = Y.numpy()
        self.x = X.numpy()
        for w1, b1 in zip(w, b):
            count2 = 0
            for w2, b2 in zip(w1, b1):
                Z[count1, count2] = np.mean((self.y - w2 * self.x + b2) ** 2)
                count2 += 1
            count1 += 1
        self.Z = Z
        self.w = w
        self.b = b
        self.LOSS_list = {}

    # Setter
    def set_para_loss(self, model, name, loss):
        if (not (name in self.LOSS_list)):
            self.LOSS_list[name] = []
        w = list(model.parameters())[0].item()
        b = list(model.parameters())[1].item()
        self.LOSS_list[name].append({"loss": loss, "w": w, "b": b})

    # Plot the diagram
    def plot_ps(self, iteration = 0):
        plt.contour(self.w, self.b, self.Z)
        count = 1
        if (len(self.LOSS_list) > 0):
            for key, value in self.LOSS_list.items():
                w = [v for d in value for (k, v) in d.items() if "w" == k]
                b = [v for d in value for (k, v) in d.items() if "b" == k]
                plt.scatter(w, b, cmap = 'viridis', marker = 'x', label = key)
            plt.title('Loss Surface Contour not to scale, Iteration: ' + str(iteration))
            plt.legend()
        plt.xlabel('w')
        plt.ylabel('b')
        plt.show()


I am working on the file "DL0110EN/5.4.2High Condition Number.ipynb".
The first task is:
The plot above shows the different parameter values for each model in different iterations of SGD. The values are overlaid over the cost or total loss surface. The contour lines somewhat miss scaled but it is evident that in the vertical direction they are much closer together implying a larger gradient in that direction. The model trained with momentum shows somewhat more displacement in the hozontal direction.
The plot below shows the log of the cost or total loss, we see that the term with momentum converges to a minimum faster and to an overall smaller value.
Can you write Python code for it?

# Plot the loss

loss = [v for d in get_surface.LOSS_list["no momentum"] for (k, v) in d.items() if "loss" == k]
loss_m = [v for d in get_surface.LOSS_list["momentum"] for (k, v) in d.items() if "loss" == k]
plt.plot(np.log(loss), 'r', label = 'no momentum' )
plt.plot(np.log(loss_m), 'b', label = 'momentum' )
plt.title('Cost or Total Loss' )
plt.xlabel('Iterations ')
plt.ylabel('Cost')
plt.legend()
plt.show()