I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Extra credit -- Activation functions in numpy:
Can you write Python code for it?

import numpy as np

def sigmoid(x, derivative=False):
    if (derivative == True):
        return x * (1 - x)
    return 1 / (1 + np.exp(-x))

def tanh(x, derivative=False):
    if (derivative == True):
        return (1 - (x ** 2))
    return np.tanh(x)

def relu(x, derivative=False):
    if (derivative == True):
        for i in range(0, len(x)):
            for k in range(len(x[i])):
                if x[i][k] > 0:
                    x[i][k] = 1
                else:
                    x[i][k] = 0
        return x
    for i in range(0, len(x)):
        for k in range(0, len(x[i])):
            if x[i][k] > 0:
                pass  # do nothing since it would be effectively replacing x with x
            else:
                x[i][k] = 0
    return x

def arctan(x, derivative=False):
    if (derivative == True):
        return (np.cos(x) ** 2)
    return np.arctan(x)

def step(x, derivative=False):
    if (derivative == True):
        for i in range(0, len(x)):
            for k in range(len(x[i])):
                if x[i][k] > 0:
                    x[i][k] = 0
        return x
    for i in range(0, len(x)):
        for k in range(0, len(x[i])):
            if x[i][k] > 0:
                x[i][k] = 1
            else:
                x[i][k] = 0
    return x

def squash(x, derivative=False):
    if (derivative == True):
        for i in range(0, len(x)):
            for k in range(0, len(x[i])):
                if x[i][k] > 0:
                    x[i][k] = (x[i][k]) / (1 + x[i][k])
                else:
                    x[i][k] = (x[i][k]) / (1 - x[i][k])
        return x
    for i in range(0, len(x)):
        for k in range(0, len(x[i])):
            x[i][k] = (x[i][k]) / (1 + abs(x[i][k]))
    return x

def gaussian(x, derivative=False):
    if (derivative == True):
        for i in range(0, len(x)):
            for k in range(0, len(x[i])):
                x[i][k] = -2* x[i][k] * np.exp(-x[i][k] ** 2)
    for i in range(0, len(x)):
        for k in range(0, len(x[i])):
            x[i][k] =...


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Function that predicts the output of either 0 or 1:
Can you write Python code for it?

def predict(model, x):
    W1, b1, W2, b2 = model['W1'], \
                     model['b1'], \
                     model['W2'], \
                     model['b2']
    # Design a network with forward propagation
    z1 = x.dot(W1) + b1
    a1 = np.tanh(z1)
    z2 = a1.dot(W2) + b2
    exp_scores = np.exp(z2)
    probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
    return np.argmax(probs, axis=1)


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Generate a dataset and create a plot:
Can you write Python code for it?

np.random.seed(0)
X, y = sklearn.datasets.make_moons(200, noise=0.20)
plt.scatter(X[:,0], X[:,1], s=40, c=y, cmap=plt.cm.Spectral)


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Gradient descent optimizer:

Imagine the valley with a ball.
The goal of the optimizer is to localize the ball to the lowest point in the valley.
Loss function will be reduced over the training.
Mean squared error as the loss function.
Can you write Python code for it?

lr = 0.01

loss = tf.reduce_mean(tf.square(y - y_data))
optimizer = tf.train.GradientDescentOptimizer(lr)


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Gradient descent parameters:
Can you write Python code for it?

epsilon = 0.01 # learning rate for gradient descent
reg_lambda = 0.01 # regularization strength


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Load model weights
Can you write Python code for it?

if os.path.exists(MODEL_PATH):
  model.load_weights(MODEL_PATH)


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Part 01a -- Simple neural network as XOR gate using sigmoid activation function:
The XOR gate neural network implemention uses a two layer perceptron with sigmoid activation function. This portion of the notebook is a modified fork of the neural network implementation in numpy by Milo Harper.
Import the dependent libraries -- numpy and matplotlib:
Can you write Python code for it?

import numpy as np
import matplotlib.pyplot as plt


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Part 01b -- Backpropagation
This backpropagation example implments the logistic function:
$ \frac{\partial E}{\partial o_j} = \frac{\partial E}{\partial o_j} = \frac{\partial }{\partial o_j} {\frac{1}{2}}(t-y)^2 {= y-t}$
and computes layer delta using:
$\Delta w_{ij} = -\eta \frac{\partial E}{\partial w_{ij}} $
In this implementation, learning rate ($\eta$) = 1
Read more by following the backpropogation link above.
Implement a signle backprop pass
Can you write Python code for it?

outputLoss_derivative = (layer_2 - y)

outputLoss_derivative

layer_2_delta = (outputLoss_derivative*sigmoid(layer_2,derivative=True))

layer_2_delta

layer_1_error = (layer_2_delta.dot(synapse_1.T))

layer_1_error

layer_1_delta=layer_1_error*sigmoid(layer_1,derivative=True)

layer_1_delta


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Part 01b -- Neural network based XOR gate using rectified linear units activation function:
Can you write Python code for it?


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Part 03 -- Example illustrating the importance of learning rate in hyper-parameter tuning:

Learning rate is a decreasing function of time.
Two forms that are commonly used are:
1) a linear function of time
2) a function that is inversely proportional to the time t

Create a noisier, more complex dataset:
Can you write Python code for it?

np.random.seed(0)
X, y = sklearn.datasets.make_moons(20000, noise=0.5)
plt.scatter(X[:,0], X[:,1], s=40, c=y, cmap=plt.cm.Spectral)

linear_classifier = sklearn.linear_model.LogisticRegressionCV()
linear_classifier.fit(X, y)

plot_decision_boundary(lambda x: linear_classifier.predict(x))
plt.title("Logistic Regression")

num_examples = len(X) # training set size
nn_input_dim = 2 # input layer dimensionality
nn_output_dim = 2 # output layer dimensionality

epsilon = 0.01 # learning rate for gradient descent
reg_lambda = 0.01 # regularization strength

model = build_model(50, print_loss=True)


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Part 04 -- Building a neural network using tensorflow:

A neural network that predicts the y value given an x value.
Implemented using tensorflow, an open-source deep-learning library.

Import dependent libraries:
Can you write Python code for it?

import tensorflow as tf
import numpy as np


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Part 05 -- Neural net XOR gate solver using Tensorflow and Keras
Can you write Python code for it?

import keras
import numpy as np
import os

from keras import Sequential
from keras.layers import Dense
from keras.optimizers import SGD

MODEL_PATH = './XOR_gate_keras_network.h5'

! wget https://github.com/rahulremanan/python_tutorial/raw/master/Fundamentals_of_deep-learning/weights/XOR_gate_keras_network.h5 -O XOR_gate_keras_network.h5


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Plotting hyperbolic tan (tanh) activation function:
Can you write Python code for it?

def tanh(x, derivative=False):
    if (derivative == True):
        return (1 - (x ** 2))
    return np.tanh(x)

x = list(np.arange(-6.0, 6.0, 0.1))
y = []
for i in x:
  y_i = tanh(i)
  y.append(y_i)

xmin=-6
xmax = 6
ymin = -1.1
ymax = 1.1
axis = [xmin, xmax, ymin, ymax]
plt.axhline(y=0, color='C2', alpha=0.5)
plt.axvline(x=0, color='C2', alpha=0.5)
plt.axis(axis)
plt.plot(x, y, linewidth=2.0)


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Plotting output of the model that failed to learn, given a set of hyper-parameters:
Can you write Python code for it?

plot_decision_boundary(lambda x: predict(model, x))
plt.title("Decision Boundary for hidden layer size  50")


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Plotting sigmoid activation function:
Can you write Python code for it?

xmin= -10
xmax = 10
ymin = -0.1
ymax = 1.1
step_size = 0.01

x = list(np.arange(xmin, xmax, step_size))
y = []
for i in x:
  y_i = sigmoid(i)
  y.append(y_i)

axis = [xmin, xmax, ymin, ymax]
plt.axhline(y=0.5, color='C2', alpha=0.5)
plt.axvline(x=0, color='C2', alpha=0.5)
plt.axis(axis)
plt.plot(x, y, linewidth=2.0)


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Plotting the decision boundary layer generated by an improved neural network model:
Can you write Python code for it?


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Plotting the rectified linear units (ReLU) activation function:
Can you write Python code for it?

def ReLU(x, h = None, derivative=False):
  if derivative:
    return x[h < 0]
  x_relu = np.maximum(x, 0)
  return x_relu

x = list(np.arange(-6.0, 6.0, 0.1))
y = []
for i in x:
  y_i = ReLU(i)
  y.append(y_i)

xmin= -6
xmax = 6
ymin = 0
ymax = 1
axis = [xmin, xmax, ymin, ymax]
plt.axhline(y=0.5, color='C2', alpha=0.5)
plt.axvline(x=0, color='C2', alpha=0.5)
plt.axis(axis)
plt.plot(x, y, linewidth=2.0)


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Randomly initialize weights:
Can you write Python code for it?

w1 = np.random.randn(D_in, H)
w2 = np.random.randn(H, D_out)

learning_rate = 0.002
update_freq = 10
training_steps = 200

loss_col = []


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
ReLu as the activation function and squared error as the loss function:
Can you write Python code for it?

for t in range(training_steps):
    # Forward pass: compute predicted y
    h = x.dot(w1)
    h_relu = np.maximum(h, 0)  # using ReLU as activate function
    y_pred = h_relu.dot(w2)

    # Compute and print loss
    loss = np.square(y_pred - y).sum() # squared error as the loss function
    loss_col.append(loss)
    if ((t*update_freq) % training_steps ==0):
      print ('Training step :' + str(t))
      print ('Loss function during training :' + str(loss))

    # Backprop to compute gradients of w1 and w2 with respect to loss
    grad_y_pred = 2.0 * (y_pred - y) # the last layer's error
    grad_w2 = h_relu.T.dot(grad_y_pred)
    grad_h_relu = grad_y_pred.dot(w2.T) # the second layer's error
    grad_h = grad_h_relu.copy()
    grad_h[h < 0] = 0  # the derivate of ReLU
    grad_w1 = x.T.dot(grad_h)

    # Update weights
    w1 -= learning_rate * grad_w1
    w2 -= learning_rate * grad_w2

print ('Training completed ...')
print ('Predictions :' + str (y_pred))

plt.plot(loss_col)
plt.show()

delete_model = True

if delete_model:
  try:
    del loss_col
  except:
    pass
  try:
    del input_data
  except:
    pass
  try:
    del output_data
  except:
    pass
  try:
    del x
  except:
    pass
  try:
    del y
  except:
    pass
  try:
    del output_data
  except:
    pass

  import gc
  gc.collect()


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Save model weights
Can you write Python code for it?

model.save_weights(MODEL_PATH)

model.predict(x)

from google.colab import files
files.download(MODEL_PATH)


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Select optimizer
Can you write Python code for it?


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
This function learns parameters for the neural network and returns the model:

nn_hdim: Number of nodes in the hidden layer
num_passes: Number of passes through the training data for gradient descent
print_loss: If True, print the loss every 1000 iterations
Can you write Python code for it?

def build_model(nn_hdim, num_passes=20000, print_loss=False):

    # Initialize the parameters to random values. We need to learn these.
    np.random.seed(0)
    W1 = np.random.randn(nn_input_dim, nn_hdim) / np.sqrt(nn_input_dim)
    b1 = np.zeros((1, nn_hdim))
    W2 = np.random.randn(nn_hdim, nn_output_dim) / np.sqrt(nn_hdim)
    b2 = np.zeros((1, nn_output_dim))

    # This is what we return at the end
    model = {}

    # Gradient descent. For each batch...
    for i in range(0, num_passes):

        # Forward propagation
        z1 = X.dot(W1) + b1
        a1 = np.tanh(z1)
        z2 = a1.dot(W2) + b2
        exp_scores = np.exp(z2)
        probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)

        # Backpropagation
        delta3 = probs
        delta3[range(num_examples), y] -= 1
        dW2 = (a1.T).dot(delta3)
        db2 = np.sum(delta3, axis=0, keepdims=True)
        delta2 = delta3.dot(W2.T) * (1 - np.power(a1, 2))
        dW1 = np.dot(X.T, delta2)
        db1 = np.sum(delta2, axis=0)

        # Add regularization terms (b1 and b2 don't have regularization terms)
        dW2 += reg_lambda * W2
        dW1 += reg_lambda * W1

        # Gradient descent parameter update
        W1 += -epsilon * dW1
        b1 += -epsilon * db1
        W2 += -epsilon * dW2
        b2 += -epsilon * db2

        # Assign new parameters to the model
        model = { 'W1': W1, 'b1': b1, 'W2': W2, 'b2': b2}

        # Optionally print the loss.
        # This is expensive because it uses the whole dataset, so we don't want to do it too often.
        if print_loss and i % 1000 == 0:
          print("Loss after iteration %i: %f" %(i, loss_function(model)))

    return model


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Train model
Can you write Python code for it?


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Training the model:
Can you write Python code for it?

training_steps = 60000

for step in range (0, training_steps):
  sess.run(train)
  if step % 1000 == 0:
    print ('Current training session: ' + str(step) + str(sess.run(synapses))+ str(sess.run(bias)))


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Training the simple XOR gate neural network:

Note: There is no function that defines a neuron! In practice neuron is just an abstract concept to understand the probability function.
Continuously feeding the data throught the neural network.
Updating the weights of the network through backpropagation.
During the training the model becomes better and better in predicting the output values.
The layers are just matrix multiplication functions that apply the sigmoid function to the synapse matrix and the corresponding layer.
Backpropagation portion of the training is the machine learning portion of this code.
Backpropagation function reduces the prediction errors during each training step.
Synapses and weights are synonymous.
Can you write Python code for it?

training_steps = 50000
update_freq = 10

input_data = x
output_data = y

bias_val_1 = 1e-2
bias_val_2 = 10

learning_rate = 0.1

for t in range(training_steps):
  # Creating the layers of the neural network:
  layer_0 = input_data
  layer_1 = sigmoid(np.dot(layer_0, synapse_0)+bias_val_1)
  layer_2 = sigmoid(np.dot(layer_1, synapse_1)+bias_val_2)

  # Backpropagation:
  outputLoss_derivative = output_data - layer_2
  loss_col.append(np.mean(np.abs(outputLoss_derivative)))
  if ((t*update_freq) % training_steps == 0):
    print ('Training step :' + str(t))
    print ('Prediction error during training :' + str(np.mean(np.abs(outputLoss_derivative))))

  # Layer-wsie delta function:
  layer_2_delta = (learning_rate*outputLoss_derivative*sigmoid(layer_2, derivative = True))
  layer_1_error = layer_2_delta.dot(synapse_1.T) # Matrix multiplication of the layer 2 delta with the transpose of the first synapse function.
  layer_1_delta = (layer_1_error*learning_rate)*(sigmoid(layer_1, derivative = True))

  # Updating synapses or weights:
  synapse_1 += layer_1.T.dot(layer_2_delta)
  synapse_0 += layer_0.T.dot(layer_1_delta)
  del layer_0
  del layer_1

print ('Training completed ...')
print ('Predictions :' + str (layer_2))

plt.plot(loss_col)
plt.show()

delete_model = True

if delete_model:
  try:
    del loss_col
  except:
    pass
  try:
    del input_data
  except:
    pass
  try:
    del output_data
  except:
    pass
  try:
    del x
  except:
    pass
  try:
    del y
  except:
    pass
  try:
    del layer_2
  except:
    pass
  try:
    del output_data
  except:
    pass
  try:
    del synapse_0
  except:
    pass
  try:
    del synapse_1
  except:
    pass
  import gc
  gc.collect()


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Updating the weights/synapses of the neural network
Can you write Python code for it?

synapse_1 += layer_1.T.dot(layer_2_delta)

synapse_1.shape

synapse_0 += layer_0.T.dot(layer_1_delta)

synapse_0


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Visualize the logistic regression classifier output:
Can you write Python code for it?

def plot_decision_boundary(prediction_function):
  # Setting minimum and maximum values for giving the plot function some padding
  x_min, x_max = X[:, 0].min() - .5, \
                 X[:, 0].max() + .5
  y_min, y_max = X[:, 1].min() - .5, \
                 X[:, 1].max() + .5
  h = 0.01
  # Generate a grid of points with distance h between them
  xx, yy = np.meshgrid(np.arange(x_min, x_max, h), \
                       np.arange(y_min, y_max, h))
  # Predict the function value for the whole grid
  Z = prediction_function(np.c_[xx.ravel(), yy.ravel()])
  Z = Z.reshape(xx.shape)
  # Plotting the contour and training examples
  plt.contourf(xx, yy, Z, cmap=plt.cm.get_cmap("Spectral"))
  plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.get_cmap("Spectral"))


I am working on the file "Fundamentals_of_deep-learning/notebooks/01_Introduction_Building_a_neural_network.ipynb".
The first task is:
Visualizing the hidden layers with varying sizes:
Can you write Python code for it?

plt.figure(figsize=(16, 32))
hidden_layer_dimensions = [1, 2, 3, 4, 5, 20, 50]
for i, nn_hdim in enumerate(hidden_layer_dimensions):
    plt.subplot(5, 2, i+1)
    plt.title('Hidden Layer size %d' % nn_hdim)
    model = build_model(nn_hdim)
    plot_decision_boundary(lambda x: predict(model, x))
plt.show()


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Create a Matmul op that takes 'matrix1' and 'matrix2' as inputs. The returned value, 'product', represents the result of the matrix multiplication.
Can you write Python code for it?


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Create another Constant that produces a 2x1 matrix
Can you write Python code for it?


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Define constant tensors
Can you write Python code for it?

print("Initialize constant tensors ...")
a = tf.constant(2)
print("a = %i" % a)
b = tf.constant(3)
print("b = %i" % b)


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Initialize the tf session
Can you write Python code for it?


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Iteration operations through a tensor
Can you write Python code for it?

print("Iterating through Tensor 'a': ...")
for i in range(a.shape[0]):
    for j in range(a.shape[1]):
        print(a[i][j])


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Launching the default graph and perform some basic functions

Launch tensorflow graph
Return the values of constant op
Return the sum and product of the graph input
Can you write Python code for it?

with tf.Session() as sess:
    print ("a: %i" % sess.run(a), "b: %i" % sess.run(b))
    print ("Addition with constants using tensorflow: %i" % sess.run(a+b))
    print ("Multiplication with constants using tensorflow: %i" % sess.run(a*b))


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Matrix operations using tensorflow eager API

Run matrix operations without explicitly calling tf.Session
Can you write Python code for it?

print("Running matrix operations without tf.Session ...")

c = a + b
print("a + b = %s" % c)

d = tf.matmul(a, b)
print("a * b = %s" % d)


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Part 04 -- Object Oriented Programming Using Tensorflow
Object oriented example of tensorflow addition and multiplication:
Can you write Python code for it?

import tensorflow as tf
class tf_basic_ops:
  def __init__(self, a = None, b = None):
    self.a = a
    self.b = b
    a = tf.placeholder(tf.int16)
    b = tf.placeholder(tf.int16)
    self.input_dict = {a: self.a, b: self.b}
  def tf_add(self):
    add = tf.add(self.a, self.b)
    with tf.Session() as sess:
      return sess.run(add, feed_dict=self.input_dict)
  def tf_mul(self):
    mul = tf.multiply(self.a , self.b)
    with tf.Session() as sess:
      return sess.run(mul, feed_dict=self.input_dict)
  def tf_mat_mul(self):
    mat_mul = tf.matmul(self.a , self.b)
    with tf.Session() as sess:
      return sess.run(mat_mul, feed_dict=self.input_dict)

tf_ops = tf_basic_ops(a=2, b = 3)

tf_ops.tf_add()

tf_ops.tf_mul()

tf_matrix_ops = tf_basic_ops(a=[[3., 3.]], b = [[2.],[2.]])

tf_matrix_ops.tf_mat_mul()


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Passing a variable as the tensorflow graph input
In this portion of the notebook, basic operations with variable as graph input is discussed. The value returned by the constructor represents the output of the Variable op. It is defined as an input when running session.
Initialize a tensorflow placeholder
Can you write Python code for it?

a = tf.placeholder(tf.int16)
b = tf.placeholder(tf.int16)


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Run the tensorflow graph
Can you write Python code for it?


I am working on the file "Fundamentals_of_deep-learning/notebooks/03_Introduction_to_tensorflow.ipynb".
The first task is:
Running the tensorflow operations without calling the tf.Session explicitly
Can you write Python code for it?

print("Running operations, without tf.Session")
c = a + b
print("a + b = %i" % c)
d = a * b
print("a * b = %i" % d)


I am working on the file "Fundamentals_of_deep-learning/notebooks/05a_Introduction_to_GPU_accelerated_Tensorflow.ipynb".
The first task is:
Part 02 -- Manually specifying devices for running Tensorflow code
Can you write Python code for it?

import tensorflow as tf

# Creates a graph.
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)

# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

# Runs the op.
print(sess.run(c))


I am working on the file "Fundamentals_of_deep-learning/notebooks/05a_Introduction_to_GPU_accelerated_Tensorflow.ipynb".
The first task is:
Part 03 -- Benchmarking Tensorflow GPU vs CPU
Can you write Python code for it?

import time
import tensorflow as tf

def tf_benchmark(a=None, shape_a=None, b=None, shape_b=None, enable_GPU = False):
    device = 'cpu'
    if enable_GPU:
        device = 'gpu'
    start_time = time.time()
    with tf.device('/{}:0'.format(device)):
        a = tf.constant(a, shape=shape_a, name = 'a')
        b = tf.constant(b, shape=shape_b, name='b')
    c = tf.matmul(a, b)
    sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
    output = sess.run(c)
    execution_time = time.time()-start_time
    return {'output': output, 'execution time': execution_time}

a=[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
b=[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
shape_a=[2, 3]
shape_b=[3,2]

CPU_exec_time = tf_benchmark(a=a,
                             b=b,
                             shape_a=shape_a,
                             shape_b=shape_b,
                             enable_GPU=False)

GPU_exec_time = tf_benchmark(a=a,
                             b=b,
                             shape_a=shape_a,
                             shape_b=shape_b,
                             enable_GPU=True)

print ("CPU execution time: {}".format(CPU_exec_time['execution time']))
print ("GPU execution time: {}".format(GPU_exec_time['execution time']))
print ("GPU vs CPU execution time delta: {}".format(GPU_exec_time['execution time'] - CPU_exec_time['execution time']))
print ("GPU acceleration factor: {}".format(CPU_exec_time['execution time'] / GPU_exec_time['execution time']))


I am working on the file "Fundamentals_of_deep-learning/notebooks/05a_Introduction_to_GPU_accelerated_Tensorflow.ipynb".
The first task is:
Setting up Tensorflow to run on CPU
Can you write Python code for it?

# Creates a graph.
with tf.device('/cpu:0'):
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)

# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

# Runs the op.
print(sess.run(c))


I am working on the file "Fundamentals_of_deep-learning/notebooks/05a_Introduction_to_GPU_accelerated_Tensorflow.ipynb".
The first task is:
Setting up Tensorflow to run on GPU
Can you write Python code for it?

with tf.device('/gpu:0'):
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)

# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

with tf.Session() as sess:
    print (sess.run(c))


I am working on the file "Funções de Ativação.ipynb".
The first task is:
Exponential Linear Unit (eLU)
$$y = \begin{cases}\alpha(e^x -1) &,\ x \leq 0\x &,\ x > 0\end{cases}$$
$$\frac{\partial y}{\partial x} = \begin{cases}y + \alpha &,\  x \leq 0\1 &,\  x > 0\end{cases}$$
Can you write Python code for it?

interactive_plot = interactive(plot_interactive, w=(-2.0, 2.0), b=(-3, 3, 0.5), func=fixed(elu), ylim=fixed((-2, 10)))
interactive_plot


I am working on the file "Funções de Ativação.ipynb".
The first task is:
Leaky ReLU
$$y = \begin{cases}\alpha x &,\ x \leq 0\x &,\ x > 0\end{cases}$$
$$\frac{\partial y}{\partial x} = \begin{cases}\alpha &,\  x \leq 0\1 &,\  x > 0\end{cases}$$
Can you write Python code for it?

interactive_plot = interactive(plot_interactive, w=(-2.0, 2.0), b=(-3, 3, 0.5), func=fixed(leaky_relu), ylim=fixed((-1, 10)))
interactive_plot


I am working on the file "Funções de Ativação.ipynb".
The first task is:
Linear
$$y=x$$
$$y^\prime = 1$$
Can you write Python code for it?

interactive_plot = interactive(plot_interactive, w=(-2.0, 2.0), b=(-3, 3, 0.5), func=fixed(linear), ylim=fixed((-10, 10)))
interactive_plot


I am working on the file "Funções de Ativação.ipynb".
The first task is:
Rectified Linear Unit (ReLU)
$$y = max(0, x)$$
$$\frac{\partial y}{\partial x} = \begin{cases}0 &,\  x \leq 0\1 &,\ x > 0\end{cases}$$
Obs.: Lembrando que a derivada da ReLU quando x = 0 não existe matematicamente, mas é convencionalmente definida como 0.
Can you write Python code for it?

interactive_plot = interactive(plot_interactive, w=(-2.0, 2.0), b=(-3, 3, 0.5), func=fixed(relu), ylim=fixed((-1, 10)))
interactive_plot


I am working on the file "Funções de Ativação.ipynb".
The first task is:
Sigmoid
$$y = \frac{1}{1+e^{-x}}$$
$$y^\prime = y(1-y)$$
Can you write Python code for it?

interactive_plot = interactive(plot_interactive, w=(-2.0, 2.0), b=(-3, 3, 0.5), func=fixed(sigmoid))
interactive_plot


I am working on the file "Funções de Ativação.ipynb".
The first task is:
Tanh
$$y = \frac{e^x - e^{-x}}{e^x+e^{-x}}$$
$$y^\prime = 1 - y^2$$
Can you write Python code for it?

interactive_plot = interactive(plot_interactive, w=(-2.0, 2.0), b=(-3, 3, 0.5), func=fixed(tanh), ylim=fixed((-2, 2)))
interactive_plot


I am working on the file "Furemu/Frame Layout Scoring Notebook.ipynb".
The first task is:
Collisions
Can you write Python code for it?

Collisions are scored on an exponential curve, dictated by a shape parameter

@interact(collision_shape = (0.01, 1, 0.01))

def plt_col(collision_shape = 0.15):
    collisions = np.arange(0, len(part_shelf), 1)
    collision_score = np.exp(-(collisions/len(part_shelf))/collision_shape)
    plt.plot(collisions, collision_score)
    plt.show()


I am working on the file "Furemu/Frame Layout Scoring Notebook.ipynb".
The first task is:
Scoring
Weight Distribution- X
Weight distribution in X (fore-aft) is scored using an exponential. Scoring curves are defined by two parameters: Shape and minimum theoretical collapsible length.
Alternatively, a logit curve is proposed as a CG_x scoring method.
Can you write Python code for it?

# Define shape variable

# Get theoretical max-forward CG_x:
min_length = em.get_min_collapsible_x(part_shelf, strategy = 'avg')
print("Min Length:", min_length)

# Plot scoring method
CG_x = np.arange(0, 500, 0.01)
CG_x_precalc = (CG_x - min_length) / (frame.frame_length - min_length)

@interact(CG_x_shape=(0.05, 1, 0.05))
def plt_CG_x(CG_x_shape = 0.3):
    CG_x_score = np.exp(-CG_x_precalc / CG_x_shape)

    plt.plot(CG_x, CG_x_score)
    plt.plot((min_length, min_length), (0, max(CG_x_score)), 'red')
    plt.show()

def plt_CG_x_logit(s, min_length):
    u = min_length + (frame.frame_length / 10)
    CG_x_logit = 1 - (1 / (1 + np.exp(-(CG_x- u)/s)))
    plt.plot(CG_x, CG_x_logit)
    plt.plot((min_length, min_length), (0, max(CG_x_logit)), 'red')
    plt.show()
interact(plt_CG_x_logit, s = (1, 500, 1))


I am working on the file "Furemu/Frame Layout Scoring Notebook.ipynb".
The first task is:
Time
Score decays slowly over time in a linear manner with every frame
Can you write Python code for it?

seconds = np.arange(0, 300, 1/30)

time_penalty = seconds * -0.001

plt.plot(seconds, time_penalty)
plt.show()


I am working on the file "Furemu/Frame Layout Scoring Notebook.ipynb".
The first task is:
We must first setup a sample environment with a frame and components.
Can you write Python code for it?

### Setup frame environment
y_centerline = 200
left_buffer = 0

base_bom = frame_methods.read_bom(bom_name = "bom1.json")
part_shelf = base_bom.items
frame = frame_methods.frame_layout(bom_frame_spec = base_bom.frame_spec, frame_width = 35)


I am working on the file "Furemu/Frame Layout Scoring Notebook.ipynb".
The first task is:
Weight Distribution- Y
Y weight distribution is scored symmetrically about the y centerline. The curve is a normal distribution, with a shape define by 1) the frame width and 2) a shape factor.
Can you write Python code for it?

@interact(CG_y_shape = (0.05, 1, 0.05))

def plt_CG_Y(CG_y_shape = 1.0):
    rv = norm(loc = 0.0, scale = frame.frame_width * CG_y_shape)
    x = np.arange(-60, 60, .1)
    CG_y_score = rv.pdf(x) / max(rv.pdf(x))
    #plot the pdfs of these normal distributions
    plt.plot(x, CG_y_score)
    plt.show()


I am working on the file "Février 2021 un mini challenge arithmético-algorithmique.ipynb".
The first task is:
(le kernel ocaml-jupyter est génial, mais plante un peu, j'ai galéré à écrire cette douzaine de cellules sans devoir relancer Jupyter plusieurs fois... bug signalé, résolution en cours...)

Réponse en Rust (par un de mes élèves (Théo Degioanni))
Le code Rust proposé peut être executé depuis le bac à sable Rust :
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=2ab9c57e9d114a344363e21f9493bf22
Mais on peut aussi utiliser le kernel Jupyter proposé par le projet evcxr de Google :

il faut installer Rust, je n'avais jamais fait j'ai donc suivi rustup.rs le site officiel de présentation de l'installation de Rust ;
puis j'ai suivi les explications pour installer le kernel sur GitHub @google/evcxr ;
puis j'ai écrit cette cellule ci-dessous, j'ai changé le Noyau en Rust, et j'ai exécuté la cellule ;
notez qu'avec l'extension jupyter nbextension ExecuteTime, j'ai vu que la première cellule avait pris quasiment 10 secondes... mais je pense qu'il s'agit du temps d'installer et de compilation du module chrono (je ne suis pas encore très familier avec Rust).
Les exécutions suivantes prennent environ 300ms pour la définition (y a-t-il recompilation même si le texte ne change pas ?) de la fonction ;

Et environ 700ms pour l'exécution. C'est bien plus lent que les 35 ms de mon code naïf en OCaml (qui est juste interprété et pas compilé), que les 10 ms de Python, ou les 100 ms de Bash. Mais pas grave !

Date et heure : lundi 01/02/2021, 21h20
Can you write Python code for it?

:dep chrono = "0.4"
use chrono::TimeZone;
use chrono::Datelike;

fn main() {
    let n = (1994..=2077)
        .filter(|n| chrono::Utc.ymd_opt(*n, 2, 29) == chrono::LocalResult::None)
        .map(|n| chrono::Utc.ymd(n, 2, 1))
        .filter(|t| t.weekday() == chrono::Weekday::Mon)
        .count();

    println!("{}", n);
}

main()


I am working on the file "Février 2021 un mini challenge arithmético-algorithmique.ipynb".
The first task is:
Et donc on a juste à compter les années, de 1994 à 2077 inclus, qui ne sont pas des années bissextiles et qui satisfont le filtre :
Can you write Python code for it?

%%time
len(list(filter(filter_annee, ( annee
        for annee in range(1994, 2077 + 1)
        # if not calendar.isleap(annee) # en fait c'est inutile
    )
)))


I am working on the file "Février 2021 un mini challenge arithmético-algorithmique.ipynb".
The first task is:
Je vais tricher un peu et utiliser la connaissance que le 01/01/1994 est un samedi :
Can you write Python code for it?

Sys.command "ncal Jan 1994 | grep '  1'";

let start_date = {d=1; q=6; m=1; y=1994};; (* 01/01/1994 était un samedi, q=6 *)
(* let start_date = {d=1; q=3; m=1; y=2020};; (* 01/01/2020 était un mercredi, q=3 *) *)

let end_date   = {d=31; q=0; m=1; y=2077};; (* on se fiche du quantième ici ! *)
(* let end_date   = {d=31; q=0; m=1; y=2021};; (* on se fiche du quantième ici ! *) *)

let aujourdhui = ref start_date;; (* les champs sont pas mutables, go reference *)
let nb_bon_mois_fevrier = ref 0;;
let aujourdhuis = ref [];;
let solutions = ref [];;

while (!aujourdhui.y <= end_date.y || !aujourdhui.y <= end_date.y || !aujourdhui.y <= end_date.y) do
    if (!aujourdhui.d = 1 && !aujourdhui.m = 2) then begin
        (* on a un début de février, est-ce qu'il vérifie les critères ? *)
        let date_suivante    = iterate 27 next_day !aujourdhui in
        let date_suivante_p1 = next_day date_suivante in
        if (
               date_suivante.d = 28 && date_suivante.m = 2
            && date_suivante_p1.d != 29 (* année pas bisextile *)
            && !aujourdhui.q = 1        (* mois février commence par lundi *)
        ) then begin
            solutions := !aujourdhui :: !solutions;
            incr nb_bon_mois_fevrier;
        end;
    end;
    (* on a un jour quelconque, on avance d'un mois *)
    aujourdhui := start_of_next_month !aujourdhui;
    aujourdhuis := !aujourdhui :: !aujourdhuis;
done;;

!nb_bon_mois_fevrier;;


I am working on the file "Février 2021 un mini challenge arithmético-algorithmique.ipynb".
The first task is:
Réponse en OCaml (par Lilian Besson)
En installant et utilisant ocaml-calendar cela ne doit pas être trop compliqué. On peut s'inspirer du code Java ci-dessus, qui a une approche purement fonctionnelle.

Date et heure : ?
Can you write Python code for it?

(* cette cellule est en OCaml *)
(* avec la solution en Bash et Sys.command... *)
Sys.command "bash -c \"for ((annee=1994; annee<=2077; annee+=1)); do ncal February \\$annee | grep 'lu  1  8 15 22' | grep -v 29; done | wc -l\"";;
(* mais ça ne compte pas ! *)


I am working on the file "Février 2021 un mini challenge arithmético-algorithmique.ipynb".
The first task is:
Si les cellules précédentes ne s'exécute pas, a priori c'est normal : ce notebook est déclaré en Python !
Il faudrait utiliser une des astuces suivantes, mais flemme.
Can you write Python code for it?

System.out.println("Test d'une cellule en Java dans un notebook déclaré comme Java");
// ==> ça marche !

%%java
System.out.println("Test d'une cellule en Java dans un notebook déclaré comme Python");
// cela ne marche pas !

# On peut aussi écrire une cellule Python qui fait appel à une commande Bash
!echo 'System.out.println("\nTest d\'une ligne de Java dans un notebook déclaré comme Python");' | jshell -q

%%bash
# voir une commande Bash directement !
# mais uniquement depuis un notebook Python
echo 'System.out.println("\nok");' | jshell -q


I am working on the file "Física Computacional/Ficha 1 - Interpolacao.ipynb".
The first task is:
Como o método funciona, faz-se a interpolação em grau $n \in [ 1, 2, 3, 4 , 5]$ com $n+1$ pontos equidistantes no intervalo $x \in [-5, 5]$
A verde está representada a função original, $f(x) =\frac{1}{1+x^{2}}$, e a azul o polinómio obtido para cada caso.
Os pontos representados são os requeridos:
[-4.92, -2.67, -1.58, 0.88, 2.22, 3.14, 4.37]
Can you write Python code for it?

fig = plt.figure(figsize= (20, 12))
x_span = np.linspace(-5, 5, num = 200)

x_set = [-4.92, -2.67, -1.58, 0.88, 2.22, 3.14, 4.37]
y_set = list(map(func_x, x_set))

for i in range(1, 6):
    x_p = xinterval(-5, 5, i+1)
    y_p = func_x(x_p)
    y_gen, y_poli = newtoninterp(x_p, y_p, x_span)
    ip = fig.add_subplot(3, 2, i)
    ip.set_title('Interpolação de grau: ' + str(i))
    print('Coeficientes da Interpolação de grau ' + str(i) + ':')
    s=''
    for p in range(0, len(y_poli)): s += 'C' + str(p) + ': ' + str(y_poli[p]) + ';  '
    print(s)
    #plt.scatter(x_p, y_p)
    plt.plot(x_span, y_gen, 'c' , label = 'polinómio')
    plt.plot(x_span, func_x(x_span), 'r', label = r'$\frac{1}{1+x^{2}}$') # original function
    plt.scatter(x_set, y_set)
    if i == 4 or i == 5: plt.xlabel('$x_{i}$', size=20)
    if i %2 != 0: plt.ylabel('$f(x_{i})$', size = 16)
    plt.legend(loc='best')
    # plt.plot(x_span, exact_gen)
plt.show()


I am working on the file "Física Computacional/Ficha 1 - Interpolacao.ipynb".
The first task is:
Dados um conjunto de pontos $x$, suas respectivas imagens $y$ e um intervalo de pontos x_new:
+ Calculam-se os Coeficientes com o método das diferenças divididas
+ Para cada ponto de x_new, aplica-se o polinómio, obtendo os valores de y_new.
Can you write Python code for it?

func_x = lambda x: 1/ (1 + x**2) # Função dada

def xinterval(x_i, x_f, n):
    " Gera um array de 'n' pontos equidistantes entre 'x_i' e 'x_f' "
    xn = np.linspace(x_i, x_f, num = n)
    return xn

def newtoninterp(x, y, x_new):
    """ Dados um conjunto de pontos 'x', suas respectivas imagens 'y', e
    um intervalo de pontos 'x_new' para o qual se vão aplicar o
    polinómios """

    n = len(x)

    def difdiv(xi):
        "Cálculo dos coeficientes através do método das diferenças divididas"
        d = list(y)
        for j in range(1, n):
            for i in range (n-1, j-1, -1): # intervalo escolhido de forma a evitar dependências de variáveis no cálculo
                d[i] = (d[i]-d[i-1]) / (xi[i] - xi[i-j])
        return d

    def interpol(coef, x_pts, x_new):
        y_new = []
        for pt in x_new:
            co = coef[len(coef)-1] # último coeficiente
            for i in range(n-2, -1, -1): # intervalo para multiplicar do último ponto para o primeiro, sem dependências
                co *= pt - x_pts[i]
                co += coef[i]
            y_new.append(co)
        return y_new
    coef_l = difdiv(x)
    ypol = interpol(coef_l, x, x_new)
    return ypol, coef_l


I am working on the file "Física Computacional/Ficha 1 - Interpolacao.ipynb".
The first task is:
Implementa-se o método de Polinómio Interpolador pelo método de Lagrange:
Can you write Python code for it?

def Lagrangepolinterp(x, y, x_new):
    n = len(x)
    y_new = []
    for x_n in x_new:
        lag_pols = []
        for i in range(n):
            l = 1
            for k in range(n):
                if k != i:
                    l *= (x_n - x[k]) / (x[i] - x[k])
            lag_pols.append(l)

        point = 0
        for i, j in zip(y, lag_pols):
            point += i * j
        y_new.append(point)
    return y_new


I am working on the file "Física Computacional/Ficha 1 - Interpolacao.ipynb".
The first task is:
Para um polinómio de grau 20, observa-se um óptimo ajuste em volta dos pontos centrais,
e uma oscilação imensa nos pontos junto aos extremos iniciais e finais, o que é efectivamente um mau resultado.
Isto permite verificar, como esperado, o fenómeno de Runge.
Nos pontos $x = -4.75$ e $x = 4.8$ encontramos as oscilações que fogem muito ao valor real.
Can you write Python code for it?

x_20 = xinterval(-5, 5, 21)
y_20 = list(map(func_x, x_20))
evalx_20 = [-4.75, 4.8]
evaly_20 = list(map(func_x, evalx_20))
pol_20, lpol_20 = newtoninterp(x_20, y_20, x_span)
print('Coeficientes da Interpolação:')
for p in range(0, len(lpol_20)): print('C' + str(p) + ': ' + str(lpol_20[p]))
fig = plt.figure(figsize = (10, 4))
i20 = fig.add_subplot(111)

i20.set_title('Interpolação de grau 20')
plt.plot(x_span, pol_20, 'c', label = 'interpolado')
plt.plot(x_span, func_x(x_span), 'r' '--' , label = r'$\frac{1}{1+x^{2}}$')
plt.scatter(evalx_20, evaly_20)
plt.xlabel('$x_{i}$', size=20)
plt.ylabel('$f(x_{i})$', size = 16)
plt.legend(loc = 'best')
plt.show()


I am working on the file "Física Computacional/Ficha 1 - Interpolacao.ipynb".
The first task is:
Tal como para o caso do Método de Interpolação de Newton, confirma-se que o método de Interpolação de Lagrange (polinómico) implementado se comporta conforme o esperado.
Can you write Python code for it?

lag_pol_test = Lagrangepolinterp(test_x, test_y, plot_x_range)
plt.figure(figsize=(12, 5))
plt.plot(plot_x_range, lag_pol_test, 'r' '--', label = 'interpolação Lagrange')
plt.plot(plot_x_range, pol_dado(plot_x_range), 'c' '-', label = 'polinómio dado')
plt.scatter(test_x, test_y)
plt.legend(loc='upper center')
plt.title('Teste de implementação do método polinomial de Lagrange')
plt.xlabel('$x_{i}$', size=20)
plt.ylabel('$f(x_{i})$', size = 16)

plt.show()

x_in, x_fi, dx = 3500, 8500, 0.01   #intervalo de pontos entre 3000 e 9000 a cada 0.01
wl_range = np.arange(x_in, x_fi, dx)
n_refr = Lagrangepolinterp(wl, n_wl, wl_range)
plt.figure(figsize = (12,5))
plt.scatter(wl, n_wl, color = 'red', label = '$\lambda (n)$ da tabela')
plt.plot(wl_range, n_refr, 'c', label = '$\lambda (n)$ interpolado')
plt.title('Interpolação de $\lambda (n)$ utilizando o método polinomial de Lagrange')
plt.xlabel('$\lambda$', size=16)
plt.ylabel('$n$', size=18)
plt.legend(loc='upper right')
plt.show()


I am working on the file "Física Computacional/Ficha 1 - Interpolacao.ipynb".
The first task is:
Utilizando o polinómio interpolador pelo método de Lagrange, determina-se o índice de refracção para $\lambda = 5000 \mathring A$ como:
Can you write Python code for it?


I am working on the file "Física Computacional/Ficha 2.ipynb".
The first task is:
Define-se a nossa função:
$f(x) = \sin (x)$
e a sua derivada:
$f'(x) = \cos (x)$
Can you write Python code for it?

f = lambda x: np.sin(x)
df = lambda x: np.cos(x)
my_stop = 1.e-4
my_nitmax = 100000
my_cdif = 1.e-6


I am working on the file "Física Computacional/Ficha 2.ipynb".
The first task is:
Exercício 2
$f(x) = x + e^{-k x^{2}}$
Can you write Python code for it?

from scipy.misc import derivative

def newtraphd(c0, fun, eps, nitmax):
    c = c0
    dfun = lambda x: derivative(fun, x, 0.0001)
    it = 1
    while np.abs(fun(c)) > eps and it < nitmax:
        c = c - (fun(c) / dfun(c))
        it += 1
    return it, c, fun(c), dfun(c)

f2 = lambda x, k: x + np.e ** (-k * x**2) * np.cos(x)

f2_k1 = lambda x: f2(x, 1)
newtraph(0, f2_k1, df2_k1, 1e-4, my_nitmax)

f2_k50 = lambda x: f2(x, 50)
for i in range(1, 10+1): print(newtraphd(0, f2_k50, 1e-4, i))

for i in range(1, 10+1): print(newtraphd(-0.1, f2_k50, 1e-4, i))


I am working on the file "Física Computacional/Ficha 2.ipynb".
The first task is:
Método da Bisecção
Can you write Python code for it?

def bi(a, b, fun, eps, nitmax):
    c = (a + b) / 2
    it = 1
    while np.abs(fun(c)) > eps and it < nitmax:
        if fun(a)*fun(c) < 0: b = c
        else: a = c
        c = (a + b) / 2
        it += 1
    return it, c, fun(c)

bi(2, 4, f, my_stop, my_nitmax)


I am working on the file "Física Computacional/Ficha 2.ipynb".
The first task is:
Método de Newton-Raphson:
Can you write Python code for it?

def newtraph(c0, fun, dfun, eps, nitmax):
    c = c0
    it = 1
    while np.abs(fun(c)) > eps and it < nitmax:
        c = c - fun(c) / dfun(c)
        it += 1
    return it, c, fun(c)

newtraph(2, f, df, my_stop, my_nitmax)


I am working on the file "Física Computacional/Ficha 3.ipynb".
The first task is:
Calcular o integral:
$\int _{0}^{\infty} \frac{x dx}{(1+x)^{4}}$
Can you write Python code for it?

xi, wi = leggauss(100)

f3 = lambda x: x / (1+x)**4
gausslegendre(f3, 0, oo, xi, wi)


I am working on the file "Física Computacional/Ficha 3.ipynb".
The first task is:
Derivar $e^{x} \sin(x) + e^{-x} \cos(x)$ nos pontos:
x = 0, $\frac{\pi}{4}$, $\frac{\pi}{2}$, $\frac{3\pi}{4}$ e $\pi$.
Utilizando as fórmulas a 2, 3 e 5 pontos.
com passos h = 0.1, 0.05, 0.01
Can you write Python code for it?

f2 = lambda x: exp(x)*sin(x) + exp(-x)*cos(x)
f2_sol = lambda x: (exp(x)-exp(-x)) * (sin(x) + cos(x))
x_2 = [0, pi/4, pi/2, 3*pi/4, pi]
h_2 = [0.1, 0.05, 0.01]

df_2pts = lambda f, x, h: (f(x+h) - f(x)) / h
df_3pts = lambda f, x, h: (-f(x + 2*h) + 4*f(x+h) - 3*f(x)) / (2*h)
df_5pts = lambda f, x, h: (-3*f(x+4*h) + 16*f(x+3*h) - 36*f(x+2*h) + 48*f(x+h) - 25*f(x)) / (12*h)

for x in x_2:
    print('\nDerivada de f(' + str(x) + ') :' )
    d_sol = f2_sol(x)
    print('Valor real = ' + str(d_sol))
    for h in h_2:
        print('com passo \'h\' = ' + str(h) + ' :')
        d2r = df_2pts(f2, x, h)
        print('Fórmula a 2 pontos: ' + str(d2r) + '   Erro relativo: ' + str(erro_rel(d2r, d_sol)))
        d3r = df_3pts(f2, x, h)
        print('Fórmula a 3 pontos: ' + str(d3r) + '   Erro relativo: ' + str(erro_rel(d3r, d_sol)))
        d5r = df_5pts(f2, x, h)
        print('Fórmula a 5 pontos: ' + str(d5r) + '   Erro relativo: ' + str(erro_rel(d5r, d_sol)))


I am working on the file "Física Computacional/Ficha 3.ipynb".
The first task is:
Integral Duplo de
$\int {0}^{1} \left( \int {-\sqrt{1-y^{2}}} ^{\sqrt{1-y^{2}}} e^{-xy} \, dx \right) dy$
Can you write Python code for it?

gausslegendre((lambda y: gausslegendre(lambda x: e**(-x*y), -(1-y**2)**(1/2), (1-y**2)**(1/2), xi, wi)), 0, 1, xi, wi)


I am working on the file "Física Computacional/Ficha 3.ipynb".
The first task is:
Integrando $e^{x} \cos (x)$
Can you write Python code for it?

aval_simples(f1, 0, pi, f1_sol)

for n in n_list:
    x_i, w_i = leggauss(n)
    aval_composta(f1, 0, pi, n, x_i, w_i, f1_sol)


I am working on the file "Física Computacional/Ficha 3.ipynb".
The first task is:
Usando as transformações
$x = \frac{y}{1-y}$
$x = \tan \left[ \frac{\pi}{4} (1+y) \right]$
Integral Duplo de
$\int {0}^{1} \left( \int {-\sqrt{1-y^{2}}} ^{\sqrt{1-y^{2}}} \, dx \right) dy$
Can you write Python code for it?

gausslegendre((lambda y: gausslegendre((lambda x: 1), -(1-y**2)**(1/2), (1-y**2)**(1/2) , xi, wi)), 0, 1, xi, wi)


I am working on the file "Física Computacional/Ficha 4.ipynb".
The first task is:
Exercício 4 - Resolver um sistema para entender a necessidade do pivoting
Can you write Python code for it?


I am working on the file "Física Computacional/Ficha 4.ipynb".
The first task is:
Forward Substitution
Can you write Python code for it?

def forward_sub(n, L, b):
    y = np.zeros_like(b)
    y[0] = b[1] / L[1, 1]
    for k in range(1, n):
        soma = 0
        for j in range(0, k - 1): soma += L[k, j] * y[j]
        y[k] = (b[k] - soma) / L[k, k]
    return y


I am working on the file "Física Computacional/Ficha 4.ipynb".
The first task is:
Função para gerar Matrizes de Hilbert de orden n
2 - Resolver a Matriz de Hilbert com ordem 3, 5 e 10.
Can you write Python code for it?

def hilbert_gen(n):
    H = np.array([[1.]*n]*n)
    H_b = []
    for b in range(1, n+1): H_b.append(float(b))
    for i in range(0, n):
        for j in range(0, n):
            H[i, j] = (1/((i + 1) + (j + 1) - 1))
    return H, np.array(H_b)

Hil_3, Hil_3_b = hilbert_gen(3)
print(Hil_3, Hil_3_b)
print(np.linalg.solve(Hil_3, Hil_3_b))
print(np.linalg.det(Hil_3))

H3_S, H3_A, H3_B, H3_D = gauss_elim_nopiv(3, Hil_3, Hil_3_b)
backward_sub(3, H3_S, H3_B)

print(gauss_elim_nopiv(3, Hil_3, Hil_3_b))

Hil_5, Hil_5_b = hilbert_gen(5)
gauss_elim_nopiv(5, Hil_5, Hil_5_b)

Hil_10, Hil_10_b = hilbert_gen(10)
print(Hil_10, Hil_10_b)
print(np.linalg.solve(Hil_10, Hil_10_b))
print(np.linalg.det(Hil_10))

gauss_elim_nopiv(10, Hil_10, Hil_10_b)


I am working on the file "Física Computacional/Ficha 4.ipynb".
The first task is:
Método de Eliminação de Gauss (com pivoting parcial)
Can you write Python code for it?

def gauss_elim_partial(n, matrix, b):
    origin_m = np.copy(matrix)
    origin_b = np.copy(b)

    for k in range(0, n - 1):
        maxi = 0
        km = k
        for k1 in range(k, n):
            if abs(origin_m[k1, k]) > maxi:
                maxi = abs(origin_m[k1, k])
                km = k1
                if km != k:
                    origin_m[k, k:n-1], origin_m[km, k:n-1] = origin_m[km, k:n-1], origin_m[k, k:n-1]
                    origin_b[k], origin_b[km] = origin_b[km], origin_b[k]
    U_A, aux_A, c_A, det_A = gauss_elim_nopiv(n, A, b)
    return U_A, aux_A, c_A, det_A


I am working on the file "Física Computacional/Ficha 4.ipynb".
The first task is:
Método de Eliminação de Gauss (sem pivoting)
Can you write Python code for it?

def gauss_elim_nopiv(n, matrix, b):
    origin_m = np.copy(matrix)
    origin_b = np.copy(b)
    aux_m = np.array([[0.]*n]*n)
    for k in range(0, n):
        # print(origin_m, origin_b)
        for i in range(k+1, n):
            aux_m[i, k] = -origin_m[i, k] / origin_m[k, k]
            origin_m[i, k] = 0
            origin_b[i] = origin_b[i] + aux_m[i, k] * origin_b[k]
            for j in range(k + 1, n):
                origin_m[i, j] = origin_m[i, j] + aux_m[i, k] * origin_m[k, j]

    determ = origin_m[0, 0]
    for d in range(1, n):
        determ *= origin_m[d, d]
    return origin_m, aux_m, origin_b, determ


I am working on the file "Física Computacional/Ficha 4.ipynb".
The first task is:
O determinante deu 2.164480500135833e-53, que é parecido com o obtido pelo cálculo do determinante utilizando o NumPy
Exercício 3
Can you write Python code for it?


I am working on the file "Física Computacional/Ficha 4.ipynb".
The first task is:
Utilizando o NumPy determina-se a inversa de A como:
Can you write Python code for it?


I am working on the file "Física Computacional/Ficha 6.ipynb".
The first task is:
Cálculo de Valores e Vectores Próprios utilizando uma função interna do numpy
Can you write Python code for it?


I am working on the file "Física Computacional/Ficha 6.ipynb".
The first task is:
Cálculo utilizando o método sem critério de parada, mas em vez disso, em função do k
Retornando:
+ Iteração
+ Vector Normalizado
+ Valor próprio
Can you write Python code for it?


I am working on the file "Física Computacional/Ficha 6.ipynb".
The first task is:
Estimativa de $\lambda$ em cada iteração ímpar:
Abaixo, número de iterações necessárias, valor próprio encontrado ao final, vector próprio associado, valor próprio do passo anterior e seu vector próprio, e diferença entre último e penúltimo vectores próprios encontrados.
Can you write Python code for it?


I am working on the file "Física Computacional/Ficha 6.ipynb".
The first task is:
Exercício 2 - Método de Jacobi Cíclico
Can you write Python code for it?

def jacobi(A_i, eps, nitmax):
    A = np.copy(A_i)            # para cortar dependências
    m = len(A)
    iteration = 0
    Q = np.identity(m)
    def off(mat):
        off_sum = 0
        for i in range(m):
            for j in range(m):
                if j != i: off_sum += mat[i, j]**2
        return np.sqrt(off_sum)
    def frobenius_norm(mat):
        norm = 0
        for i in range(m):
            for j in range(m):
                norm += mat[i, j]**2
        return np.sqrt(norm)
    while (off(A) > eps and iteration < nitmax):
        j, k, ajk = 0, 0, 0.
        for ji in range(m-1):
            for ki in range(ji+1, m):
                absjk = abs(A[ji, ki])
                if absjk >= ajk:
                    j, k, ajk = ji, ki, absjk
        def CSjk(mati, j, k):
            mat = np.copy(mati)
            if mat[j, j] == mat[k, k]:
                C, S = np.cos(np.pi / 4), np.sin(np.pi / 4)
            else:
                tau = 2*mat[j, k] / (mat[k, k] - mat[j, j])
                chi = 1 / np.sqrt(1 + tau**2)
                C = np.sqrt((1 + chi) / 2)
                S = np.sign(tau) * np.sqrt((1 - chi) / 2)
            return C, S
        C, S = CSjk(A, j, k)
        A_l = np.zeros_like(A)
        for r in range(m):
            if r != j and r != k:
                A_l[r, j] = C * A[r, j] - S * A[r, k]
                A_l[j, r] = C * A[r, j] - S * A[r, k]
                A_l[r, k] = S * A[r, j] + C * A[r, k]
                A_l[k, r] = S * A[r, j] + C * A[r, k]
                for s in range(m):
                    if s != j and s != k:
                        A_l[r, s] = np.copy(A[r, s])

        A_l[j, j] = np.copy((C**2 * A[j, j]) + (S**2 * A[k, k]) - (2 * S * C * A[j, k]))
        A_l[j, k] = S * C * (A[j, j] - A[k, k]) + ((C**2 - S**2) * A[j, k])
        # A_l[j, k] = 0
        A_l[k, j] = np.copy(A_l[j, k])
        A_l[k, k] = np.copy((S**2 * A[j, j]) + (C**2 * A[k, k]) + (2 * S * C * A[j, k]))
        A = A_l
        Q_l = np.zeros_like(Q)
        for r in range(m):
            for s...


I am working on the file "G1Errou.ipynb".
The first task is:
OK. Tudo certo. Bate com os gráficos mostrados pelo G1, apenas está sendo mostrado de uma forma diferente.
Maaas...
Can you write Python code for it?

datas = ['2014-03-14',
         '2014-05-15', # pré-volume morto
         '2014-05-16', # estréia da "primeira reserva técnica", a.k.a. volume morto
         '2014-07-12', # data em que a "reserva normal" é esgotada
         '2014-10-23',
         '2014-10-24', # "segunda reserva técnica" ou "VOLUME MORTO 2: ELECTRIC BOOGALOO"
         '2015-01-01', # feliz ano novo ?
         '2015-03-14']
import numpy as np
df = pd.DataFrame(pd.concat(map(getData, datas), axis=1))
df = df.T

from sabesPy import plotSideBySide
dados = df.ix[['2014-05-15','2014-05-16']], df.ix[['2014-10-23','2014-10-24']]

plotSideBySide(dados, titles=['$1^a$ Reserva técnica', '$2^a$ Reserva técnica'])


I am working on the file "G1Errou.ipynb".
The first task is:
Pronto. Agora faz sentido, sem aquela variação absurda.
Finalmente, comparemos o grafico com os dados usados pelo G1 e com dados corrigidos
Pronto.
O G1 errou
$-14.8 \neq 14.5$
~~vai ver só confundiram o sinal~~
Can you write Python code for it?

dados = df.ix[datasG1], dFixed.ix[datasG1]
plotSideBySide(dados,cm=[None,None],
    titles=["S/ CORRIGIR as %'s", "%'s CORRIGIDAS"])


I am working on the file "G1Errou.ipynb".
The first task is:
Viu?
A porcentagem divulgada pela Sabesp é falsa, pois faz de conta que a capacidade máxima do cantareira são os mesmos 980 bi L anteriores à aquisição das reservas técnicas. (A capacidade atual do sistema é 1.3 tri L!!!)
Can you write Python code for it?

dadosCantareira = pd.DataFrame(pd.concat([df.Cantareira, dFixed2.Cantareira, dFixed.Cantareira], axis=1))

titles = ['$\%$ divulgada pela Sabesp ($p$)',
    '$ Vol_{atual} = p \\times (Vol_{max} +Vol_{morto})$',
    '$ Vol_{atual} = p \\times Vol_{max}$']
dadosCantareira.columns = titles
#dadosCantareira.index = pd.to_datetime(dadosCantareira.index)
from sabesPy import reverseDate
dadosCantareira.index = map(reverseDate, dadosCantareira.index)

sns.set_context("poster")
ax = dadosCantareira.plot(kind='bar', ylim=(-25,30), yticks=range(-25,30,5), rot=20, title='Sistema Cantareira\n')
ax.set_ylabel("$\%$", fontsize=30)
ax.text(2.5, -10, r" $ \% =\frac{Vol_{atual} - Vol_{morto}}{Vol_{max}}  $", fontsize=30, ha='right')
plt.show()


I am working on the file "G1Errou.ipynb".
The first task is:
o cantareira tem capacidade total de quase 1 trilhão de litros, segundo a matéria do G1.
Então, entre os dias 15 e 16 de março, POOF: 180 bilhões de litros surgiram num passe de mágica!
Depois, em outubro, POOF. Surgem mais 100 bilhões.
QUE BRUXARIA É ESSA?!?
O próprio site da sabesp esclarece:

A primeira reserva técnica entrou em operação em 16/05/2014 e acrescentou mais 182,5 bilhões de litros ao sistema - 18,5% de acréscimo;

A segunda reserva técnica entrou em operação em 24/10/2014 e acrescentou mais 105,4 bilhões de litros ao sistema - 10,7% de acréscimo

Ou seja, o percentual divulgado pela sabesp não é corrigido para descontar o volume morto e, portanto, comparar 14 de março desse ano com o ano passado não faz sentido.
O G1 errou. Mas o quanto errou? Será que já, pelo menos, pagamos nossa "dívida" com o volume morto?
corrigindo o percentual do cantareira
Simples assim:
$$ Vol_1 = p \times Vol_0 $$
$$ Vol_{corrigido} = Vol_1 - Vol_{morto}$$
$$ p_{corrigido} = \frac{Vol_{corrigido}}{Vol_0}  $$

             $Vol_1$
         o volume atual do sistema

             $Vol_0$
          a capacidade máxima do reservatório (sem contar o volume morto)

             $Vol_{corrigido}$
          volume atual $-$ volume morto)

           $Vol_{morto}$
         advinha!

          $p$
          o percentual divulgado pela sabesp

        $p_{corrigido}$
         o percentual corrigido

essa correção foi implementada no "módulo" sabesPy
Can you write Python code for it?

from sabesPy import fixPerc
dFixed = df.copy()
dFixed.Cantareira = ([fixPerc(p, dia) for p, dia in zip(df.Cantareira, df.index)])

dados = dFixed.ix[['2014-05-15','2014-05-16']], dFixed.ix[['2014-10-23','2014-10-24']]
plotSideBySide(dados,
    titles=['1$^a$ reserva técnica\n(percentuais corrigidos)',
            '2$^a$ reserva técnica\n(percentuais corrigidos)'])


I am working on the file "GA-Python.ipynb".
The first task is:
Here is where we define the problem and choose an optimizer.  Various options exist, I've set a few, see documentation for details.
Can you write Python code for it?

from pyoptsparse import NSGA2

# choose optimizer and define options
optimizer = NSGA2()
optimizer.setOption('maxGen', 200)
optimizer.setOption('PopSize', 40)
optimizer.setOption('pMut_real', 0.01)
optimizer.setOption('pCross_real', 1.0)


I am working on the file "GA-Python.ipynb".
The first task is:
NSGA, like many genetic algorithms, doesn't have any speicific convergence criteria other than the maximum number of generations.  I set it at 200 in this case.  Notice that the answer is ok, but not super great.
Let's also try with SNOPT and start fairly far away (and I won't supply gradients):
Can you write Python code for it?

from pyoptsparse import SNOPT

optimizer = SNOPT()

xopt, fopt, info = optimize(rosen, x0, lb, ub, optimizer)
print 'results:', xopt, fopt, info


I am working on the file "GA-Python.ipynb".
The first task is:
Now we can run the optimizer and parse the results.
Can you write Python code for it?

from pyoptwrapper import optimize

x0 = [4.0, 4.0]
lb = [-5.0, -5.0]
ub = [5.0, 5.0]

xopt, fopt, info = optimize(rosen, x0, lb, ub, optimizer)
print 'results:', xopt, fopt, info


I am working on the file "GAFE tutorial.ipynb".
The first task is:
If, we didnt use GAFE, after testing 49 (7*7) combinations of FE-ES this configuration would be the best for each classifier. Note we are applying the same FE-ES pair for every data column
Can you write Python code for it?

bestSingleMatch = {'knn': [(1,5) for x in range(4)], 'cart': [(3,2) for x in range(4)], 'svm': [(7,4) for x in range(4)]}


I am working on the file "GAFE tutorial.ipynb".
The first task is:
Import modules from scikit-learn, numpy and pandas to help us deal with the data
Can you write Python code for it?

from sklearn.preprocessing import MinMaxScaler
import numpy as np
import pandas as pd


I am working on the file "GAFE tutorial.ipynb".
The first task is:
Load data using pandas. We will use the famous Iris Dataset
Can you write Python code for it?

iris = pd.read_csv("data/iris.data", sep=",")
#Isolate the attributes columns
irisAtts = iris.drop("class", 1)
#Isolate the class column
target = iris["class"]


I am working on the file "GAFE tutorial.ipynb".
The first task is:
Now lets calculate the accuracy results for original data, single match and GAFE.
Can you write Python code for it?

functionalExp = FE()

for cl in ['knn', 'cart', 'svm']:
        #Folds are the number of folds used in crossvalidation
        #Jobs are the number of CPUS used in crossvalidation and some classifiers training step.
        #You can also change some classifier parameters, such as k_neigh for neighbors in knn, C in svm and others.
        #If you do not specify, it will use the articles default.
        model = Classifier(cl, target, folds=10, jobs=6)
        #The class internally normalizes data, so no need to send normalized data when classifying
        #accuracy without expanding
        print("original accuracy " + cl + " " + str(model.getAccuracy(irisAtts)))
        #Expand the scaled data
        expandedData = functionalExp.expandMatrix(scaledIris, bestSingleMatch[cl])
        print("single match expansion accuracy " + cl + " " + str(model.getAccuracy(expandedData)))
        #If scaled is False, it will scale data in range [0,1]
        gafe = ga.GAFE(model, scaledIris, target, scaled=True)
        #Specify how many iterations of GAFE you wish with n_iter
        #Note that this is a slow method, so have patience if n_iter is high
        avg, bestPair = gafe.runGAFE(n_population=21, n_iter=1, verbose=True)
        print("gafe " + cl + " " + str(avg) )


I am working on the file "GAFE tutorial.ipynb".
The first task is:
Prior to expanding the data, put all values to interval [0,1] for better results
Can you write Python code for it?


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
All BPT settings
X-axis
Can you write Python code for it?

xbpt = np.log10(nii_flux_bpt/h_alpha_flux_bpt)

xbpt_uvup = np.log10(nii_flux_bpt[idx_uvup_bpt]/h_alpha_flux_bpt[idx_uvup_bpt])
xbpt_uvwk = np.log10(nii_flux_bpt[idx_uvwk_bpt]/h_alpha_flux_bpt[idx_uvwk_bpt])
xbpt_rsf  = np.log10(nii_flux_bpt[idx_rsf_bpt]/h_alpha_flux_bpt[idx_rsf_bpt])


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
All WHAN settings
X-axis
Can you write Python code for it?

xwhan = np.log10(nii_flux_whan/h_alpha_flux_whan)

xwhan_uvup = np.log10(nii_flux_whan[idx_uvup_whan]/h_alpha_flux_whan[idx_uvup_whan])
xwhan_uvwk = np.log10(nii_flux_whan[idx_uvwk_whan]/h_alpha_flux_whan[idx_uvwk_whan])
xwhan_rsf  = np.log10(nii_flux_whan[idx_rsf_whan]/h_alpha_flux_whan[idx_rsf_whan])


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Analysing the impact of the cross-correlation between BPT, WHAN and UV classes
Can you write Python code for it?

print xbpt.size
print xwhan.size

whan_class = []
for i in range(xwhan.size):
    if (xwhan[i]<-0.4)*(ywhan[i]>0.5):
        whan_class_i = 'SF'
    elif (xwhan[i]>-0.4)*(ywhan[i]>0.82):
        whan_class_i = 'sAGN'
    elif (xwhan[i]>-0.4)*(ywhan[i]<0.82)*(ywhan[i]>0.5):
        whan_class_i = 'wAGN'
    elif (ywhan[i]<0.5):
        whan_class_i = 'Retired/Passive'
    else:
        print 'error'

    whan_class.append(whan_class_i)
whan_class = np.array(whan_class)

idx_sf   = np.where(whan_class=='SF')
idx_sagn = np.where(whan_class=='sAGN')
idx_wagn = np.where(whan_class=='wAGN')
idx_rp   = np.where(whan_class=='Retired/Passive')


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Characterizing the emission-lines profile of these objects
BPT
Can you write Python code for it?

xbpt_k01 = np.linspace(-2.2, 0.4, 1000)     # Kewyley et al. 2001
xbpt_k03 = np.linspace(-2.2, -0.01, 1000)   # Kaufmann et al. 2003
xbpt_s06 = np.linspace(-2.2, -0.05, 1000)   # Stasinska et al. 2006
xbpt_s07 = np.linspace(-0.182, 2.0, 1000)   # Schawinski et al. 2007

ybpt_k01 = []
for j in range(len(xbpt_k01)):
    ybpt_k01j = 0.61 / (xbpt_k01[j] - 0.47) + 1.19  # Kewyley et al. 2001
    ybpt_k01.append(ybpt_k01j)
ybpt_k01 = np.array(ybpt_k01)

ybpt_k03 = []
for j in range(len(xbpt_k03)):
    ybpt_k03j = 0.61 / (xbpt_k03[j] - 0.05) + 1.3  # Kaufmann et al. 2003
    ybpt_k03.append(ybpt_k03j)
ybpt_k03 = np.array(ybpt_k03)

ybpt_s06 = []
for j in range(len(xbpt_s06)):
    # Stasinska et al. 2006
    ybpt_s06j = (-30.787 + (1.1358 * xbpt_s06[j]) + 0.27297) * np.tanh(5.7409 * xbpt_s06[j]) - 31.093
    ybpt_s06.append(ybpt_s06j)
ybpt_s06 = np.array(ybpt_s06)

ybpt_s07 = []
for j in range(len(xbpt_s07)):
    ybpt_s07j = 1.05 * xbpt_s07[j] + 0.45    # Schawinski et al. 2007
    ybpt_s07.append(ybpt_s07j)
ybpt_s07 = np.array(ybpt_s07)


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Creating new arrays of clean h alpha, h beta, nii, oiii to simplify the notation
BPT
Can you write Python code for it?

h_alpha_flux_bpt = h_alpha_flux[index_all][idx_bpt_clean]
h_beta_flux_bpt  = h_beta_flux[index_all][idx_bpt_clean]
nii_flux_bpt     = nii_flux[index_all][idx_bpt_clean]
oiii_flux_bpt    = oiii_flux[index_all][idx_bpt_clean]
uv_class_bpt     = uv_class[index_all][idx_bpt_clean]

print h_alpha_flux_bpt.size
print uv_class_bpt.size

print np.unique(uv_class_bpt)

idx_uvup_bpt = np.where(uv_class_bpt=='UV_UPTURN')

idx_uvwk_bpt = np.where(uv_class_bpt=='UV_WEAK')

idx_rsf_bpt = np.where(uv_class_bpt=='RSF')


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Estimating the amount of objects in each category of the BPT diagram
Can you write Python code for it?

bpt_class = []
idx_co = []
idx_sf = []
idx_sy = []
idx_ln = []

for i in range(xbpt.size):
    # checking the proximity of each observation to each curve
    idx_k01   = np.abs(xbpt_k01-xbpt[i]).argmin()  # index of the K01 curve closest to my observation i
    idx_s06   = np.abs(xbpt_s06-xbpt[i]).argmin()  # index of the S06 curve closest to my observation i
    idx_s07   = np.abs(xbpt_s07-xbpt[i]).argmin()  # index of the S07 curve closest to my observation i

    if (ybpt[i]>ybpt_k01[idx_k01])*(ybpt[i]>ybpt_s07[idx_s07]):
        idx_sy.append(i)
        bpt_class.append('Seyfert')
    elif (ybpt[i]>ybpt_k01[idx_k01])*(ybpt[i]<ybpt_s07[idx_s07]):
        idx_ln.append(i)
        bpt_class.append('LINER')
    elif (ybpt[i]<ybpt_k01[idx_k01])*(ybpt[i]>ybpt_s06[idx_s06]):
        idx_co.append(i)
        bpt_class.append('Composite')
    else:
        idx_sf.append(i)
        bpt_class.append('SF')

bpt_class = np.array(bpt_class)
print bpt_class.size

plot_co  = plt.scatter(xbpt[idx_co], ybpt[idx_co], c='#a6611a', s=10, alpha=1)
plot_sf  = plt.scatter(xbpt[idx_sf], ybpt[idx_sf], c='green', s=10, alpha=0.8)
plot_sy  = plt.scatter(xbpt[idx_sy], ybpt[idx_sy], c='blue', s=10, alpha=0.8)
plot_ln  = plt.scatter(xbpt[idx_ln], ybpt[idx_ln], c='magenta', s=10, alpha=0.8)
# plot_na  = plt.scatter(xbpt[idx_na], ybpt[idx_na], c='red', s=10, alpha=1)
plot02, = plt.plot(xbpt_k01, ybpt_k01, ':', color='black',  label='Kewley+01')
plot03, = plt.plot(xbpt_k03, ybpt_k03, '-', color='black',  label='Kauffman+03')
plot04, = plt.plot(xbpt_s06, ybpt_s06, '-.', color='black', label='Stasinska+06')
plot05, = plt.plot(xbpt_s07, ybpt_s07, '--', color='black', label='Schawinski+07')
plt.xlim(bpt_xlim)
plt.ylim(bpt_ylim)
plt.minorticks_on()
plt.tick_params('both', labelsize='15')
plt.xticks(np.arange(bpt_xlim[0], bpt_xlim[1], 1.))
plt.grid(alpha=0.0)
plt.show()


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
I now have both classifications: UV and WHAN. Let's crossmatch these informations and count the amount of galaxies in each situation.
Can you write Python code for it?

print r"RSF objects in SF region in WHAN diagram is %d" % list(uv_class_whan[idx_sf]).count('RSF')
print r"UV weak objects in SF region in WHAN diagram is %d" % list(uv_class_whan[idx_sf]).count('UV_WEAK')
print r"UV upturn objects in SF region in WHAN diagram is %d" % list(uv_class_whan[idx_sf]).count('UV_UPTURN')

print r"RSF objects in sAGN region in WHAN diagram is %d" % list(uv_class_whan[idx_sagn]).count('RSF')
print r"UV weak objects in sAGN region in WHAN diagram is %d" % list(uv_class_whan[idx_sagn]).count('UV_WEAK')
print r"UV upturn objects in sAGN region in WHAN diagram is %d" % list(uv_class_whan[idx_sagn]).count('UV_UPTURN')

print r"RSF objects in wAGN region in WHAN diagram is %d" % list(uv_class_whan[idx_wagn]).count('RSF')
print r"UV weak objects in wAGN region in WHAN diagram is %d" % list(uv_class_whan[idx_wagn]).count('UV_WEAK')
print r"UV upturn objects in wAGN region in WHAN diagram is %d" % list(uv_class_whan[idx_wagn]).count('UV_UPTURN')

print r"RSF objects in Retired/Passive region in WHAN diagram is %d" % list(uv_class_whan[idx_rp]).count('RSF')
print r"UV weak objects in Retired/Passive region in WHAN diagram is %d" % list(uv_class_whan[idx_rp]).count('UV_WEAK')
print r"UV upturn objects in Retired/Passive region in WHAN diagram is %d" % list(uv_class_whan[idx_rp]).count('UV_UPTURN')


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Loading Dataset
Can you write Python code for it?

my_data = np.loadtxt('../Catalogue/Match07_small_mags_doubleclean_type.csv', delimiter=',', dtype=str)

my_dictionary = {}
for i in range(len(my_data[0, :])):                                         # Converting numpy array into dictionary
    my_dictionary[my_data[0, i]] = np.array(my_data[0 + 1:, i], dtype=str)

print my_data.shape
print my_data.size
print my_data[0,:]

redshift     = my_dictionary['Z'].astype(float)
cataid       = my_dictionary['CATAID'].astype(str)
z_prob       = my_dictionary['PROB'].astype(float)
z_quality    = my_dictionary['NQ'].astype(int)
fuv_band     = my_dictionary['MAG_AB_FUV'].astype(float)
nuv_band     = my_dictionary['MAG_AB_NUV'].astype(float)
u_band       = my_dictionary['MAG_AB_U'].astype(float)
g_band       = my_dictionary['MAG_AB_G'].astype(float)
r_band       = my_dictionary['MAG_AB_R'].astype(float)
mag_abs_r    = my_dictionary['MAG_ABSOLUTE_R'].astype(float)
stellar_mass = my_dictionary['logmstar'].astype(float)   # stellar mass from sed fitting - log scale
stellar_age  = my_dictionary['logage'].astype(float)     # stellar age - log scale
stellar_met  = my_dictionary['metal'].astype(float)      # stellar metallicity
dn4000       = my_dictionary['D4000N'].astype(float)
h_alpha_flux = my_dictionary['HA_FLUX_COMP'].astype(float)
h_alpha_ew   = my_dictionary['HA_EW_COMP'].astype(float)
h_beta_flux  = my_dictionary['HB_FLUX_COMP'].astype(float)
h_beta_ew    = my_dictionary['HB_EW_COMP'].astype(float)
nii_flux     = my_dictionary['NIIR_FLUX_COMP'].astype(float)  # R for red or 6583A -- see http://www.gama-survey.org/dr3/schema/dmu.php?id=8
oiii_flux    = my_dictionary['OIIIR_EW_COMP'].astype(float)   # R for red or 5007A -- see http://www.gama-survey.org/dr3/schema/dmu.php?id=8
uv_class     = my_dictionary['UV_CLASS_YI2011'].astype(str)
obj_type     = my_dictionary['TYPE'].astype(int)

print np.unique(uv_class)

print my_data[:,0].shape


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Now we are going to add these information into the whole dataset, in case we need it in the future
Can you write Python code for it?

bpt_catai_temp1 = pd.DataFrame(cataid[index_all][idx_bpt_clean])
bpt_class_temp1 = pd.DataFrame(bpt_class)
bpt_pos = pd.concat([bpt_catai_temp1, bpt_class_temp1], axis=1)
bpt_pos.columns = ['CATAID', 'BPT_CLASS']

bpt_catai_temp2 = pd.DataFrame(cataid[index_all][idx_not_bpt])
bpt_class_temp2 = pd.DataFrame(emlines_not_bpt)
bpt_neg = pd.concat([bpt_catai_temp2, bpt_class_temp2], axis=1)
bpt_neg.columns = ['CATAID', 'BPT_CLASS']

bpt_all = pd.concat([bpt_pos, bpt_neg])

print np.unique(bpt_all['CATAID']).size

whan_catai_temp1 = pd.DataFrame(cataid[index_all][idx_whan_clean])
whan_class_temp1 = pd.DataFrame(whan_class)
whan_pos = pd.concat([whan_catai_temp1, whan_class_temp1], axis=1)
whan_pos.columns = ['CATAID', 'WHAN_CLASS']

whan_catai_temp2 = pd.DataFrame(cataid[index_all][idx_not_whan])
whan_class_temp2 = pd.DataFrame(emlines_not_whan)
whan_neg = pd.concat([whan_catai_temp2, whan_class_temp2], axis=1)
whan_neg.columns = ['CATAID', 'WHAN_CLASS']

whan_all = pd.concat([whan_pos, whan_neg])

print np.unique(whan_all['CATAID']).size

my_df_temp = pd.DataFrame(my_data[index_all+1])
my_df_temp.columns = my_data[0,:]

my_df_temp2 = my_df_temp.set_index('CATAID').join(bpt_all.set_index('CATAID')).join(whan_all.set_index('CATAID'))

my_df_temp2['BPT_CLASS'].value_counts(dropna=False)

my_df_temp2['WHAN_CLASS'].value_counts(dropna=False)

my_df_temp2.to_csv('../Catalogue/Match07_smalldoubleclean_emlines.csv', index=True)


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Plot 01: Yi et al. (2011) UV diagnosis plot
Can you write Python code for it?

sns.set_style("whitegrid")
plt.rcParams["axes.edgecolor"] = "0.15"
plt.rcParams["axes.linewidth"] = 1.
plt.subplots(1,1, figsize=(8,5))
plot01, = plt.plot((nuv_band - r_band)[index_rsf], (fuv_band - nuv_band)[index_rsf], 'o', markersize=6,
                   color = '#018571', alpha=0.5, label="RSF")
plot02, = plt.plot((nuv_band - r_band)[index_uvweak], (fuv_band - nuv_band)[index_uvweak], 's',
                   markersize=6, color = '#dfc27d', alpha=0.5, label="UV Weak")
plot03, = plt.plot((nuv_band - r_band)[index_uvup], (fuv_band - nuv_band)[index_uvup], 'D', markersize=6,
                   color = '#a6611a', alpha=0.5, label="UV upturn")
plt.legend(numpoints=1, loc='best', fontsize=14, frameon=True, framealpha=0.85)
# plt.text(0.0, 4, r"RSF", fontsize=18)
# plt.text(7.5, 4, r"UV Weak", fontsize=18)
# plt.text(7.5, -1.8, r"UV upturn", fontsize=18)
plt.axvline(x=5.4, color='black', linewidth=2.)
plt.axhline(y=0.9, xmin=0.659, xmax=2, color='black', linewidth=2.)
plt.xlabel("NUV-r", fontsize=15)
plt.ylabel("FUV-NUV", fontsize=15)
plt.tick_params('both', labelsize='14')
plt.grid(alpha=0.00)
plt.savefig('../Figs/g2_maglim_yi_diagram.pdf')
plt.savefig('../Figs/g2_maglim_yi_diagram.png')
plt.show()


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Plot 02: BPT and WHAN
Can you write Python code for it?

# INITIAL SETTINGS
plt.rcParams["axes.edgecolor"] = "0.15"
plt.rcParams["axes.linewidth"] = 1.
plt.subplots(figsize=(10.5,5))

# FIRST PLOT -- BPT
plt.subplot(1,2,1)
plot01c = plt.scatter(xbpt_rsf, ybpt_rsf,   c='#018571', s=5, alpha=0.3, marker='o')
plot01b = plt.scatter(xbpt_uvwk, ybpt_uvwk, c='#dfc27d', s=10, alpha=1, marker='s')
plot01a = plt.scatter(xbpt_uvup, ybpt_uvup, c='#a6611a', s=20, alpha=1, marker='D')
plot02, = plt.plot(xbpt_k01, ybpt_k01, ':', color='black',  label='Kewley+01')
plot03, = plt.plot(xbpt_k03, ybpt_k03, '-', color='black',  label='Kauffman+03')
plot04, = plt.plot(xbpt_s06, ybpt_s06, '-.', color='black', label='Stasinska+06')
plot05, = plt.plot(xbpt_s07, ybpt_s07, '--', color='black', label='Schawinski+07')
l1 = plt.legend([plot01a, plot01b, plot01c], [r"UV Upturn", r"UV Weak", r"RSF"], numpoints=1, loc='lower left',
           fontsize=12, frameon=True, framealpha=0.85)
l2 = plt.legend([plot02, plot03, plot04, plot05], [r"Kewley+01", r"Kauffman+03", r"Stasińska+06", r"Schawinski+07"],
          numpoints=3, loc='lower right', fontsize=12, frameon=True, framealpha=0.85)
l1.get_frame().set_edgecolor('black')
l2.get_frame().set_edgecolor('black')
plt.gca().add_artist(l1)
plt.fill_betweenx(ybpt_k01, xbpt_k01, xbpt_s06, where=(xbpt_s06>-1.242)*(ybpt_s06<0.835), facecolor='gray', alpha=0.2)
plt.fill_between(xbpt_s06, ybpt_k01, ybpt_s06, where=(xbpt_s06>-1.242)*(ybpt_k01<0.835), facecolor='gray', alpha=0.2)
plt.xlabel(r"$\log ([NII]/H{\alpha})$", fontweight='bold', size=14)
plt.ylabel(r"$\log (\left[OIII\right]/H \beta) $", fontweight='bold', fontsize=14)
plt.text(-1.9, 0.2, r"Star Forming", fontsize=16)
# plt.text(-0.5, 1.6, r"AGN", fontsize=13)
plt.text(-0.5, 1.3, r"Seyfert", fontsize=13)
plt.text(0.9, -1, r"LINER", fontsize=13)
plt.annotate(r"Composite", xy=(0., -2), xycoords='data', xytext=(0.5, 0.5), size=13,
             arrowprops=dict(arrowstyle='wedge',facecolor='black', connectionstyle="angle3,angleA=90,angleB=0"))
plt.xlim([-2, 1.5])
plt.ylim([-5,...


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Plot 03: BPT and WHAN fragmented
Can you write Python code for it?

# OVERALL SETTINGS
plt.subplots(figsize=(8,5))
plt.rcParams["axes.edgecolor"] = "0.15"
plt.rcParams["axes.linewidth"] = 1.
bpt_xlim  = [-2, 1.5]
bpt_ylim  = [-5, 2]
whan_xlim = [-2, 1.5]
whan_ylim = [-1.0, 3.5]

# BPT
## RSF
ax1=plt.subplot(2,3,1)
plot01c = plt.scatter(xbpt_rsf, ybpt_rsf, c='#018571', s=10, alpha=0.7, label='RSF')
plot02, = plt.plot(xbpt_k01, ybpt_k01, ':', color='black',  label='Kewley+01')
plot03, = plt.plot(xbpt_k03, ybpt_k03, '-', color='black',  label='Kauffman+03')
plot04, = plt.plot(xbpt_s06, ybpt_s06, '-.', color='black', label='Stasinska+06')
plot05, = plt.plot(xbpt_s07, ybpt_s07, '--', color='black', label='Schawinski+07')
plt.ylabel(r"$\log (\left[OIII\right]/H \beta) $", fontweight='bold', fontsize=15)
# plt.legend([plot01c], [r"RSF"], numpoints=500, loc='upper right', fontsize=10, frameon=True, framealpha=1.)
plt.title("RSF", fontsize=14)
plt.xlim(bpt_xlim)
plt.ylim(bpt_ylim)
plt.minorticks_on()
plt.tick_params('both', labelsize='14')
plt.xticks(np.arange(bpt_xlim[0], bpt_xlim[1], 1.))
plt.yticks(np.arange(bpt_ylim[0], bpt_ylim[1], 2))
plt.grid(alpha=0.0)

## UV Weak
ax2=plt.subplot(2,3,2)
plot01b = plt.scatter(xbpt_uvwk, ybpt_uvwk, c='#dfc27d', s=10, alpha=1, label='UV weak')
plot02, = plt.plot(xbpt_k01, ybpt_k01, ':', color='black',  label='Kewley+01')
plot03, = plt.plot(xbpt_k03, ybpt_k03, '-', color='black',  label='Kauffman+03')
plot04, = plt.plot(xbpt_s06, ybpt_s06, '-.', color='black', label='Stasinska+06')
plot05, = plt.plot(xbpt_s07, ybpt_s07, '--', color='black', label='Schawinski+07')
# plt.xlabel(r"$\log ([NII]/H{\alpha})$", fontweight='bold', size=19)
plt.title("UV weak", fontsize=14)
plt.xlim(bpt_xlim)
plt.ylim(bpt_ylim)
ax2.yaxis.set_visible(False)
plt.minorticks_on()
plt.tick_params('both', labelsize='14')
plt.xticks(np.arange(bpt_xlim[0], bpt_xlim[1], 1.))
plt.grid(alpha=0.0)

## UV Upturn
ax3=plt.subplot(2,3,3)
plot01a = plt.scatter(xbpt_uvup, ybpt_uvup, c='#a6611a', s=10, alpha=1, label='UV upturn')
plot02, = plt.plot(xbpt_k01, ybpt_k01, ':', color='black', ...


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Selecting the subsample
Can you write Python code for it?

plt.hist(z_prob, bins=200)
plt.yscale('log')
plt.show()

indexes = np.arange(redshift.size)

index_all = indexes[(r_band>0)*(r_band<19.8)*(nuv_band>0)*(fuv_band>0)*((fuv_band-nuv_band)<50)
                     *((fuv_band-nuv_band)>(-20))*(redshift>=z_min)*(z_prob>0.8)*(obj_type==3)]

print my_data[index_all].shape

print (np.unique(cataid[index_all])).size


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
WHAN
Can you write Python code for it?

h_alpha_flux_whan = h_alpha_flux[index_all][idx_whan_clean]
h_beta_flux_whan  = h_beta_flux[index_all][idx_whan_clean]
nii_flux_whan     = nii_flux[index_all][idx_whan_clean]
h_alpha_ew_whan   = h_alpha_ew[index_all][idx_whan_clean]
uv_class_whan     = uv_class[index_all][idx_whan_clean]

print h_alpha_flux_whan.size
print uv_class_whan.size

print np.unique(uv_class_whan)

idx_uvup_whan = np.where(uv_class_whan=='UV_UPTURN')

idx_uvwk_whan = np.where(uv_class_whan=='UV_WEAK')

idx_rsf_whan = np.where(uv_class_whan=='RSF')


I am working on the file "GAMAII/Coding/CatAnalysis_Part01.ipynb".
The first task is:
Y-axis
Can you write Python code for it?

ybpt = np.log10(oiii_flux_bpt/h_beta_flux_bpt)

ybpt_uvup = np.log10(oiii_flux_bpt[idx_uvup_bpt]/h_beta_flux_bpt[idx_uvup_bpt])
ybpt_uvwk = np.log10(oiii_flux_bpt[idx_uvwk_bpt]/h_beta_flux_bpt[idx_uvwk_bpt])
ybpt_rsf  = np.log10(oiii_flux_bpt[idx_rsf_bpt]/h_beta_flux_bpt[idx_rsf_bpt])


ywhan = np.log10(h_alpha_ew_whan)

ywhan_uvup = np.log10(h_alpha_ew_whan[idx_uvup_whan])
ywhan_uvwk = np.log10(h_alpha_ew_whan[idx_uvwk_whan])
ywhan_rsf  = np.log10(h_alpha_ew_whan[idx_rsf_whan])

print xbpt_rsf.size, ybpt_rsf.size
print xwhan_rsf.size, ywhan_rsf.size

print xwhan.size
print (xwhan_uvup.size+xwhan_uvwk.size+xwhan_rsf.size)


I am working on the file "GAN/GAN 101.ipynb".
The first task is:
Define model
Remarks
Initialize the variables with caution
In the two loss function defined below, singulars will occur in case the output of the network contains $1$ or $0$, given that the loss function contains the term $\log(x)$ or $\log(1-x)$. In our case the output layer of both generator and discriminator network contains a simgoid function. In $y=sigmoid(wx+b)$, if the weights are too large then the result of this term quickly approaches $0$ or $1$ as $x$ moves away from $0$. Hence the mean and stddev of weight distribution at the output layer are set to $0$ a relatively small value, respectively, to opt out the occurence of singulars in loss functions.
Can you write Python code for it?

class GAN:
    def __init__(self, noise_input_tensor, image_input_tensor, generator_hidden_dim, discriminator_hidden_dim):
        self._noise_input = noise_input_tensor
        self._image_input = image_input_tensor
        with tf.variable_scope("generator"):
            self._generator_output, self._generator_parameters = self._fnn(
                noise_input_tensor, generator_hidden_dim, image_input_tensor.shape[1],
                activation=lambda x: 255 * tf.sigmoid(x)) # (0, 1) -> (0.0, 255.0)
        with tf.variable_scope("discriminator"):
            self._discriminator_output_for_real_data, self._discriminator_parameters = self._fnn(
                image_input_tensor, discriminator_hidden_dim, 1, activation=tf.sigmoid)
        with tf.variable_scope("discriminator", reuse=True):  # Share weights and biases
            d_o_fake = self._discriminator_output_for_synth, _ = self._fnn(self._generator_output,
                                                                           discriminator_hidden_dim,
                                                                           1,
                                                                           activation=tf.sigmoid)

    def _fnn(self, input_tensor, hidden_dim, output_dim, activation=None):
        w_xh = tf.get_variable(initializer=tf.truncated_normal_initializer(mean=0.0, stddev=0.01),
                            shape=[input_tensor.shape[1], hidden_dim], name="W_xh")
        b_xh = tf.get_variable(initializer=tf.truncated_normal_initializer(mean=0.0, stddev=0.01),
                            shape=[hidden_dim], name="b_xh")
        hidden = tf.nn.relu(tf.add(tf.matmul(input_tensor, w_xh), b_xh))
        w_ho = tf.get_variable(initializer=tf.truncated_normal_initializer(mean=0.0, stddev=0.01),
                            shape=[hidden_dim, output_dim], name="W_ho")
        b_ho = tf.get_variable(initializer=tf.truncated_normal_initializer(mean=0.0, stddev=0.01),
                            shape=[output_dim],...


I am working on the file "GAN/GAN 101.ipynb".
The first task is:
Prepare Dataset and Start Training
Can you write Python code for it?

with tf.variable_scope("training_configuration", auxiliary_name_scope=False): # Re-entering the name scope
    batch_size_t = tf.placeholder(dtype=tf.int64, shape=[], name="batch_size")
    training_set = tf.data.Dataset.from_tensor_slices((x_train, y_train))
    training_set = training_set.batch(batch_size=batch_size_t).map(
        lambda x, y: (tf.reshape(tensor=x, shape=[-1, 28 * 28]), y))
    batch_iter_train = training_set.make_initializable_iterator()
    next_batch_train = batch_iter_train.get_next()

n_epoch = 100
batch_size = 50
g_lr, d_lr = 2e-5, 2e-5
lr_decay = 0.97
num_batch = int(x_train.shape[0]/batch_size)
k = 1
model_param_path = os.path.normpath(r"./model_checkpoints")
saver=tf.train.Saver()
def batch_sampler(batch_size):
    return np.random.uniform(-5.0, 5.0, size=[batch_size, noise_dim])
with tf.Session() as sess, tf.summary.FileWriter(logdir=model_param_path) as writer:
    writer.add_graph(graph=tf.get_default_graph())
    sess.run(tf.global_variables_initializer())
    for epoch in range(n_epoch):
        sess.run([tf.assign(g_learing_rate_tensor, g_lr), tf.assign(d_learing_rate_tensor, d_lr)])
        sess.run(batch_iter_train.initializer, feed_dict={batch_size_t: batch_size})
        for batch_no in range(num_batch):
            x, _ = sess.run(next_batch_train)
            # print(x.shape)
            # x = x.reshape([-1, image_dim]) # flatten each sample manually
            # Train the discriminator network k times
            for _ in range(k):
                noise_batch = batch_sampler(batch_size)
                feed_dict = {gan.image_input: x, gan.noise_input: noise_batch}
                sess.run(d_train_op, feed_dict=feed_dict)
            # Train the generator network once
            noise_batch = batch_sampler(batch_size)
            feed_dict = {gan.image_input: x, gan.noise_input: noise_batch}
            _, summary, step = sess.run([g_train_op, summary_all, global_step], feed_dict=feed_dict)
            writer.add_summary(summary=summary, global_step=step)
       ...


I am working on the file "GAN/GAN 101.ipynb".
The first task is:
Test Network
Can you write Python code for it?

n = 10
canvas = np.empty((28 * n, 28 * n))
with tf.Session() as sess:
    ckpt = tf.train.get_checkpoint_state(model_param_path)
    if ckpt and ckpt.model_checkpoint_path:
        saver.restore(sess, ckpt.model_checkpoint_path)
        for i in range(n):
            # Noise input.
            z = batch_sampler(n)
            # Generate image from noise.
            g = sess.run(gan.generator_output, feed_dict={gan.noise_input: z})
            # Reverse colours for better display
            # g = -1 * (g - 1)
            g = (-1 * (g - 255)).astype(np.int32)
            for j in range(n):
                # Draw the generated digits
                canvas[i * 28:(i + 1) * 28, j * 28:(j + 1) * 28] = g[j].reshape([28, 28])
        plt.figure(figsize=(n, n))
        plt.imshow(canvas, origin="upper", cmap="gray")
        plt.show()
    else:
        print("Failed to load model checkpoint.")


I am working on the file "GAN/GAN 101.ipynb".
The first task is:
Verify data strucutre
Can you write Python code for it?


I am working on the file "GAN/fashion_mnist_gan/fashion_mnist_gan_v2.ipynb".
The first task is:
Start the training
Can you write Python code for it?


I am working on the file "GAN/fashion_mnist_gan/fashion_mnist_gan_v2.ipynb".
The first task is:
You can run this cell below as a standalone

you need the weights file of this model :

you need the weights of the model that trained above

the files need to be like this:
- dcgan_generator.h5
- fashion-cnn-weights.best.hdf5
Can you write Python code for it?

from tensorflow.keras.models import load_model
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
from tensorflow import keras

LABEL_NAMES = ['t_shirt', 'trouser', 'pullover', 'dress', 'coat', 'sandal', 'shirt', 'sneaker', 'bag', 'ankle_boots']

PREDICT = 'trouser'

# Create a wall of generated images
def plot_images(imgs , dim=(5,5), figsize=(5, 5)):
    plt.figure(figsize=figsize)
    for i in range(imgs.shape[0]):
        plt.subplot(dim[0], dim[1], i+1)
        plt.imshow(imgs[i, 0], interpolation='nearest', cmap='gray_r')
        plt.axis('off')
    plt.tight_layout()
    plt.show()

label  = " "
runs =0
fashion_cnn =  keras.models.Sequential()
fashion_cnn.add(keras.layers.Reshape((28,28,1)))

generator  = load_model("dcgan_generator.h5")
model = load_model('fashion-cnn-weights.best.hdf5')
fashion_cnn.add(model)

#fashion_cnn = keras.models.Model(keras.layers.Reshape((28,28,1)), model.output)

while label != PREDICT:
  runs +=1
  noise = np.random.normal(0, 1, size=[1, 100]) # generate one image
  imgs = generator.predict(noise)
  score  = fashion_cnn.predict(imgs)
  indx  = list(score[0]).index(max(score[0]))
  #print(score[0])
  #print(indx)
  label = LABEL_NAMES[indx]
print("Runs:",runs)
print(label)
plot_images(imgs)


I am working on the file "GAN/fashion_mnist_gan/fashion_mnist_gan_v2.ipynb".
The first task is:
define helpfull methods
Can you write Python code for it?

import matplotlib.pyplot as plt

 # Plot the loss from each batch
def plot_loss(epoch,d_Losses,g_Losses):
    plt.figure(figsize=(10, 8))
    plt.plot(d_Losses, label='Discriminitive loss')
    plt.plot(d_Losses, label='Generative loss')
    plt.xlabel('Epoch')
    plt.ylabel('Loss')
    plt.legend()
    plt.savefig('dcgan_loss_epoch_%d.png' % epoch)

# Create a wall of generated images
def plot_generated_images(imgs , epoch, examples=25, dim=(5,5), figsize=(5, 5)):
    noise = np.random.normal(0, 1, size=[examples, 100])
    plt.figure(figsize=figsize)
    for i in range(imgs.shape[0]):
        plt.subplot(dim[0], dim[1], i+1)
        plt.imshow(imgs[i, 0], interpolation='nearest', cmap='gray_r')
        plt.axis('off')
    plt.tight_layout()
    name = "fashion_mnist_v2_{}.png".format(epoch)
    plt.savefig('./samples/fashion_mnist_v2/' + name)


I am working on the file "GBT/filterbank_tutorial/Filterbank Tutorial (public).ipynb".
The first task is:
As you can see, we get the frequencies in MHz in a 1-D numpy array.
Extracting Power Data
Now let's take a look at the object that contains the actual power values:
Can you write Python code for it?

data = fil.data
print data.shape
print type(data)


I am working on the file "GBT/filterbank_tutorial/Filterbank Tutorial (public).ipynb".
The first task is:
Header Information
We can use another function to easily print the header information from the file:
Can you write Python code for it?


I am working on the file "GBT/filterbank_tutorial/Filterbank Tutorial (public).ipynb".
The first task is:
Power Spectrum for a Given Integration
Now imagine you were to pull a horizontal slice of the waterfall plot out of the screen like pulling a card out of a deck, and flip it 90 degrees so that the edge of the card that is initially closest to you is now on top. You would then be looking at a power spectrum for one particular integration. It shows frequency on the x-axis and power on the y-axis. We can display a power spectrum using another function contained in blimpy. Let's choose to display the 100th integration (which would be indexed as 99 since the integration indices start at 0):
Can you write Python code for it?

plt.figure(figsize=(10,6))
fil.plot_spectrum(t=99) #Leaving arguments blank will cause it to show integration 0 by default


I am working on the file "GBT/filterbank_tutorial/Filterbank Tutorial (public).ipynb".
The first task is:
Testing out the extraction of a parameter:
Can you write Python code for it?


I am working on the file "GBT/filterbank_tutorial/Filterbank Tutorial (public).ipynb".
The first task is:
The power values are stored in a 3-D numpy array. The first dimension corresponds to the number of integrations in the file (notice from the header info above that "num ints in file:" reads 273). Meanwhile the third dimension corresponds to the number of frequency channels (notice in the header that "nchans:" reads 65536).
So, by picking a value along the zeroth axis (i.e. first dimension) of the array, I can extract the power values associated with that integration. The array of power values will have length 65536, since there is one power value associated with each channel.
For example, suppose I want to get the power values from integration 100 (index 99) as before. For this I simply do:
Can you write Python code for it?

powers100 = data[99]
print powers100
powers100 = powers100[0]
print powers100
print len(powers100)


I am working on the file "GBT/filterbank_tutorial/Filterbank Tutorial (public).ipynb".
The first task is:
The resultant dictionary contains all the same information as the one you obtain from self.header.
5. Miscellaneous Helpful Functions
Here are some miscellaneous functions that I have written during my time with Breakthrough Listen that you may find useful:
Can you write Python code for it?

"""
All these functions show the required modules imported within the function. This is simply to show as
clearly as possible which modules are required by each function. Of course, in general you should
import all modules at the top of your script.
"""

def maxfreq(file):
    """Return central frequency of the highest-frequency bin in a .fil file."""

    from blimpy import read_header
    return read_header(file)['fch1']

def minfreq(file):
    """Return central frequency of the lowest-frequency bin in a .fil file"""

    from blimpy import read_header
    fch1 = read_header(file)['fch1']
    nchans = read_header(file)['nchans']
    ch_bandwidth = read_header(file)['foff']
    return fch1 + nchans*ch_bandwidth

def currentMJD():
    """Returns current MJD including decimals."""

    from astropy.time import Time
    print Time.now().mjd

def AA(file):
    """Returns Alt-Az coordinates (deg) for a given GBT .fil file.

    Args:
        file (str): .fil file

    Returns:
        altazdict (dict): Dictionary with alt and az in degrees.
    """

    from astropy import units as u
    from astropy.time import Time
    from astropy.coordinates import EarthLocation, SkyCoord, AltAz
    from blimpy import read_header

    GreenBank = EarthLocation(lat=38.4322*u.deg, lon=-79.8398*u.deg) #Western longitudes are negative
    MJD = read_header(file)['tstart']
    ra = read_header(file)['src_raj']
    dec = read_header(file)['src_dej']
    target = SkyCoord(ra, dec)
    altaz = target.transform_to(AltAz(location=GreenBank, obstime=Time(MJD, format='mjd')))
    altazdict = { 'alt' : altaz.alt.degree, 'az' : altaz.az.degree }
    return altazdict

def totalpower(file, integration, fmin, fmax):

    """Use Simpson's rule numerical integration to find total power between
    given freq bounds, for given time integration.

    Args:
        file (str): .fil file
        integration (int): .fil file integration number to analyze
        fmin (float): Lower freq bound
       ...


I am working on the file "GBT/filterbank_tutorial/Filterbank Tutorial (public).ipynb".
The first task is:
There are many useful pieces of information here, which you wil probably use at some point or another. I'll explain the most important ones, but all the header keywords are detailed in blimpy:

source_name: The name of the target observed. In this case it is HIP35136, a star listed in the Hipparcos catalog.
tstart: The start date of the observation, in Modified Julian Date format
src_raj: The right ascension of the target in hours:minutes:seconds.
src_dej: The declination of the target in degrees:arcminutes:arcseconds.
nchans: The number of channels (i.e. frequency bins)
fch1: The center frequency of the first channel (i.e. the highest-frequency channel).
foff: The bandwidth of each channel, in MHz. It is negative because the channels are numbered in increasing order as they decrease in frequency.

4. Getting Your Hands on the Actual Data
Now that I've showed you how to visualize the data and print the header info, I'll show you how to extract the data so that you can use it for analysis purposes.
Extracting Frequency Bins
First let's get ahold of the values of the center frequencies of the channels. This is easy to do:
Can you write Python code for it?

freqs = fil.freqs
print freqs
print type(freqs)


I am working on the file "GBT/filterbank_tutorial/Filterbank Tutorial (public).ipynb".
The first task is:
Waterfall Plots
Now that we have done that, there are several things we can look at. Remember that a filterbank file shows how power varies with frequency and time. As such, it contains many time integrations (a time integration is just a period of time over which the light is collected), and each integration has a power spectrum associated with it. The data contained in the filterbank file therefore has three dimensions: frequency, power, and time. So, if I wanted to display all of the data contained in the filterbank file in one visualization, I would show it as a waterfall plot, like this:
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt
plt.figure(figsize=(10,6))
fil.plot_waterfall()