I am working on the file "Recurrent_Neural_Networks_1.1.0.ipynb".
The first task is:
Multi-layer RNN with Keras
When stacking RNNs with Keras remember to set return_sequences=True on hidden layers.
Can you write Python code for it?

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.layers import SimpleRNN
from keras import initializers
from keras.optimizers import RMSprop

from keras.models import Model
from keras.layers import Input, Dense

keras.backend.clear_session()

batch_size = 150
num_classes = 10
epochs = 50 # instead of 100 (too much time)
hidden_units_1 = 150
hidden_units_2 = 100

learning_rate = 0.001

# the data, shuffled and split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()

x_train = x_train.reshape(x_train.shape[0], 28, 28)
x_test = x_test.reshape(x_test.shape[0], 28, 28)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

print('Evaluate IRNN...')
a = Input(shape=x_train.shape[1:])
b = SimpleRNN(hidden_units_1,
                    kernel_initializer=initializers.RandomNormal(stddev=0.001),
                    recurrent_initializer=initializers.Identity(),
                    activation='relu' , return_sequences=True)(a)
b = SimpleRNN(hidden_units_2,
                    kernel_initializer=initializers.RandomNormal(stddev=0.001),
                    recurrent_initializer=initializers.Identity(),
                    activation='relu')(b)
b = Dense(num_classes)(b)
b = Activation('softmax')(b)
optimizer = keras.optimizers.Adamax(lr=learning_rate)
model = Model(inputs=[a], outputs=[b])
model.compile(loss='categorical_crossentropy',
              optimizer=optimizer,
              metrics=['accuracy'])

model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test, y_test))

scores =...


I am working on the file "Recurrent_Neural_Networks_1.1.0.ipynb".
The first task is:
Packing sequences
Can you write Python code for it?

tf.reset_default_graph()

n_steps = 2
n_inputs = 3
n_neurons = 5

X = tf.placeholder(tf.float32, [None, n_steps, n_inputs])
seq_length = tf.placeholder(tf.int32, [None])   ### <----------------------------------------

basic_cell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)
outputs, states = tf.nn.dynamic_rnn(basic_cell, X, sequence_length=seq_length, dtype=tf.float32)

init = tf.global_variables_initializer()

X_batch = np.array([
        # step 0     step 1
        [[0, 1, 2], [9, 8, 7]], # instance 1
        [[3, 4, 5], [0, 0, 0]], # instance 2 (padded with zero vectors)
        [[6, 7, 8], [6, 5, 4]], # instance 3
        [[9, 0, 1], [3, 2, 1]], # instance 4
    ])

seq_length_batch = np.array([2, 1, 2, 2])  ### <------------------------

with tf.Session() as sess:
    init.run()
    outputs_val, states_val = sess.run(
        [outputs, states], feed_dict={X: X_batch, seq_length: seq_length_batch})

print(outputs_val)

print(states_val)


I am working on the file "Recurrent_Neural_Networks_1.1.0.ipynb".
The first task is:
Recurrent convolutional network on the IMDB sentiment
Can you write Python code for it?

from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.layers import Embedding
from keras.layers import LSTM
from keras.layers import Conv1D, MaxPooling1D
from keras.datasets import imdb

del model
keras.backend.clear_session()

# Embedding
max_features = 20000
maxlen = 100
embedding_size = 128

# Convolution
kernel_size = 5
filters = 64
pool_size = 4

# LSTM
lstm_output_size = 70

# Training
batch_size = 30
epochs = 4

'''
Note:
batch_size is highly sensitive.
Only 2 epochs are needed as the dataset is very small.
'''

print('Loading data...')
(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
print(len(x_train), 'train sequences')
print(len(x_test), 'test sequences')

print('Pad sequences (samples x time)')
x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
x_test = sequence.pad_sequences(x_test, maxlen=maxlen)
print('x_train shape:', x_train.shape)
print('x_test shape:', x_test.shape)

print('Build model...')

model = Sequential()
model.add(Embedding(max_features, embedding_size, input_length=maxlen))
model.add(Dropout(0.25))
model.add(Conv1D(filters,
                 kernel_size,
                 padding='valid',
                 activation='relu',
                 strides=1))
model.add(MaxPooling1D(pool_size=pool_size))
model.add(LSTM(lstm_output_size))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

print('Train...')
model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          validation_data=(x_test, y_test))
score, acc = model.evaluate(x_test, y_test, batch_size=batch_size)
print('Test score:', score)
print('Test accuracy:', acc)


I am working on the file "Recurrent_Neural_Networks_1.1.0.ipynb".
The first task is:
This fails because a BasicRNNCell is a cell factory, not a cell per se; no cells get created when you create the factory, and thus no variables do either. The device block is simply ignored. The cells actually get created later. When you call dynamic_rnn(), it calls the MultiRNNCell, which calls each individual BasicRNNCell, which create the actual cells (including their variables). Unfortunately, none of these classes provide any way to control the devices on which the variables get created. If you try to put the dynamic_rnn() call within a device block, the whole RNN gets pinned to a single device.
The trick is to create your own cell wrapper
Can you write Python code for it?

import tensorflow as tf

class DeviceCellWrapper(tf.contrib.rnn.RNNCell):
  def __init__(self, device, cell):
    self._cell = cell
    self._device = device

  @property
  def state_size(self):
    return self._cell.state_size

  @property
  def output_size(self):
    return self._cell.output_size

  def __call__(self, inputs, state, scope=None):
    with tf.device(self._device):
        return self._cell(inputs, state, scope)

tf.reset_default_graph()

n_inputs = 5
n_neurons = 100
devices = ["/cpu:0"]*5
n_steps = 20
X = tf.placeholder(tf.float32, shape=[None, n_steps, n_inputs])
lstm_cells = [DeviceCellWrapper(device, tf.contrib.rnn.BasicRNNCell(num_units=n_neurons))
              for device in devices]
multi_layer_cell = tf.contrib.rnn.MultiRNNCell(lstm_cells)
outputs, states = tf.nn.dynamic_rnn(multi_layer_cell, X, dtype=tf.float32)
init = tf.global_variables_initializer()

with tf.Session() as sess:
    init.run()
    print(sess.run(outputs, feed_dict={X: rnd.rand(2, n_steps, n_inputs)}))


I am working on the file "Recurrent_Neural_Networks_1.1.0.ipynb".
The first task is:
Time series
Now let’s take a look at how to handle time series, such as stock prices, air temperature, brain wave patterns, and so on. In this section we will train an RNN to predict the next value in a generated time series. Each training instance is a randomly selected sequence of 20 consecutive values from the time series, and the target sequence is the same as the input sequence, except it is shifted by one time step into the future.
Can you write Python code for it?

t_min, t_max = 0, 30

n_steps = 20

def time_series(t):
    return t * np.sin(t) / 3 + 2 * np.sin(t*5)

def next_batch(batch_size, n_steps,resolution = 0.1):
    t0 = np.random.rand(batch_size, 1) * (t_max - t_min - n_steps * resolution)
    Ts = t0 + np.arange(0., n_steps + 1) * resolution
    ys = time_series(Ts)
    return ys[:, :-1].reshape(-1, n_steps, 1), ys[:, 1:].reshape(-1, n_steps, 1)

t = np.linspace(t_min, t_max, (t_max - t_min) // resolution)
t_instance = np.linspace(12.2, 12.2 + resolution * (n_steps + 1), n_steps + 1)

plt.figure(figsize=(11,4))
plt.subplot(121)
plt.title("A time series (generated)", fontsize=14)
plt.plot(t, time_series(t), label=r"$t . \sin(t) / 3 + 2 . \sin(5t)$")
plt.plot(t_instance[:-1], time_series(t_instance[:-1]), "b-", linewidth=3, label="A training instance")
plt.legend(loc="lower left", fontsize=14)
plt.axis([0, 30, -17, 13])
plt.xlabel("Time")
plt.ylabel("Value")

plt.subplot(122)
plt.title("A training instance", fontsize=14)
plt.plot(t_instance[:-1], time_series(t_instance[:-1]), "bo", markersize=10, label="instance")
plt.plot(t_instance[1:], time_series(t_instance[1:]), "w*", markersize=10, label="target")
plt.legend(loc="upper left")
plt.xlabel("Time")
plt.show()

X_batch, y_batch = next_batch(1, n_steps)
np.c_[X_batch[0], y_batch[0]]


I am working on the file "Recurrent_Neural_Networks_1.1.0.ipynb".
The first task is:
Training the same sequence classifier with Keras
Can you write Python code for it?

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.layers import SimpleRNN
from keras import initializers
from keras.optimizers import RMSprop

from keras.models import Model
from keras.layers import Input, Dense

batch_size = 150
num_classes = 10
epochs = 100
hidden_units = 150

learning_rate = 0.001

# the data, shuffled and split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()

x_train = x_train.reshape(x_train.shape[0], 28, 28)
x_test = x_test.reshape(x_test.shape[0], 28, 28)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

print('Evaluate IRNN...')
a = Input(shape=x_train.shape[1:])
b = SimpleRNN(hidden_units,
                    kernel_initializer=initializers.RandomNormal(stddev=0.001),
                    recurrent_initializer=initializers.Identity(),
                    activation='relu')(a)
b = Dense(num_classes)(b)
b = Activation('softmax')(b)
optimizer = keras.optimizers.Adamax(lr=learning_rate)
model = Model(inputs=[a], outputs=[b])
model.compile(loss='categorical_crossentropy',
              optimizer=optimizer,
              metrics=['accuracy'])

model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test, y_test))

scores = model.evaluate(x_test, y_test, verbose=0)
print('IRNN test score:', scores[0])
print('IRNN test accuracy:', scores[1])


I am working on the file "Recurrent_Neural_Networks_1.1.0.ipynb".
The first task is:
Using an OuputProjectionWrapper
Can you write Python code for it?

tf.reset_default_graph()

from tensorflow.contrib.layers import fully_connected

n_steps = 20
n_inputs = 1
n_neurons = 100
n_outputs = 1

X = tf.placeholder(tf.float32, [None, n_steps, n_inputs])
y = tf.placeholder(tf.float32, [None, n_steps, n_outputs])

cell = tf.contrib.rnn.OutputProjectionWrapper(
    tf.contrib.rnn.BasicRNNCell(num_units=n_neurons, activation=tf.nn.relu),
    output_size=n_outputs)
outputs, states = tf.nn.dynamic_rnn(cell, X, dtype=tf.float32)

n_outputs = 1
learning_rate = 0.001

loss = tf.reduce_sum(tf.square(outputs - y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
training_op = optimizer.minimize(loss)

init = tf.global_variables_initializer()

n_iterations = 1000
batch_size = 50

with tf.Session() as sess:
    init.run()
    for iteration in range(n_iterations):
        X_batch, y_batch = next_batch(batch_size, n_steps)
        sess.run(training_op, feed_dict={X: X_batch, y: y_batch})
        if iteration % 100 == 0:
            mse = loss.eval(feed_dict={X: X_batch, y: y_batch})
            print(iteration, "\tMSE:", mse)

    X_new = time_series(np.array(t_instance[:-1].reshape(-1, n_steps, n_inputs)))
    y_pred = sess.run(outputs, feed_dict={X: X_new})
    print(y_pred)

plt.title("Testing the model", fontsize=14)
plt.plot(t_instance[:-1], time_series(t_instance[:-1]), "bo", markersize=10, label="instance")
plt.plot(t_instance[1:], time_series(t_instance[1:]), "w*", markersize=10, label="target")
plt.plot(t_instance[1:], y_pred[0,:,0], "r.", markersize=10, label="prediction")
plt.legend(loc="upper left")
plt.xlabel("Time")
plt.show()


I am working on the file "Recurrent_Neural_Networks_1.1.0.ipynb".
The first task is:
Without using an OutputProjectionWrapper
Can you write Python code for it?

tf.reset_default_graph()

from tensorflow.contrib.layers import fully_connected

n_steps = 20
n_inputs = 1
n_neurons = 100

X = tf.placeholder(tf.float32, [None, n_steps, n_inputs])
y = tf.placeholder(tf.float32, [None, n_steps, n_outputs])

basic_cell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons, activation=tf.nn.relu)
rnn_outputs, states = tf.nn.dynamic_rnn(basic_cell, X, dtype=tf.float32)

n_outputs = 1
learning_rate = 0.001

stacked_rnn_outputs = tf.reshape(rnn_outputs, [-1, n_neurons])
stacked_outputs = fully_connected(stacked_rnn_outputs, n_outputs, activation_fn=None)
outputs = tf.reshape(stacked_outputs, [-1, n_steps, n_outputs])

loss = tf.reduce_mean(tf.square(outputs - y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
training_op = optimizer.minimize(loss)

init = tf.global_variables_initializer()

n_iterations = 1000
batch_size = 50

with tf.Session() as sess:
    init.run()
    for iteration in range(n_iterations):
        X_batch, y_batch = next_batch(batch_size, n_steps)
        sess.run(training_op, feed_dict={X: X_batch, y: y_batch})
        if iteration % 100 == 0:
            mse = loss.eval(feed_dict={X: X_batch, y: y_batch})
            print(iteration, "\tMSE:", mse)

    X_new = time_series(np.array(t_instance[:-1].reshape(-1, n_steps, n_inputs)))
    y_pred = sess.run(outputs, feed_dict={X: X_new})
    print(y_pred)

plt.title("Testing the model", fontsize=14)
plt.plot(t_instance[:-1], time_series(t_instance[:-1]), "bo", markersize=10, label="instance")
plt.plot(t_instance[1:], time_series(t_instance[1:]), "w*", markersize=10, label="target")
plt.plot(t_instance[1:], y_pred[0,:,0], "r.", markersize=10, label="prediction")
plt.legend(loc="upper left")
plt.xlabel("Time")
plt.show()


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
As we can see from the above example, the data is not very tidy as it contains line brakes such as '\n' - these actually represent a new rewiew. Using a few tidy rules, I will be able to clean up the raw so that we will have less noise when training our model.
Data cleansing
Before we can build a RNN using TensorFlow, we need to get the data into a proper shape. As I will be using an embedding layer, I need to convert each word into an integer. I will also be doing a bit of data tidying.
Can you write Python code for it?

# Removing punctuations and 'br' line brake tags
reviews_complete_text = ''.join([char for char in reviews if char != '.'])
reviews_complete_text = reviews_complete_text.replace('  br    br  ', '')

# Splitting the reviews sring into a list of reviews
reviews_list = reviews_complete_text.split('\n')


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
Embedding layer
I'll now create an embedding layer. We need this because there are 74,072 words in our vocabulary. It is massively inefficient to one-hot encode our classes.
Can you write Python code for it?

# Number of units in the embedding layer
embed_size = 300

with graph.as_default():
    embedding = tf.Variable(tf.random_uniform((n_words, embed_size), -1, 1))
    embed = tf.nn.embedding_lookup(embedding, inputs_)


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
Encoding words
To implement an embedding layer, we must pass integers to our network. To prepare our reviews for this, we will create a dictionary that map the words in our vocabulary to integers. Using that, we can convert our reviews into integers.
Can you write Python code for it?

from collections import Counter

# Create word counter and sort by the number of occurences of each word in descending order
word_counts = Counter(words_in_reviews)
vocabulary = sorted(word_counts, key=word_counts.get, reverse=True)

# Create a dictionary of words
vocabulary_to_int = {word: i for i, word in enumerate(vocabulary, 1)}

# Create empty reviews list
reviews_int = []
for each in reviews_list:
    reviews_int.append([vocabulary_to_int[word] for word in each.split()])


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
Import data
As for the raw movie reviews and their positive/negative classification, I will be using the 25,000 reviews available on the Udacity Deep Learning GibHub page @ https://github.com/udacity/deep-learning/blob/master/sentiment_network/.
Can you write Python code for it?

with open('data/reviews.txt', 'r') as raw_reviews:
    reviews = raw_reviews.read()
with open('data/labels.txt', 'r') as raw_labels:
    labels = raw_labels.read()


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
LSTM Cell
Using the already defined hyperparameters, I will define my LSTM cells here.
Can you write Python code for it?

with graph.as_default():
    # Your basic LSTM cell
    lstm = tf.contrib.rnn.BasicLSTMCell(lstm_size)

    # Add dropout to the cell
    drop = tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)

    # Stack up multiple LSTM layers, for deep learning
    cell = tf.contrib.rnn.MultiRNNCell([drop] * lstm_layers)

    # Getting an initial state of all zeros
    initial_state = cell.zero_state(batch_size, tf.float32)


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
Our labels are now encoded as 1 and 0. See here:
Can you write Python code for it?


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
Shortering reviews
To help pur RNN learn, we want to ensure that none of the reviews are too long just as we want to ensure that we don't have any reviews with zero length - just in case we are working with bad data, which we often are - that is life of a data scientist.
Can you write Python code for it?

# Review length counter
review_length = Counter([len(review) for review in reviews_int])

# Reviews with 0 length and longest review
review_length[0], max(review_length)


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
TensorFlow RNN graph
Hyperparameters
For use in our graph, we will need the following four hyper parameters:
Can you write Python code for it?

lstm_size = 512       # Number of units in each of our hidden layers in the LSTM cells
lstm_layers = 2       # Number of LSTM layers in the network
batch_size = 500      # Number of reviews, out of the 20,000 in our training set, that we feed into the network in one go
learning_rate = 0.005 # Our learning rate for use in the Adam optimizer


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
The reviews are now seperated from eachother and stored in seperate list objects. Here are the first two reviews:
Can you write Python code for it?


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
Training, validating and testing dataset
My reviews and labels data is now cleanred up as much as required to demonstrate how a RNN works with TensorFlow so its time to split it into a training, validation and testing set. Lets set aside 80% for training and 10% for validation and testing, respectively.
Can you write Python code for it?

# Split faction
training_faction = 0.8

# Index to split the dataset at for training and validation
training_idx = int(len(features) * training_faction)

# Splitting the dataset for train and val
train_x, val_x = features[:training_idx], features[training_idx:]
train_y, val_y = labels[:training_idx], labels[training_idx:]

# Index to split the dataset at for validation and testing
validation_idx = int(len(val_x) * 0.5)

# Splitting the dataset for val and testing
val_x, test_x = val_x[:validation_idx], val_x[validation_idx:]
val_y, test_y = val_y[:validation_idx], val_y[validation_idx:]


I am working on the file "Recurring Neural Network.ipynb".
The first task is:
Validation accuracy
Standard code for calculating accuracy.
Can you write Python code for it?

with graph.as_default():
    correct_pred = tf.equal(tf.cast(tf.round(predictions), tf.int32), labels_)
    accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))


I am working on the file "Recursion.ipynb".
The first task is:
Any question ???

Why so complex ???
Is it really necessary ???
They can be easier and faster !!!
Can you write Python code for it?

def factorial_s(n):
    '''
    n: integer
    returns n! (1*2*3*..*n)
    '''

    ret = 1
    for i in range(1, n+1): # from 1 to n
        ret *= i

    return ret

print('4!=', factorial_s(4))
print('10!=', factorial_s(10))

def fib2(n):
    '''
    n: integer
    return Fibonacchi number of n
    '''

    if n < 2:
        return 1

    n2 = n1 = 1
    for i in range(2, n+1):
        ret = n1 + n2
        n2 = n1
        n1 = ret
    return ret

for i in range(10):
    print('Fibonnachi_simple(', i, ')=' ,fib2(i), sep='')


I am working on the file "Recursion.ipynb".
The first task is:
Java sample code of Merge Sort
```
import java.util.Scanner;
//import java.util.Arrays ;
// import java.InputMismatchException;
// alds_5b, Aizu Online Judge accept only class name: Main
class Main {
public static long merge_count = 0 ;

static void Merge(int [] A, int low, int mid, int high) {
    int l_size = mid-low ;
    int h_size = high-mid ;
    int []L = new int[l_size] ;
    int []H = new int[h_size] ;

    //System.out.printf("Merge: %d %d %d\n", low, mid, high) ;
    for (int i=0; i<l_size; i++) {
        L[i] = A[low+i] ;
    }
    for (int i=0; i<h_size; i++) {
        H[i] = A[mid+i] ;
    }

    int a_pos=low, l_pos=0, h_pos=0 ;

    while (l_size != 0 && h_size != 0) {
        if (L[l_pos] <= H[h_pos]) {
            A[a_pos] = L[l_pos] ;
            l_size -= 1 ;
            l_pos += 1 ;
        }
        else {
            A[a_pos] = H[h_pos] ;
            h_size -= 1 ;
            h_pos += 1 ;
        }
        a_pos += 1 ;
        merge_count++ ;
    }

    while (l_size != 0) {
        A[a_pos] = L[l_pos] ;
        l_size -= 1 ;
        l_pos += 1 ;
        a_pos += 1 ;
        merge_count++ ;
    }

    while (h_size != 0) {
        A[a_pos] = H[h_pos] ;
        h_size -= 1 ;
        h_pos += 1 ;
        a_pos += 1 ;
        merge_count++ ;
    }

}

static void Merge_sort(int [] A, int low, int high) {
    //System.out.printf("Merge_sort: %d %d\n", low, high) ;
    if (low + 1 < high) {   // if 2 or more elements in the List
        int mid = (low + high) / 2 ;
        Merge_sort(A, low, mid) ;
        Merge_sort(A, mid, high) ;
        Merge(A, low, mid, high) ;
    }
}

public static void main(String args[]) {
    Scanner scanner = new Scanner(System.in) ;
    String s = scanner.nextLine() ;

    int elem_count ;
    elem_count = Integer.parseInt(s) ;
    int A[] = new int[elem_count] ;

    for (int i=0; i<elem_count; i++) {
        A[i] = scanner.nextInt() ;
    }

    scanner.close() ;
    Merge_sort(A, 0, elem_count) ;

    for (int i=0; i<elem_count; i++)...
Can you write Python code for it?

def f(n):
    '''
    n: integer
    return sum of each digit
    '''

    digits = list(str(n))
    return (sum(map(int, digits)))

import sys
def g(n):
    print('g()', n, file=sys.stderr, end = ': ')
    while n>10:
        n = f(n)
        print(n, file=sys.stderr, end = ' ')
    print(file=sys.stderr)
    return n

import random

for i in range(5):
    n = random.randrange(1000, 2000000000)
    print(n, g(n))

print(1234567892, g(1234567892))


I am working on the file "Recursion.ipynb".
The first task is:
So why recursion is important ?

Sometime it is difficult to calculate from start point
Recursion sometimes shows drastic effect

Example: Merge Sort  https://www.geeksforgeeks.org/merge-sort/
Can you write Python code for it?

from IPython.display import Image, display_png

display_png(Image("Merge-Sort-Tutorial.png"))

import sys

def merge(list1, list2):
    print('Merge:', list1, list2, end= '->', file=sys.stderr)
    merged_list = list()  # Create empty list

    while (list1 and list2):  # While there are elements in both list
        if list2[0] < list1[0]:
            merged_list.append(list2.pop(0))
        else:
            merged_list.append(list1.pop(0))

    if list1: merged_list.extend(list1)  # append if list1 is not empty
    if list2: merged_list.extend(list2)

    print(merged_list, file=sys.stderr)
    return merged_list

def mergeSort(list_x):
    print('Merge Sort:', list_x, file=sys.stderr)
    if len(list_x) > 1:
        mid_index = len(list_x) // 2
        list1 = mergeSort(list_x[:mid_index])  # not include mid_index
        list2 = mergeSort(list_x[mid_index:])  # include mid_index
        list_x = merge(list1, list2)
    return list_x

import random

num_elem = random.randrange(10) + 10

r_list = [random.randrange(100) for i in range(num_elem)]
#r_list = list()
#for i in range(num_elem):
#    r_list.append(random.randrange(100))

print('Initial List:', r_list)
print('Sorted List:', mergeSort(r_list))


I am working on the file "Red Light Camera Locations.ipynb".
The first task is:
How does this list compare to the one currently published on the Chicago Data Portal?
Can you write Python code for it?

from csv import DictReader
from StringIO import StringIO

data_portal_url = 'https://data.cityofchicago.org/api/views/thvf-6diy/rows.csv?accessType=DOWNLOAD'
r = requests.get(data_portal_url)
fh = StringIO(r.text)
reader = DictReader(fh)

def cleaner(str):
    filters = [
        ('Stony?Island', 'Stony Island'),
        ('Van?Buren', 'Van Buren'),
        (' (SOUTH INTERSECTION)', '')
    ]
    for filter in filters:
        str = str.replace(filter[0], filter[1])
    return str

for line in reader:
    line['INTERSECTION'] = cleaner(line['INTERSECTION'])
    cameras[line['INTERSECTION']]['attributes']['current'] = line

counter = {
    'not current': [],
    'current': [],
    'not current and slated for removal': [],
    'not current and not slated for removal': [],
    'current and slated for removal': []
}
for camera in cameras:
    if 'current' not in cameras[camera]['attributes']:
        counter['not current'].append(camera)
        if cameras[camera]['attributes']['To be removed']:
            counter['not current and slated for removal'].append(camera)
        else:
            counter['not current and not slated for removal'].append(camera)
    else:
        counter['current'].append(camera)
        if cameras[camera]['attributes']['To be removed']:
            counter['current and slated for removal'].append(camera)

for key in counter:
    print key, len(counter[key])
    print '; '.join(counter[key]), '\n'


I am working on the file "Red Light Camera Locations.ipynb".
The first task is:
Now it's time to ask some specific questions. First: how much money has the program raised overall? (Note that this data only goes back to 7/1/2014, several years after the program began.)
Can you write Python code for it?

import locale
locale.setlocale( locale.LC_ALL, '' )

total = 0
missing_tickets = []
for camera in cameras:
    try:
        total += cameras[camera]['attributes']['total tickets']
    except KeyError:
        missing_tickets.append(camera)

print '%d tickets have been issued since 7/1/2014, raising %s' % (total, locale.currency(total * 100, grouping=True))
print 'The following %d intersections appear to never have issued a ticket in that time: %s' % (
    len(missing_tickets), '; '.join(missing_tickets))


I am working on the file "Red Light Camera Locations.ipynb".
The first task is:
Now let's generate a CSV of the cameras data for export.
Can you write Python code for it?

from csv import DictWriter
output = []

for camera in cameras:
    data = {
        'intersection': camera,
        'last ticket date': cameras[camera]['attributes'].get('last ticket date', ''),
        'tickets since 7/1/2014': cameras[camera]['attributes'].get('total tickets', 0),
        'revenue since 7/1/2014': cameras[camera]['attributes'].get('total tickets', 0) * 100,
        'tickets since 12/22/2014': cameras[camera]['attributes'].get('tickets since 12/22/2014', 0),
        'revenue since 12/22/2014': cameras[camera]['attributes'].get('tickets since 12/22/2014', 0) * 100,
        'was slated for removal': cameras[camera]['attributes'].get('To be removed', False),
        'had few crashes': cameras[camera]['attributes'].get('Few crashes', False),
        'is currently active': True if 'current' in cameras[camera]['attributes'] else False,
        'latitude': cameras[camera]['attributes'].get('LAT', 0),
        'longitude': cameras[camera]['attributes'].get('LNG', 0)
    }
    output.append(data)

with open('/tmp/red_light_intersections.csv', 'w+') as fh:
    writer = DictWriter(fh, sorted(output[0].keys()))
    writer.writeheader()
    writer.writerows(output)


I am working on the file "Red Light Camera Locations.ipynb".
The first task is:
Now we need to compute how much money has been generated at each intersection - assuming $100 fine for each violation. In order to do that, we need to make the violation data line up with the camera location data.
Then, we'll add 3 fields: number of violations overall; number on/after 12/22/2014; number on/after 3/6/2015.
Can you write Python code for it?

import requests
from csv import DictReader
from datetime import datetime
from StringIO import StringIO

data_portal_url = 'https://data.cityofchicago.org/api/views/spqx-js37/rows.csv?accessType=DOWNLOAD'
r = requests.get(data_portal_url)
fh = StringIO(r.text)
reader = DictReader(fh)

def violation_cleaner(str):
    filters = [
        (' AND ', '-'),
        (' and ', '-'),
        ('/', '-'),
        # These are streets spelled one way in ticket data, another way in location data
        ('STONEY ISLAND', 'STONY ISLAND'),
        ('CORNELL DRIVE', 'CORNELL'),
        ('NORTHWEST HWY', 'NORTHWEST HIGHWAY'),
        ('CICERO-I55', 'CICERO-STEVENSON NB'),
        ('31ST ST-MARTIN LUTHER KING DRIVE', 'DR MARTIN LUTHER KING-31ST'),
        ('4700 WESTERN', 'WESTERN-47TH'),
        ('LAKE SHORE DR-BELMONT', 'LAKE SHORE-BELMONT'),
        # These are 3-street intersections where the ticket data has 2 streets, location data has 2 other streets
        ('KIMBALL-DIVERSEY', 'MILWAUKEE-DIVERSEY'),
        ('PULASKI-ARCHER', 'PULASKI-ARCHER-50TH'),
        ('KOSTNER-NORTH', 'KOSTNER-GRAND-NORTH'),
        ('79TH-KEDZIE', 'KEDZIE-79TH-COLUMBUS'),
        ('LINCOLN-MCCORMICK', 'KIMBALL-LINCOLN-MCCORMICK'),
        ('KIMBALL-LINCOLN', 'KIMBALL-LINCOLN-MCCORMICK'),
        ('DIVERSEY-WESTERN', 'WESTERN-DIVERSEY-ELSTON'),
        ('HALSTED-FULLERTON', 'HALSTED-FULLERTON-LINCOLN'),
        ('COTTAGE GROVE-71ST', 'COTTAGE GROVE-71ST-SOUTH CHICAGO'),
        ('DAMEN-FULLERTON', 'DAMEN-FULLERTON-ELSTON'),
        ('DAMEN-DIVERSEY', 'DAMEN-DIVERSEY-CLYBOURN'),
        ('ELSTON-FOSTER', 'ELSTON-LAPORTE-FOSTER'),
        ('STONY ISLAND-79TH', 'STONY ISLAND-79TH-SOUTH CHICAGO'),
        # This last one is an artifact of the filter application process
        ('KIMBALL-LINCOLN-MCCORMICK-MCCORMICK', 'KIMBALL-LINCOLN-MCCORMICK')
    ]
    for filter in filters:
        str = str.replace(filter[0], filter[1])
    return str

def intersection_is_reversed(key, intersection):
    split_key = key.upper().split('-')
    split_intersection =...


I am working on the file "Red Light Camera Locations.ipynb".
The first task is:
Set the 'Few crashes' flag to True for those intersections that show up in filtered_few_crashes.
Can you write Python code for it?

for point in filtered_few_crashes:
    label = point['attributes']['LABEL']
    if label not in cameras:
        print 'Missing label %s' % label
    else:
        cameras[label]['attributes']['Few crashes'] = True


I am working on the file "Red Light Camera Locations.ipynb".
The first task is:
Since 12/22/2014, how much money has been generated by low-crash intersections?
Can you write Python code for it?

total = 0
low_crash_total = 0
for camera in cameras:
    try:
        total += cameras[camera]['attributes']['tickets since 12/22/2014']
        if cameras[camera]['attributes']['Few crashes']:
            low_crash_total += cameras[camera]['attributes']['tickets since 12/22/2014']
    except KeyError:
        continue

print '%d tickets have been issued at low-crash intersections since 12/22/2014, raising %s' % (
    low_crash_total, locale.currency(low_crash_total * 100, grouping=True))
print '%d tickets have been issued overall since 12/22/2014, raising %s' % (
    total, locale.currency(total * 100, grouping=True))


I am working on the file "Rede Neural.ipynb".
The first task is:
4 Clusters
Can you write Python code for it?

x, y = make_blobs(n_samples=500, n_features=2, cluster_std=0.9, centers=[(-3, -3), (3, 3), (-3, 3), (3, -3)], random_state=1234)
y = y.reshape(-1, 1)
y = np.where(y >= 2, 1, 0)

print(x.shape, y.shape)
plt.scatter(x[:,0], x[:,1], c=list(np.array(y).ravel()), s=15, cmap='bwr')

input_dim, output_dim = x.shape[1], y.shape[1]

# insira sua rede aqui!

y_pred = nn.predict(x)
print('Acurácia: {:.2f}%'.format(100*accuracy_score(y, y_pred > 0.5)))

plot.classification_predictions(x, y, is_binary=True, nn=nn, threshold=0.5, cmap='bwr')


I am working on the file "Rede Neural.ipynb".
The first task is:
4 Clusters Multiclasse
Can you write Python code for it?

x, y = make_blobs(n_samples=400, n_features=2, centers=[(-3, 0), (3, 0), (0, 3), (0, -3)], random_state=1234)
y = y.reshape(-1, 1)

print(x.shape, y.shape)
plt.scatter(x[:,0], x[:,1], c=list(np.array(y).ravel()), s=15, cmap=plt.cm.viridis)

onehot = OneHotEncoder(sparse=False)
y_onehot = onehot.fit_transform(y)
print(y_onehot[::70])

input_dim, output_dim = x.shape[1], y.shape[1]

# insira sua rede aqui!

y_pred = np.argmax(nn.predict(x), axis=1)
print('Acurácia: {:.2f}%'.format(100*accuracy_score(y, y_pred)))

plot.classification_predictions(x, y, is_binary=False, nn=nn)


I am working on the file "Rede Neural.ipynb".
The first task is:
Classificação Multiclasse
3 Clusters Multiclasse
Can you write Python code for it?

x, y = make_blobs(n_samples=300, n_features=2, centers=[(0, -3), (-3, 3), (3, 3)], random_state=1234)
y = y.reshape(-1, 1)

print(x.shape, y.shape)
plt.scatter(x[:,0], x[:,1], c=list(np.array(y).ravel()), s=15, cmap=plt.cm.viridis)

onehot = OneHotEncoder(sparse=False)
y_onehot = onehot.fit_transform(y)
print(y_onehot[::60])

input_dim, output_dim = x.shape[1], y.shape[1]

# insira sua rede aqui!

y_pred = np.argmax(nn.predict(x), axis=1)
print('Acurácia: {:.2f}%'.format(100*accuracy_score(y, y_pred)))

plot.classification_predictions(x, y, is_binary=False, nn=nn)


I am working on the file "Rede Neural.ipynb".
The first task is:
Círculos
Can you write Python code for it?

x, y = make_circles(n_samples=500, noise=0.1, factor=0.4, random_state=1234)
y = y.reshape(-1, 1)

print(x.shape, y.shape)
plt.scatter(x[:,0], x[:,1], c=list(np.array(y).ravel()), s=15, cmap='bwr')

input_dim, output_dim = x.shape[1], y.shape[1]

# insira sua rede aqui!

y_pred = nn.predict(x)
print('Acurácia: {:.2f}%'.format(100*accuracy_score(y, y_pred > 0.5)))

plot.classification_predictions(x, y, is_binary=True, nn=nn, threshold=0.5, cmap='bwr')


I am working on the file "Rede Neural.ipynb".
The first task is:
Espiral - 5 Classes
Can you write Python code for it?

x, y = make_spiral(n_samples=100, n_class=5, radius=1, laps=0.5)
y = y.reshape(-1, 1)

print(x.shape, y.shape)
plt.scatter(x[:,0], x[:,1], c=list(np.array(y).ravel()), s=15, cmap=plt.cm.viridis)

onehot = OneHotEncoder(sparse=False)
y_onehot = onehot.fit_transform(y)
print(y_onehot[::100])

input_dim, output_dim = x.shape[1], y.shape[1]

# insira sua rede aqui!

y_pred = np.argmax(nn.predict(x), axis=1)
print('Acurácia: {:.2f}%'.format(100*accuracy_score(y, y_pred)))

plot.classification_predictions(x, y, is_binary=False, nn=nn)


I am working on the file "Rede Neural.ipynb".
The first task is:
Exemplo 2
Can you write Python code for it?


I am working on the file "Rede Neural.ipynb".
The first task is:
Funções Auxiliares
Can you write Python code for it?

def softmax(x, y_oh=None, derivative=False):
    if derivative:
        y_pred = softmax(x)
        k = np.nonzero(y_pred * y_oh)
        pk = y_pred[k]
        y_pred[k] = pk * (1.0 - pk)
        return y_pred
    exp = np.exp(x)
    return exp / np.sum(exp, axis=1, keepdims=True)


I am working on the file "Rede Neural.ipynb".
The first task is:
Moons
Can you write Python code for it?

x, y = make_moons(200, noise=0.20)
y = y.reshape(-1, 1)

print(x.shape, y.shape)
plt.scatter(x[:,0], x[:,1], c=list(np.array(y).ravel()), s=15, cmap='bwr')

input_dim, output_dim = x.shape[1], y.shape[1]

# insira sua rede aqui!

y_pred = nn.predict(x)
print('Acurácia: {:.2f}%'.format(100*accuracy_score(y, y_pred > 0.5)))

plot.classification_predictions(x, y, is_binary=True, nn=nn, threshold=0.5, cmap='bwr')


I am working on the file "Rede Neural.ipynb".
The first task is:
Porta XOR
Can you write Python code for it?

x = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([0, 1, 1, 0]).reshape(-1, 1)

print(x.shape, y.shape)
plt.scatter(x[:,0], x[:,1], c=list(np.array(y).ravel()), s=15, cmap='bwr')

input_dim, output_dim = x.shape[1], y.shape[1]

# insira sua rede aqui!

y_pred = nn.predict(x)
print('Predições:', y_pred, sep='\n')
print('Acurácia: {:.2f}%'.format(100*accuracy_score(y, y_pred > 0.5)))

plot.classification_predictions(x, y, is_binary=True, nn=nn, cmap='bwr')


I am working on the file "Rede Neural.ipynb".
The first task is:
Regressão Cúbica
Can you write Python code for it?

x, y = make_cubic(n_samples=100, x_min=-4, x_max=4, a=1, b=0, c=-10, d=0, noise=3)

print(x.shape, y.shape)
plt.scatter(x, y)

minmax = MinMaxScaler(feature_range=(-1, 1))
x = minmax.fit_transform(x.astype(np.float64))

print(x.min(axis=0), x.max(axis=0))

input_dim, output_dim = x.shape[1], y.shape[1]

# insira sua rede aqui!

plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plt.scatter(x, y)
plt.plot(x, nn.predict(x), c='green')


I am working on the file "Rede Neural.ipynb".
The first task is:
Regressão Exponencial
Can you write Python code for it?

x, y = make_exp(n_samples=100, x_min=0, x_max=5, noise=10)

print(x.shape, y.shape)
plt.scatter(x, y)

minmax = MinMaxScaler(feature_range=(-1, 1))
x = minmax.fit_transform(x.astype(np.float64))

print(x.min(axis=0), x.max(axis=0))

input_dim, output_dim = x.shape[1], y.shape[1]

# insira sua rede aqui!

plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plt.scatter(x, y)
plt.plot(x, nn.predict(x), c='green')


I am working on the file "Rede Neural.ipynb".
The first task is:
Regressão Logarítimica
Can you write Python code for it?

x, y = make_log10(n_samples=100, x_min=1, x_max=100, noise=0.3)

print(x.shape, y.shape)
plt.scatter(x, y)

minmax = MinMaxScaler(feature_range=(-1, 1))
x = minmax.fit_transform(x.astype(np.float64))

print(x.min(axis=0), x.max(axis=0))

input_dim, output_dim = x.shape[1], y.shape[1]

# insira sua rede aqui!

plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plt.scatter(x, y)
plt.plot(x, nn.predict(x), c='green')


I am working on the file "Redshift_Efficiency_Study/BGS_Redshift_Efficiency_v3.ipynb".
The first task is:
All or none of the output files can be overwritten using these keywords.
Can you write Python code for it?

overwrite_spectra = False
overwrite_redshifts = False
overwrite_results = False


I am working on the file "Redshift_Efficiency_Study/BGS_Redshift_Efficiency_v3.ipynb".
The first task is:
Analyze the outputs.
We only have one simulation so the code below is hard-coded to sim1, although it would be trivial to generalize.
Can you write Python code for it?

sim = sim1
resultfile = os.path.join(simdir, sim['suffix'], 'bgs-{}-results.fits'.format(sim['suffix']))
log.info('Reading {}'.format(resultfile))
result = Table.read(resultfile)
result

from desistudy import qa_zmag, qa_efficiency, qa_zwarn4, qa_radec

qa_zmag(result['ZTRUE'], result['RMAG'], maglabel=r'$r_{\rm DECaLS}$ (AB mag)', faintmag=20.0)

qa_efficiency(result)

qa_zwarn4(result)


I am working on the file "Redshift_Efficiency_Study/BGS_Redshift_Efficiency_v3.ipynb".
The first task is:
BGS
 NOTE  must be in branch 'fix-densities' of desitarget for the mocktarget things
data is a dictionary of lists:
    data = {'TARGET_NAME': target_name_list, 'MOCKFORMAT': 'gaussianfield',
               'OBJID': objid_list, 'MOCKID': mockid_list, 'BRICKNAME': brickname_list,
               'RA': ra_list, 'DEC': dec_list, 'Z': zz_list,'FILES': files_list, 'N_PER_FILE': n_per_file}
but data has to be run through:
_prepare_spectra(data, nside_chunk=128):
which adds the following to data with appropriate defaults
    data.update({
        'TRUESPECTYPE': 'GALAXY', 'TEMPLATETYPE': 'BGS', 'TEMPLATESUBTYPE': '',
        'SEED': seed, 'VDISP': vdisp,
        'SHAPEEXP_R': gmm['exp_r'], 'SHAPEEXP_E1': gmm['exp_e1'], 'SHAPEEXP_E2': gmm['exp_e2'],
        'SHAPEDEV_R': gmm['dev_r'], 'SHAPEDEV_E1': gmm['dev_e1'], 'SHAPEDEV_E2': gmm['dev_e2'],
        })

indx is an optional list of inde=indices to use corresponding to the indices of the above list
Once data (and optionally indx) are defined we're ready to make the spectra;
bgsmaker.make_spectra(self, data=data, indx=None)

make_spectra returns:
    Returns
    -------
    flux : :class:`numpy.ndarray`
        Target spectra.
    wave : :class:`numpy.ndarray`
        Corresponding wavelength array.
    meta : :class:`astropy.table.Table`
        Spectral metadata table.
    targets : :class:`astropy.table.Table`
        Target catalog.
    truth : :class:`astropy.table.Table`
        Corresponding truth table.
Can you write Python code for it?

#bgsmaker = BGSMaker(seed=seed)

#log.info('Reading the mock catalog for {}s'.format(bgsmaker.objtype))

#data = {'TARGET_NAME': target_name_list, 'MOCKFORMAT': 'gaussianfield',
#               'OBJID': objid_list, 'MOCKID': mockid_list, 'BRICKNAME': brickname_list,
#               'RA': ra_list, 'DEC': dec_list, 'Z': zz_list,'FILES': files_list, 'N_PER_FILE': n_per_file}

## If you want to start from an existing default dictionary, you could use:
##    tdata = bgsmaker.read(healpixels=healpixel, nside=nside, dust_dir=dust_dir)
## then either overwrite the relevant details (e.g. z, mag, ra, dec, ids)
## or actually use the defaults for your analysis

## but first we have to prepare the spectra. this internal function adds required fields
## to the data dictionary that we don't want to deal with ourselves
#data = bgsmaker._prepare_spectra(data, nside_chunk=nside_chunk)

## Finally we can make the spectra
#flux, wave, meta, targets, truth = bgsmaker.make_spectra(data)

from desistudy import write_templates
write_templates(outfile, flux, wave, meta)

from desisim.scripts.quickspectra import sim_spectra

simdata = bgs_write_simdata(sim,rand,overwrite=overwrite_spectra)

for ii, simdata1 in enumerate(simdata):

    # Generate the observing conditions dictionary.
    obs = simdata2obsconditions(simdata1)

    # Generate the rest-frame templates.  Currently not writing out the rest-frame
    # templates but we could.
    flux, wave, meta = bgs_make_templates(sim, rand, BGSmaker)

    truefile = os.path.join(simdir, sim['suffix'], 'bgs-{}-{:03}-true.fits'.format(sim['suffix'], ii))
    if overwrite or not os.path.isfile(truefile):
        write_templates(truefile, flux, wave, meta)

    spectrafile = os.path.join(simdir, sim['suffix'], 'bgs-{}-{:03}.fits'.format(sim['suffix'], ii))
    if overwrite or not os.path.isfile(spectrafile):
        sim_spectra(wave, flux, 'bgs', spectrafile, obsconditions=obs,
                    sourcetype='bgs', seed=sim['seed'], expid=ii)
    else:
        print('File {}...


I am working on the file "Redshift_Efficiency_Study/BGS_Redshift_Efficiency_v3.ipynb".
The first task is:
Establish the I/O path, random seed, and path to the dust maps and desired healpixel.
Can you write Python code for it?

simdir = os.path.join(os.getenv('DESI_ROOT'), 'spectro', 'sim', 'bgs')
dust_dir = '/Users/ioannis/research/data/sfd_dustmaps/maps'

healpixel = 26030
nside = 64

seed = 555
rand = np.random.RandomState(seed)


I am working on the file "Redshift_Efficiency_Study/BGS_Redshift_Efficiency_v3.ipynb".
The first task is:
Finally demonstrate how to generate (empyt) SKY spectra.
Can you write Python code for it?

#SKY = SKYMaker(seed=seed)
#skydata = SKY.read(healpixels=healpixel, nside=nside, dust_dir=dust_dir)
#skyflux, skywave, skymeta, skytargets, skytruth = SKY.make_spectra(skydata)
#SKY.select_targets(skytargets, skytruth)


I am working on the file "Redshift_Efficiency_Study/BGS_Redshift_Efficiency_v3.ipynb".
The first task is:
Gather the results.
Can you write Python code for it?

from desistudy import bgs_gather_results

for sim in np.atleast_1d(sim1):
    bgs_gather_results(sim, overwrite=overwrite_results)


I am working on the file "Redshift_Efficiency_Study/BGS_Redshift_Efficiency_v3.ipynb".
The first task is:
Generate Spectra
Can you write Python code for it?

from desistudy import bgs_sim_spectra

for sim in np.atleast_1d(sim1):
    bgs_sim_spectra(sim, verbose=False, overwrite=overwrite_spectra)


I am working on the file "Redshift_Efficiency_Study/BGS_moonsep_sanitycheck.ipynb".
The first task is:
Analyze the outputs.
Sims [0-4]?6 have magnitude range 19.5-20 and z-range 0.1-0.8
Below the
Sim 06 has moon sep of 120 degrees. Moon at 30 degrees above the horizon. Moon Frac 0.8. Exptime 480.
Sim 16 has moon sep of 60 degrees. Moon at 30 degrees above the horizon. Moon Frac 0.8. Exptime 480.
Sim 26 has moon sep of 20 degrees. Moon at 30 degrees above the horizon. Moon Frac 0.8. Exptime 480.
Sim 36 has moon sep of 20 degrees. Moon at 90 degrees above the horizon. Moon Frac 0.99. Exptime 480.
Sim 46 has moon sep of 20 degrees. Moon at 90 degrees above the horizon. Moon Frac 0.99. Exptime 300.
Can you write Python code for it?

from desistudy_qa import print_file_locations
print_file_locations(simdir,sims)


I am working on the file "Redshift_Efficiency_Study/BGS_moonsep_sanitycheck.ipynb".
The first task is:
Sim 06
Can you write Python code for it?

current_sim_name = 'sim06'

from desistudy_qa import print_sim_params
print_sim_params(current_sim_name, sim_names, sims, conditions)

# sim 06
if current_sim_name in sim_names:
    ind = np.where(sim_names==current_sim_name)[0][0]
    sim,obs = sims[ind],conditions[ind]

    resultfile = os.path.join(simdir, sim['suffix'], 'bgs-{}-results.fits'.format(sim['suffix']))
    if os.path.exists(resultfile):
        print('Sim: {} with nspec: {} and nsim: {}'.format(sim['suffix'],sim['nspec'],sim['nsim']))

        if 'EXPTIME' in obs.keys(): exptime = "{}".format(obs['EXPTIME'])
        else:                       exptime = "{}-{}".format(obs['exptimemin'],obs['exptimemax'])

        if 'MOONFRAC' in obs.keys(): moonfrac = "{}".format(obs['MOONFRAC'])
        else:                        moonfrac = "{}-{}".format(obs['moonfracmin'],obs['moonfracmax'])

        print('ObsConds ExpTime: {}, MoonFrac: {}, MoonAlt: {}, MoonSep: {}'.format(exptime,moonfrac,obs['MOONALT'],obs['MOONSEP']))
        result = Table.read(resultfile)
        #result.pprint()
        qa_zmag(result['ZTRUE'], result['RMAG'], maglabel=r'$r_{\rm DECaLS}$ (AB mag)', faintmag=20.)
        qa_efficiency(result)
        qa_zwarn4(result)


I am working on the file "Redshift_Efficiency_Study/BGS_moonsep_sanitycheck.ipynb".
The first task is:
Sim 26
Can you write Python code for it?

current_sim_name = 'sim26'

print_sim_params(current_sim_name, sim_names, sims, conditions)

# sim 26
if current_sim_name in sim_names:
    ind = np.where(sim_names==current_sim_name)[0][0]
    sim,obs = sims[ind],conditions[ind]

    resultfile = os.path.join(simdir, sim['suffix'], 'bgs-{}-results.fits'.format(sim['suffix']))
    if os.path.exists(resultfile):
        print('Sim: {} with nspec: {} and nsim: {}'.format(sim['suffix'],sim['nspec'],sim['nsim']))

        if 'EXPTIME' in obs.keys(): exptime = "{}".format(obs['EXPTIME'])
        else:                       exptime = "{}-{}".format(obs['exptimemin'],obs['exptimemax'])

        if 'MOONFRAC' in obs.keys(): moonfrac = "{}".format(obs['MOONFRAC'])
        else:                        moonfrac = "{}-{}".format(obs['moonfracmin'],obs['moonfracmax'])

        print('ObsConds ExpTime: {}, MoonFrac: {}, MoonAlt: {}, MoonSep: {}'.format(exptime,moonfrac,obs['MOONALT'],obs['MOONSEP']))
        result = Table.read(resultfile)
        #result.pprint()
        qa_zmag(result['ZTRUE'], result['RMAG'], maglabel=r'$r_{\rm DECaLS}$ (AB mag)', faintmag=20.)
        qa_efficiency(result)
        qa_zwarn4(result)


I am working on the file "Redshift_Efficiency_Study/BGS_moonsep_sanitycheck.ipynb".
The first task is:
Sim 46
Can you write Python code for it?

current_sim_name = 'sim46'

print_sim_params(current_sim_name, sim_names, sims, conditions)

# sim 46
if current_sim_name in sim_names:
    ind = np.where(sim_names==current_sim_name)[0][0]
    sim,obs = sims[ind],conditions[ind]

    resultfile = os.path.join(simdir, sim['suffix'], 'bgs-{}-results.fits'.format(sim['suffix']))
    if os.path.exists(resultfile):
        print('Sim: {} with nspec: {} and nsim: {}'.format(sim['suffix'],sim['nspec'],sim['nsim']))

        if 'EXPTIME' in obs.keys(): exptime = "{}".format(obs['EXPTIME'])
        else:                       exptime = "{}-{}".format(obs['exptimemin'],obs['exptimemax'])

        if 'MOONFRAC' in obs.keys(): moonfrac = "{}".format(obs['MOONFRAC'])
        else:                        moonfrac = "{}-{}".format(obs['moonfracmin'],obs['moonfracmax'])

        print('ObsConds ExpTime: {}, MoonFrac: {}, MoonAlt: {}, MoonSep: {}'.format(exptime,moonfrac,obs['MOONALT'],obs['MOONSEP']))
        result = Table.read(resultfile)
        #result.pprint()
        qa_zmag(result['ZTRUE'], result['RMAG'], maglabel=r'$r_{\rm DECaLS}$ (AB mag)', faintmag=20.)
        qa_efficiency(result)
        qa_zwarn4(result)


I am working on the file "Redshift_Efficiency_Study/BGS_z-efficiency_uniform-sampling.ipynb".
The first task is:
All or none of the output files can be overwritten using these keywords.
Can you write Python code for it?

overwrite_spectra = True
#overwrite_templates = overwrite_spectra
overwrite_redshifts = True
overwrite_results = True


I am working on the file "Redshift_Efficiency_Study/BGS_z-efficiency_uniform-sampling.ipynb".
The first task is:
Do everything in one cell
Can you write Python code for it?

# from desistudy import bgs_sim_spectra
# from desistudy import bgs_redshifts
# from desistudy import bgs_gather_results

# for sim,cond in zip(sims,conditions):
#     log.info("\n\n\n\nNow performing sim {}".format(sim['suffix']))
#     if overwrite_spectra:
#         bgs_sim_spectra(sim, cond, simdir, verbose=False, overwrite=overwrite_spectra)
#         log.info("Finished simulating templates")
#     if overwrite_redshifts:
#         bgs_redshifts(sim, simdir=simdir, overwrite=overwrite_redshifts)
#         log.info("Finished redshift fitting")
#     if overwrite_results:
#         bgs_gather_results(sim, simdir=simdir, overwrite=overwrite_results)
#         log.info("Finished gathering results")


I am working on the file "Redshift_Efficiency_Study/BGS_z-efficiency_uniform-sampling.ipynb".
The first task is:
Establish the I/O path, random seed, and path to the dust maps and desired healpixel.
Can you write Python code for it?

simdir = os.path.join(os.getenv('DESI_ROOT'), 'spectro', 'sim', 'bgs', 'kremin', 'flat_priors')

if not os.path.exists(simdir):
    os.makedirs(simdir)

seed = 626


I am working on the file "Redshift_Efficiency_Study/BGS_z-efficiency_uniform-sampling.ipynb".
The first task is:
Fit the redshifts.
This step took ~1.8 seconds per spectrum, ~3 minutes per 100 spectra, or ~30 minutes for all 1000 spectra with my 4-core laptop.
Can you write Python code for it?

from desistudy import bgs_redshifts

if overwrite_redshifts:
    for sim in sims:
        log.info("\n\n\n\nNow performing sim {}".format(sim['suffix']))
        bgs_redshifts(sim, simdir=simdir, overwrite=overwrite_redshifts)
        log.info("\n\n\n\n\nFinished redshift fitting\n\n\n")


I am working on the file "Redshift_Efficiency_Study/BGS_z-efficiency_uniform-sampling.ipynb".
The first task is:
Set up the simulation parameters.
Here we use the mock to capture the correct distribution of apparent magnitudes, galaxy properties, and redshifts.
Note that if use_mock=False then rmagmin, rmagmax, zmin, and zmax are required. For example, here's another possible simulation of 1000 spectra in which the magnitude (r=19.5) and redshift (z=0.2) are held fixed while moonfrac and moonsep are varied (as well as intrinsic galaxy properties):
python
sim2 = dict(suffix='sim02',
            use_mock=False,
            nsim=10,
            nspec=100,
            seed=22,
            zmin=0.2, zmax=0.2,
            rmagmin=19.5, rmagmax=19.5,
            moonfracmin=0.0, moonfracmax=1.0,
            moonsepmin=0.0, moonsepmax=120.0,
           )
Can you write Python code for it?

from desistudy import get_predefined_sim_dict, get_predefined_obs_dict

all_sims = []
all_obsconds = []

for simname in simnames:
    all_sims.append(get_predefined_sim_dict(simname))
    all_obsconds.append(get_predefined_obs_dict(simname))

print(all_obsconds)

sims = np.atleast_1d(all_sims)
conditions = np.atleast_1d(all_obsconds)


I am working on the file "Reduce Hospital Readmissions Using EDA/sliderule_dsi_inferential_statistics_exercise_3.ipynb".
The first task is:
A. Do you agree with the above analysis and recommendations? Why or why not?
The analysis seems to base its conclusion on one scatterplot. I cannot agree with the above analysis and recommendations because there is not enough evidence to support it. Further investigation and statistical analysis should be completed to determine whether there is a significant correlation between hospital capacity (number of discharges) and readmission rates. It may be possible that the correlation may have come from chance.
B. Provide support for your arguments and your own recommendations with a statistically sound analysis:
1) Setup an appropriate hypothesis test.
$H_0$: There is no statistically significant correlation between number of discharges and readmission rates.
$H_A$: There is a statistically significant negative correlation between number of discharges and readmission rates.
2) Compute and report the observed significance value (or p-value).
Can you write Python code for it?

number_of_discharges = hospital_dropna_df['Number of Discharges']
excess_readmission_ratio = hospital_dropna_df['Excess Readmission Ratio']

pearson_r = np.corrcoef(number_of_discharges, excess_readmission_ratio)[0, 1]

print('The Pearson correlation of the sample is', pearson_r)

permutation_replicates = np.empty(100000)

for i in range(len(permutation_replicates)):
    number_of_discharges_perm = np.random.permutation(number_of_discharges)
    permutation_replicates[i] = np.corrcoef(number_of_discharges_perm, excess_readmission_ratio)[0, 1]

p = np.sum(permutation_replicates <= pearson_r) / len(permutation_replicates)
print('p =', p)


I am working on the file "Reduce Hospital Readmissions Using EDA/sliderule_dsi_inferential_statistics_exercise_3.ipynb".
The first task is:
The p value was calculated to be extremely small above. This means our null hypothesis ($H_0$) should be rejected and the alternate hypothesis is more likely. There is a statistically significant negtive correlation between number of discharges and readmission rates. However the correlation is small as shown by the pearson correlation above of -0.097.
3) Report statistical significance for  α  = .01.
Since the p value was calculated to be extremely small, we can still conclude it to be statistically significant with an alpha of 0.01.
4) Discuss statistical significance and practical significance. Do they differ here? How does this change your recommendation to the client?
Statistical significance refers to how unlikely the Pearson correlation observed in the samples have occurred due to sampling errors. We can use it in hypothesis testing. Statisical significance tells us whether the value we were looking at, (in this case - pearson correlation) occured due to chance. Because the p value we calculated was so small, we can conclude that the correlation was statistically significant and that there likely is a negative correlation between number of discharges and readmission rates.
Practical significance looks at whether the Pearson correlation is large enough to be a value that shows a significant relationship. Practical significance looks at the value itself. In this case, the Pearson correlation was very small (-0.097). There probably is a negative correlation. However it is so small and close to zero that it is not very significant to the relationship between number of discharges and readmission rates.
Overall, size or number of discharges is not a good predictor for readmission rates. The recommendations above should not be followed. Instead further analysis should be performed to figure out what data has a higher correlation with readmission rates.
5) Look at the scatterplot above.

What are the advantages and disadvantages of using this plot to convey information?
Construct another plot that conveys the...
Can you write Python code for it?

import seaborn as sns

sns.set()
plt.scatter(number_of_discharges, excess_readmission_ratio, alpha=0.5)

slope, intercept = np.polyfit(number_of_discharges, excess_readmission_ratio, 1)

x = np.array([0, max(number_of_discharges)])
y = slope * x + intercept

plt.plot(x, y)

plt.xlabel('Number of discharges')
plt.ylabel('Excess rate of readmissions')
plt.title('Scatterplot of number of discharges vs. excess rate of readmissions')

plt.show()


I am working on the file "Reduce.ipynb".
The first task is:
Note how we keep reducing the sequence until a single final value is obtained. Lets see another example:
Can you write Python code for it?

#Find the maximum of a sequence (This already exists as max())
max_find = lambda a,b: a if (a > b) else b

#Find max
reduce(max_find,lst)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/01-Python-Crash-Course/03-Python Crash Course Exercises - Solutions.ipynb".
The first task is:
Create a basic function that returns True if the word 'dog' is contained in the input string. Don't worry about edge cases like a punctuation being attached to the word dog, but do account for capitalization.
Can you write Python code for it?

def findDog(st):
    return 'dog' in st.lower().split()

findDog('Is there a dog here?')


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/01-Python-Crash-Course/03-Python Crash Course Exercises - Solutions.ipynb".
The first task is:
Create a function that grabs the email website domain from a string in the form:
user@domain.com

So for example, passing "user@domain.com" would return: domain.com
Can you write Python code for it?

def domainGet(email):
    return email.split('@')[-1]

domainGet('user@domain.com')


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/01-Python-Crash-Course/03-Python Crash Course Exercises - Solutions.ipynb".
The first task is:
Final Problem
You are driving a little too fast, and a police officer stops you. Write a function
  to return one of 3 possible results: "No ticket", "Small ticket", or "Big Ticket".
  If your speed is 60 or less, the result is "No Ticket". If speed is between 61
  and 80 inclusive, the result is "Small Ticket". If speed is 81 or more, the result is "Big    Ticket". Unless it is your birthday (encoded as a boolean value in the parameters of the function) -- on your birthday, your speed can be 5 higher in all
  cases.
Can you write Python code for it?

def caught_speeding(speed, is_birthday):

    if is_birthday:
        speeding = speed - 5
    else:
        speeding = speed

    if speeding > 80:
        return 'Big Ticket'
    elif speeding > 60:
        return 'Small Ticket'
    else:
        return 'No Ticket'

caught_speeding(81,True)

caught_speeding(81,False)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/01-Python-Crash-Course/03-Python Crash Course Exercises - Solutions.ipynb".
The first task is:
Given the variables:
planet = "Earth"
diameter = 12742

 Use .format() to print the following string:
The diameter of Earth is 12742 kilometers.
Can you write Python code for it?

planet = "Earth"
diameter = 12742

print("The diameter of {} is {} kilometers.".format(planet,diameter))


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/01-Python-Crash-Course/03-Python Crash Course Exercises - Solutions.ipynb".
The first task is:
Given this nested list, use indexing to grab the word "hello"
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/01-Python-Crash-Course/03-Python Crash Course Exercises - Solutions.ipynb".
The first task is:
Split this string:
s = "Hi there Sam!"

into a list.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/01-Python-Crash-Course/03-Python Crash Course Exercises - Solutions.ipynb".
The first task is:
Use lambda expressions and the filter() function to filter out words from a list that don't start with the letter 's'. For example:
seq = ['soup','dog','salad','cat','great']

should be filtered down to:
['soup','salad']
Can you write Python code for it?

seq = ['soup','dog','salad','cat','great']

list(filter(lambda word: word[0]=='s',seq))


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/01-NumPy Arrays.ipynb".
The first task is:
Numpy has many built-in functions and capabilities. We won't cover them all but instead we will focus on some of the most important aspects of Numpy: vectors,arrays,matrices, and number generation. Let's start by discussing arrays.
Numpy Arrays
NumPy arrays are the main way we will use Numpy throughout the course. Numpy arrays essentially come in two flavors: vectors and matrices. Vectors are strictly 1-d arrays and matrices are 2-d (but you should note a matrix can still have only one row or one column).
Let's begin our introduction by exploring how to create NumPy arrays.
Creating NumPy Arrays
From a Python List
We can create an array by directly converting a list or list of lists:
Can you write Python code for it?

my_list = [1,2,3]
my_list

np.array(my_list)

my_matrix = [[1,2,3],[4,5,6],[7,8,9]]
my_matrix

np.array(my_matrix)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/01-NumPy Arrays.ipynb".
The first task is:
Random
Numpy also has lots of ways to create random number arrays:
rand
Create an array of the given shape and populate it with
random samples from a uniform distribution
over [0, 1).
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/01-NumPy Arrays.ipynb".
The first task is:
Reshape
Returns an array containing the same data with a new shape.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/01-NumPy Arrays.ipynb".
The first task is:
Shape
Shape is an attribute that arrays have (not a method):
Can you write Python code for it?

# Vector
arr.shape

# Notice the two sets of brackets
arr.reshape(1,25)

arr.reshape(1,25).shape

arr.reshape(25,1)

arr.reshape(25,1).shape


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/01-NumPy Arrays.ipynb".
The first task is:
dtype
You can also grab the data type of the object in the array:
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/01-NumPy Arrays.ipynb".
The first task is:
eye
Creates an identity matrix
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/01-NumPy Arrays.ipynb".
The first task is:
max,min,argmax,argmin
These are useful methods for finding max or min values. Or to find their index locations using argmin or argmax
Can you write Python code for it?

ranarr

ranarr.max()

ranarr.argmax()

ranarr.min()

ranarr.argmin()


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/01-NumPy Arrays.ipynb".
The first task is:
randn
Return a sample (or samples) from the "standard normal" distribution. Unlike rand which is uniform:
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/02-Numpy Indexing and Selection.ipynb".
The first task is:
Bracket Indexing and Selection
The simplest way to pick one or some elements of an array looks very similar to python lists:
Can you write Python code for it?

#Get a value at an index
arr[8]

#Get values in a range
arr[1:5]

#Get values in a range
arr[0:5]


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/02-Numpy Indexing and Selection.ipynb".
The first task is:
Broadcasting
Numpy arrays differ from a normal Python list because of their ability to broadcast:
Can you write Python code for it?

#Setting a value with index range (Broadcasting)
arr[0:5]=100

#Show
arr

# Reset array, we'll see why I had to reset in  a moment
arr = np.arange(0,11)

#Show
arr

#Important notes on Slices
slice_of_arr = arr[0:6]

#Show slice
slice_of_arr

#Change Slice
slice_of_arr[:]=99

#Show Slice again
slice_of_arr


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/02-Numpy Indexing and Selection.ipynb".
The first task is:
Data is not copied, it's a view of the original array! This avoids memory problems!
Can you write Python code for it?

#To get a copy, need to be explicit
arr_copy = arr.copy()

arr_copy


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/02-Numpy Indexing and Selection.ipynb".
The first task is:
More Indexing Help
Indexing a 2d matrix can be a bit confusing at first, especially when you start to add in step size. Try google image searching NumPy indexing to fins useful images, like this one:
<img src= 'http://memory.osu.edu/classes/python/_images/numpy_indexing.png' width=500/>
Selection
Let's briefly go over how to use brackets for selection based off of comparison operators.
Can you write Python code for it?

arr = np.arange(1,11)
arr

arr > 4

bool_arr = arr>4

bool_arr

arr[bool_arr]

arr[arr>2]

x = 2
arr[arr>x]


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/02-Python-for-Data-Analysis-NumPy/02-Numpy Indexing and Selection.ipynb".
The first task is:
Now note the changes also occur in our original array!
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/04-Pandas-Exercises/02-SF Salaries Exercise - Solutions.ipynb".
The first task is:
Bonus: Is there a correlation between length of the Job Title string and Salary?
Can you write Python code for it?

sal['title_len'] = sal['JobTitle'].apply(len)

sal[['title_len','TotalPayBenefits']].corr() # No correlation.


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/04-Pandas-Exercises/02-SF Salaries Exercise - Solutions.ipynb".
The first task is:
How many Job Titles were represented by only one person in 2013? (e.g. Job Titles with only one occurence in 2013?)
Can you write Python code for it?

sum(sal[sal['Year']==2013]['JobTitle'].value_counts() == 1) # pretty tricky way to do this...


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/04-Pandas-Exercises/02-SF Salaries Exercise - Solutions.ipynb".
The first task is:
How many unique job titles are there?
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/04-Pandas-Exercises/02-SF Salaries Exercise - Solutions.ipynb".
The first task is:
How much does JOSEPH DRISCOLL make (including benefits)?
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/04-Pandas-Exercises/02-SF Salaries Exercise - Solutions.ipynb".
The first task is:
Read Salaries.csv as a dataframe called sal.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/04-Pandas-Exercises/02-SF Salaries Exercise - Solutions.ipynb".
The first task is:
Use the .info() method to find out how many entries there are.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/04-Pandas-Exercises/02-SF Salaries Exercise - Solutions.ipynb".
The first task is:
What is the name of lowest paid person (including benefits)? Do you notice something strange about how much he or she is paid?
Can you write Python code for it?

sal[sal['TotalPayBenefits']== sal['TotalPayBenefits'].min()] #['EmployeeName']
# or
# sal.loc[sal['TotalPayBenefits'].idxmax()]['EmployeeName']

## ITS NEGATIVE!! VERY STRANGE


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
A common issue with matplolib is overlapping subplots or figures. We ca use fig.tight_layout() or plt.tight_layout() method, which automatically adjusts the positions of the axes on the figure canvas so that there is no overlapping content:
Can you write Python code for it?

fig, axes = plt.subplots(nrows=1, ncols=2)

for ax in axes:
    ax.plot(x, y, 'g')
    ax.set_xlabel('x')
    ax.set_ylabel('y')
    ax.set_title('title')

fig
plt.tight_layout()


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Axis labels
Similarly, with the methods set_xlabel and set_ylabel, we can set the labels of the X and Y axes:
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Basic Matplotlib Commands
We can create a very simple line plot using the following ( I encourage you to pause and use Shift+Tab along the way to check out the document strings for the functions we are using).
Can you write Python code for it?

plt.plot(x, y, 'r') # 'r' is the color red
plt.xlabel('X Axis Title Here')
plt.ylabel('Y Axis Title Here')
plt.title('String Title Here')
plt.show()


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Code is a little more complicated, but the advantage is that we now have full control of where the plot axes are placed, and we can easily add more than one axis to the figure:
Can you write Python code for it?

# Creates blank canvas
fig = plt.figure()

axes1 = fig.add_axes([0.1, 0.1, 0.8, 0.8]) # main axes
axes2 = fig.add_axes([0.2, 0.5, 0.4, 0.3]) # inset axes

# Larger Figure Axes 1
axes1.plot(x, y, 'b')
axes1.set_xlabel('X_label_axes2')
axes1.set_ylabel('Y_label_axes2')
axes1.set_title('Axes 2 Title')

# Insert Figure Axes 2
axes2.plot(y, x, 'r')
axes2.set_xlabel('X_label_axes2')
axes2.set_ylabel('Y_label_axes2')
axes2.set_title('Axes 2 Title');


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Colors with the color= parameter
We can also define colors by their names or RGB hex codes and optionally provide an alpha value using the color and alpha keyword arguments. Alpha indicates opacity.
Can you write Python code for it?

fig, ax = plt.subplots()

ax.plot(x, x+1, color="blue", alpha=0.5) # half-transparant
ax.plot(x, x+2, color="#8B008B")        # RGB hex code
ax.plot(x, x+3, color="#FF8C00")        # RGB hex code


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Control over axis appearance
In this section we will look at controlling axis sizing properties in a matplotlib figure.
Plot range
We can configure the ranges of the axes using the set_ylim and set_xlim methods in the axis object, or axis('tight') for automatically getting "tightly fitted" axes ranges:
Can you write Python code for it?

fig, axes = plt.subplots(1, 3, figsize=(12, 4))

axes[0].plot(x, x**2, x, x**3)
axes[0].set_title("default axes ranges")

axes[1].plot(x, x**2, x, x**3)
axes[1].axis('tight')
axes[1].set_title("tight axes")

axes[2].plot(x, x**2, x, x**3)
axes[2].set_ylim([0, 60])
axes[2].set_xlim([2, 5])
axes[2].set_title("custom axes range");


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Creating Multiplots on Same Canvas
Can you write Python code for it?

# plt.subplot(nrows, ncols, plot_number)
plt.subplot(1,2,1)
plt.plot(x, y, 'r--') # More on color options later
plt.subplot(1,2,2)
plt.plot(y, x, 'g*-');


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Figure size, aspect ratio and DPI
Matplotlib allows the aspect ratio, DPI and figure size to be specified when the Figure object is created. You can use the figsize and dpi keyword arguments.
* figsize is a tuple of the width and height of the figure in inches
* dpi is the dots-per-inch (pixel per inch).
For example:
Can you write Python code for it?

fig = plt.figure(figsize=(8,4), dpi=100)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Here we can also optionally specify the DPI and choose between different output formats:
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Line and marker styles
To change the line width, we can use the linewidth or lw keyword argument. The line style can be selected using the linestyle or ls keyword arguments:
Can you write Python code for it?

fig, ax = plt.subplots(figsize=(12,6))

ax.plot(x, x+1, color="red", linewidth=0.25)
ax.plot(x, x+2, color="red", linewidth=0.50)
ax.plot(x, x+3, color="red", linewidth=1.00)
ax.plot(x, x+4, color="red", linewidth=2.00)

# possible linestype options ‘-‘, ‘–’, ‘-.’, ‘:’, ‘steps’
ax.plot(x, x+5, color="green", lw=3, linestyle='-')
ax.plot(x, x+6, color="green", lw=3, ls='-.')
ax.plot(x, x+7, color="green", lw=3, ls=':')

# custom dash
line, = ax.plot(x, x+8, color="black", lw=1.50)
line.set_dashes([5, 10, 15, 10]) # format: line length, space length, ...

# possible marker symbols: marker = '+', 'o', '*', 's', ',', '.', '1', '2', '3', '4', ...
ax.plot(x, x+ 9, color="blue", lw=3, ls='-', marker='+')
ax.plot(x, x+10, color="blue", lw=3, ls='--', marker='o')
ax.plot(x, x+11, color="blue", lw=3, ls='-', marker='s')
ax.plot(x, x+12, color="blue", lw=3, ls='--', marker='1')

# marker size and color
ax.plot(x, x+13, color="purple", lw=1, ls='-', marker='o', markersize=2)
ax.plot(x, x+14, color="purple", lw=1, ls='-', marker='o', markersize=4)
ax.plot(x, x+15, color="purple", lw=1, ls='-', marker='o', markersize=8, markerfacecolor="red")
ax.plot(x, x+16, color="purple", lw=1, ls='-', marker='s', markersize=8,
        markerfacecolor="yellow", markeredgewidth=3, markeredgecolor="green");


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Matplotlib Object Oriented Method
Now that we've seen the basics, let's break it all down with a more formal introduction of Matplotlib's Object Oriented API. This means we will instantiate figure objects and then call methods or attributes from that object.
Introduction to the Object Oriented Method
The main idea in using the more formal Object Oriented method is to create figure objects and then just call methods or attributes off of that object. This approach is nicer when dealing with a canvas that has multiple plots on it.
To begin we create a figure instance. Then we can add axes to that figure:
Can you write Python code for it?

# Create Figure (empty canvas)
fig = plt.figure()

# Add set of axes to figure
axes = fig.add_axes([0.1, 0.1, 0.8, 0.8]) # left, bottom, width, height (range 0 to 1)

# Plot on that set of axes
axes.plot(x, y, 'b')
axes.set_xlabel('Set X Label') # Notice the use of set_ to begin methods
axes.set_ylabel('Set y Label')
axes.set_title('Set Title')


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Saving figures
Matplotlib can generate high-quality output in a number formats, including PNG, JPG, EPS, SVG, PGF and PDF.
To save a figure to a file we can use the savefig method in the Figure class:
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Setting colors, linewidths, linetypes
Matplotlib gives you a lot of options for customizing colors, linewidths, and linetypes.
There is the basic MATLAB like syntax (which I would suggest you avoid using for more clairty sake:
Colors with MatLab like syntax
With matplotlib, we can define the colors of lines and other graphical elements in a number of ways. First of all, we can use the MATLAB-like syntax where 'b' means blue, 'g' means green, etc. The MATLAB API for selecting line styles are also supported: where, for example, 'b.-' means a blue line with dots:
Can you write Python code for it?

# MATLAB style line color and style
fig, ax = plt.subplots()
ax.plot(x, x**2, 'b.-') # blue line with dots
ax.plot(x, x**3, 'g--') # green dashed line


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
The same arguments can also be passed to layout managers, such as the subplots function:
Can you write Python code for it?

fig, axes = plt.subplots(figsize=(12,3))

axes.plot(x, y, 'r')
axes.set_xlabel('x')
axes.set_ylabel('y')
axes.set_title('title');


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
Then you can specify the number of rows and columns when creating the subplots() object:
Can you write Python code for it?

# Empty canvas of 1 by 2 subplots
fig, axes = plt.subplots(nrows=1, ncols=2)

# Axes is an array of axes to plot on
axes


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
We can iterate through this array:
Can you write Python code for it?

for ax in axes:
    ax.plot(x, y, 'b')
    ax.set_xlabel('x')
    ax.set_ylabel('y')
    ax.set_title('title')

# Display the figure object
fig


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
You'll also need to use this line to see plots in the notebook:
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/01-Matplotlib Concepts Lecture.ipynb".
The first task is:
subplots()
The plt.subplots() object will act as a more automatic axis manager.
Basic use cases:
Can you write Python code for it?

# Use similar to plt.figure() except use tuple unpacking to grab fig and axes
fig, axes = plt.subplots()

# Now use the axes object to add stuff to plot
axes.plot(x, y, 'r')
axes.set_xlabel('x')
axes.set_ylabel('y')
axes.set_title('title');


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
3D figures
To use 3D graphics in matplotlib, we first need to create an instance of the Axes3D class. 3D axes can be added to a matplotlib figure canvas in exactly the same way as 2D axes; or, more conveniently, by passing a projection='3d' keyword argument to the add_axes or add_subplot methods.
Can you write Python code for it?

from mpl_toolkits.mplot3d.axes3d import Axes3D


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
Axes where x and y is zero
Can you write Python code for it?

fig, ax = plt.subplots()

ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')

ax.xaxis.set_ticks_position('bottom')
ax.spines['bottom'].set_position(('data',0)) # set position of x spine to x=0

ax.yaxis.set_ticks_position('left')
ax.spines['left'].set_position(('data',0))   # set position of y spine to y=0

xx = np.linspace(-0.75, 1., 100)
ax.plot(xx, xx**3);


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
Axis grid
With the grid method in the axis object, we can turn on and off grid lines. We can also customize the appearance of the grid lines using the same keyword arguments as the plot function:
Can you write Python code for it?

fig, axes = plt.subplots(1, 2, figsize=(10,3))

# default grid appearance
axes[0].plot(x, x**2, x, x**3, lw=2)
axes[0].grid(True)

# custom grid appearance
axes[1].plot(x, x**2, x, x**3, lw=2)
axes[1].grid(color='b', alpha=0.5, linestyle='dashed', linewidth=0.5)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
Axis number and axis label spacing
Can you write Python code for it?

# distance between x and y axis and the numbers on the axes
matplotlib.rcParams['xtick.major.pad'] = 5
matplotlib.rcParams['ytick.major.pad'] = 5

fig, ax = plt.subplots(1, 1)

ax.plot(x, x**2, x, np.exp(x))
ax.set_yticks([0, 50, 100, 150])

ax.set_title("label and axis spacing")

# padding between axis label and axis numbers
ax.xaxis.labelpad = 5
ax.yaxis.labelpad = 5

ax.set_xlabel("x")
ax.set_ylabel("y");

# restore defaults
matplotlib.rcParams['xtick.major.pad'] = 3
matplotlib.rcParams['ytick.major.pad'] = 3


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
Colormap and contour figures
Colormaps and contour figures are useful for plotting functions of two variables. In most of these functions we will use a colormap to encode one dimension of the data. There are a number of predefined colormaps. It is relatively straightforward to define custom colormaps. For a list of pre-defined colormaps, see: http://www.scipy.org/Cookbook/Matplotlib/Show_colormaps
Can you write Python code for it?

alpha = 0.7
phi_ext = 2 * np.pi * 0.5

def flux_qubit_potential(phi_m, phi_p):
    return 2 + alpha - 2 * np.cos(phi_p) * np.cos(phi_m) - alpha * np.cos(phi_ext - 2*phi_p)

phi_m = np.linspace(0, 2*np.pi, 100)
phi_p = np.linspace(0, 2*np.pi, 100)
X,Y = np.meshgrid(phi_p, phi_m)
Z = flux_qubit_potential(X, Y).T


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
Placement of ticks and custom tick labels
We can explicitly determine where we want the axis ticks with set_xticks and set_yticks, which both take a list of values for where on the axis the ticks are to be placed. We can also use the set_xticklabels and set_yticklabels methods to provide a list of custom text labels for each tick location:
Can you write Python code for it?

fig, ax = plt.subplots(figsize=(10, 4))

ax.plot(x, x**2, x, x**3, lw=2)

ax.set_xticks([1, 2, 3, 4, 5])
ax.set_xticklabels([r'$\alpha$', r'$\beta$', r'$\gamma$', r'$\delta$', r'$\epsilon$'], fontsize=18)

yticks = [0, 50, 100, 150]
ax.set_yticks(yticks)
ax.set_yticklabels(["$%.1f$" % y for y in yticks], fontsize=18); # use LaTeX formatted labels


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
There are a number of more advanced methods for controlling major and minor tick placement in matplotlib figures, such as automatic placement according to different policies. See http://matplotlib.org/api/ticker_api.html for details.
Scientific notation
With large numbers on axes, it is often better use scientific notation:
Can you write Python code for it?

fig, ax = plt.subplots(1, 1)

ax.plot(x, x**2, x, np.exp(x))
ax.set_title("scientific notation")

ax.set_yticks([0, 50, 100, 150])

from matplotlib import ticker
formatter = ticker.ScalarFormatter(useMathText=True)
formatter.set_scientific(True)
formatter.set_powerlimits((-1,1))
ax.yaxis.set_major_formatter(formatter)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
Twin axes
Sometimes it is useful to have dual x or y axes in a figure; for example, when plotting curves with different units together. Matplotlib supports this with the twinx and twiny functions:
Can you write Python code for it?

fig, ax1 = plt.subplots()

ax1.plot(x, x**2, lw=2, color="blue")
ax1.set_ylabel(r"area $(m^2)$", fontsize=18, color="blue")
for label in ax1.get_yticklabels():
    label.set_color("blue")

ax2 = ax1.twinx()
ax2.plot(x, x**3, lw=2, color="red")
ax2.set_ylabel(r"volume $(m^3)$", fontsize=18, color="red")
for label in ax2.get_yticklabels():
    label.set_color("red")


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
add_axes
Manually adding axes with add_axes is useful for adding insets to figures:
Can you write Python code for it?

fig, ax = plt.subplots()

ax.plot(xx, xx**2, xx, xx**3)
fig.tight_layout()

# inset
inset_ax = fig.add_axes([0.2, 0.55, 0.35, 0.35]) # X, Y, width, height

inset_ax.plot(xx, xx**2, xx, xx**3)
inset_ax.set_title('zoom near origin')

# set axis range
inset_ax.set_xlim(-.2, .2)
inset_ax.set_ylim(-.005, .01)

# set axis tick locations
inset_ax.set_yticks([0, 0.005, 0.01])
inset_ax.set_xticks([-0.1,0,.1]);


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
imshow
Can you write Python code for it?

fig, ax = plt.subplots()

im = ax.imshow(Z, cmap=matplotlib.cm.RdBu, vmin=abs(Z).min(), vmax=abs(Z).max(), extent=[0, 1, 0, 1])
im.set_interpolation('bilinear')

cb = fig.colorbar(im, ax=ax)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/05-Data-Visualization-with-Matplotlib/04-Advanced Matplotlib Concepts.ipynb".
The first task is:
pcolor
Can you write Python code for it?

fig, ax = plt.subplots()

p = ax.pcolor(X/(2*np.pi), Y/(2*np.pi), Z, cmap=matplotlib.cm.RdBu, vmin=abs(Z).min(), vmax=abs(Z).max())
cb = fig.colorbar(p, ax=ax)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
Call the style:
Can you write Python code for it?

import matplotlib.pyplot as plt
plt.style.use('ggplot')


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
Hexagonal Bin Plot
Useful for Bivariate Data, alternative to scatterplot:
Can you write Python code for it?

df = pd.DataFrame(np.random.randn(1000, 2), columns=['a', 'b'])
df.plot.hexbin(x='a',y='b',gridsize=25,cmap='Oranges')


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
Histograms
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
Kernel Density Estimation plot (KDE)
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
Let's stick with the ggplot style and actually show you how to utilize pandas built-in plotting capabilities!
Plot Types
There are several plot types built-in to pandas, most of them statistical plots by nature:

df.plot.area
df.plot.barh
df.plot.density
df.plot.hist
df.plot.line
df.plot.scatter
df.plot.bar
df.plot.box
df.plot.hexbin
df.plot.kde
df.plot.pie

You can also just call df.plot(kind='hist') or replace that kind argument with any of the key terms shown in the list above (e.g. 'box','barh', etc..)

Let's start going through them!
Area
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
Line Plots
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
Now your plots look like this:
Can you write Python code for it?

df1['A'].hist()

plt.style.use('bmh')
df1['A'].hist()

plt.style.use('dark_background')
df1['A'].hist()

plt.style.use('fivethirtyeight')
df1['A'].hist()

plt.style.use('ggplot')


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
Or use s to indicate size based off another column. s parameter needs to be an array, not just the name of a column:
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
Scatter Plots
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
Style Sheets
Matplotlib has style sheets you can use to make your plots look a little nicer. These style sheets include plot_bmh,plot_fivethirtyeight,plot_ggplot and more. They basically create a set of style rules that your plots follow. I recommend using them, they make all your plots have the same look and feel more professional. You can even create your own if you want your company's plots to all have the same look (it is a bit tedious to create on though).
Here is how to use them.
Before plt.style.use() your plots look like this:
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/07-Pandas-Built-in-Data-Viz/01-Pandas Built-in Data Visualization.ipynb".
The first task is:
You can use c to color based off another column value
Use cmap to indicate colormap to use.
For all the colormaps, check out: http://matplotlib.org/users/colormaps.html
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/09-Geographical-Plotting/02-Choropleth Maps Exercise.ipynb".
The first task is:
Import pandas and read the csv file: 2014_World_Power_Consumption
 Check the head of the DataFrame.
 Referencing the lecture notes, create a Choropleth Plot of the Power Consumption for Countries using the data and layout dictionary.
Can you write Python code for it?

choromap = go.Figure(data = [data],layout = layout)
iplot(choromap,validate=False)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/09-Geographical-Plotting/02-Choropleth Maps Exercise.ipynb".
The first task is:
USA Choropleth
 Import the 2012_Election_Data csv file using pandas.
 Check the head of the DataFrame.
 Now create a plot that displays the Voting-Age Population (VAP) per state. If you later want to play around with other columns, make sure you consider their data type. VAP has already been transformed to a float for you.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/09-Geographical-Plotting/03-Choropleth Maps Exercise - Solutions.ipynb".
The first task is:
Check the head of the DataFrame.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/09-Geographical-Plotting/03-Choropleth Maps Exercise - Solutions.ipynb".
The first task is:
Now create a plot that displays the Voting-Age Population (VAP) per state. If you later want to play around with other columns, make sure you consider their data type. VAP has already been transformed to a float for you.
Can you write Python code for it?

data = dict(type='choropleth',
            colorscale = 'Viridis',
            reversescale = True,
            locations = usdf['State Abv'],
            z = usdf['Voting-Age Population (VAP)'],
            locationmode = 'USA-states',
            text = usdf['State'],
            marker = dict(line = dict(color = 'rgb(255,255,255)',width = 1)),
            colorbar = {'title':"Voting-Age Population (VAP)"}
            )

layout = dict(title = '2012 General Election Voting Data',
              geo = dict(scope='usa',
                         showlakes = True,
                         lakecolor = 'rgb(85,173,240)')
             )

choromap = go.Figure(data = [data],layout = layout)
plot(choromap,validate=False)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/09-Geographical-Plotting/03-Choropleth Maps Exercise - Solutions.ipynb".
The first task is:
USA Choropleth
 Import the 2012_Election_Data csv file using pandas.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Basic Questions
 What are the top 5 zipcodes for 911 calls?
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Check the head of df
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Check the info() of the df
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Create a new column called 'Date' that contains the date from the timeStamp column. You'll need to use apply along with the .date() method.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Did you notice something strange about the Plot?
Can you write Python code for it?

# It is missing some months! 9,10, and 11 are not there.


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Import visualization libraries and set %matplotlib inline.
Can you write Python code for it?

import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style('whitegrid')
%matplotlib inline


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Now create a clustermap using this DataFrame.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Now do the same for Month:
Can you write Python code for it?

sns.countplot(x='Month',data=df,hue='Reason',palette='viridis')

# To relocate the legend
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Now groupby this Date column with the count() aggregate and create a plot of counts of 911 calls.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Now recreate this plot but create 3 separate plots with each plot representing a Reason for the 911 call
Can you write Python code for it?

df[df['Reason']=='Traffic'].groupby('Date').count()['twp'].plot()
plt.title('Traffic')
plt.tight_layout()

df[df['Reason']=='Fire'].groupby('Date').count()['twp'].plot()
plt.title('Fire')
plt.tight_layout()

df[df['Reason']=='EMS'].groupby('Date').count()['twp'].plot()
plt.title('EMS')
plt.tight_layout()


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Now repeat these same plots and operations, for a DataFrame that shows the Month as the column.
Can you write Python code for it?

dayMonth = df.groupby(by=['Day of Week','Month']).count()['Reason'].unstack()
dayMonth.head()

plt.figure(figsize=(12,6))
sns.heatmap(dayMonth,cmap='viridis')

sns.clustermap(dayMonth,cmap='viridis')


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Now see if you can use seaborn's lmplot() to create a linear fit on the number of calls per month. Keep in mind you may need to reset the index to a column.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Now use seaborn to create a countplot of 911 calls by Reason.
Can you write Python code for it?


I am working on the file "Refactored_Py_DS_ML_Bootcamp-master/10-Data-Capstone-Projects/02-911 Calls Data Capstone Project - Solutions.ipynb".
The first task is:
Now use seaborn to create a countplot of the Day of Week column with the hue based off of the Reason column.
Can you write Python code for it?

sns.countplot(x='Day of Week',data=df,hue='Reason',palette='viridis')

# To relocate the legend
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)