I am working on the file "CS231n/reference/CS231n-master/assignment3/LSTM_Captioning.ipynb".
The first task is:
LSTM test-time sampling
Modify the sample method of the CaptioningRNN class to handle the case where self.cell_type is lstm. This should take fewer than 10 lines of code.
When you are done run the following to sample from your overfit LSTM model on some training and validation set samples.
Can you write Python code for it?

for split in ['train', 'val']:
  minibatch = sample_coco_minibatch(small_data, split=split, batch_size=2)
  gt_captions, features, urls = minibatch
  gt_captions = decode_captions(gt_captions, data['idx_to_word'])

  sample_captions = small_lstm_model.sample(features)
  sample_captions = decode_captions(sample_captions, data['idx_to_word'])

  for gt_caption, sample_caption, url in zip(gt_captions, sample_captions, urls):
    plt.imshow(image_from_url(url))
    plt.title('%s\n%s\nGT:%s' % (split, sample_caption, gt_caption))
    plt.axis('off')
    plt.show()


I am working on the file "CS231n/reference/CS231n-master/assignment3/LSTM_Captioning.ipynb".
The first task is:
LSTM: backward
Implement the backward pass for an LSTM over an entire timeseries of data in the function lstm_backward in the file cs231n/rnn_layers.py. When you are done run the following to perform numeric gradient checking on your implementation. You should see errors around 1e-8 or less.
Can you write Python code for it?

from cs231n.rnn_layers import lstm_forward, lstm_backward

N, D, T, H = 2, 3, 10, 6

x = np.random.randn(N, T, D)
h0 = np.random.randn(N, H)
Wx = np.random.randn(D, 4 * H)
Wh = np.random.randn(H, 4 * H)
b = np.random.randn(4 * H)

out, cache = lstm_forward(x, h0, Wx, Wh, b)

dout = np.random.randn(*out.shape)

dx, dh0, dWx, dWh, db = lstm_backward(dout, cache)

fx = lambda x: lstm_forward(x, h0, Wx, Wh, b)[0]
fh0 = lambda h0: lstm_forward(x, h0, Wx, Wh, b)[0]
fWx = lambda Wx: lstm_forward(x, h0, Wx, Wh, b)[0]
fWh = lambda Wh: lstm_forward(x, h0, Wx, Wh, b)[0]
fb = lambda b: lstm_forward(x, h0, Wx, Wh, b)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
dh0_num = eval_numerical_gradient_array(fh0, h0, dout)
dWx_num = eval_numerical_gradient_array(fWx, Wx, dout)
dWh_num = eval_numerical_gradient_array(fWh, Wh, dout)
db_num = eval_numerical_gradient_array(fb, b, dout)

print 'dx error: ', rel_error(dx_num, dx)
print 'dh0 error: ', rel_error(dx_num, dx)
print 'dWx error: ', rel_error(dx_num, dx)
print 'dWh error: ', rel_error(dx_num, dx)
print 'db error: ', rel_error(dx_num, dx)


I am working on the file "CS231n/reference/CS231n-master/assignment3/LSTM_Captioning.ipynb".
The first task is:
LSTM: forward
In the function lstm_forward in the file cs231n/rnn_layers.py, implement the lstm_forward function to run an LSTM forward on an entire timeseries of data.
When you are done run the following to check your implementation. You should see an error around 1e-7.
Can you write Python code for it?


I am working on the file "CS231n/reference/CS231n-master/assignment3/LSTM_Captioning.ipynb".
The first task is:
Load MS-COCO data
As in the previous notebook, we will use the Microsoft COCO dataset for captioning.
Can you write Python code for it?

# Load COCO data from disk; this returns a dictionary
# We'll work with dimensionality-reduced features for this notebook, but feel
# free to experiment with the original features by changing the flag below.
data = load_coco_data(pca_features=True)

# Print out all the keys and values from the data dictionary
for k, v in data.iteritems():
  if type(v) == np.ndarray:
    print k, type(v), v.shape, v.dtype
  else:
    print k, type(v), len(v)


I am working on the file "CS231n/reference/CS231n-master/assignment3/LSTM_Captioning.ipynb".
The first task is:
Overfit LSTM captioning model
Run the following to overfit an LSTM captioning model on the same small dataset as we used for the RNN above.
Can you write Python code for it?

small_data = load_coco_data(max_train=50)

small_lstm_model = CaptioningRNN(
          cell_type='lstm',
          word_to_idx=data['word_to_idx'],
          input_dim=data['train_features'].shape[1],
          hidden_dim=512,
          wordvec_dim=256,
          dtype=np.float32,
        )

small_lstm_solver = CaptioningSolver(small_lstm_model, small_data,
           update_rule='adam',
           num_epochs=50,
           batch_size=25,
           optim_config={
             'learning_rate': 5e-3,
           },
           lr_decay=0.995,
           verbose=True, print_every=10,
         )

small_lstm_solver.train()

# Plot the training losses
plt.plot(small_lstm_solver.loss_history)
plt.xlabel('Iteration')
plt.ylabel('Loss')
plt.title('Training loss history')
plt.show()


I am working on the file "CS231n/reference/CS231n-master/assignment3/RNN_Captioning.ipynb".
The first task is:
Microsoft COCO
For this exercise we will use the 2014 release of the Microsoft COCO dataset which has become the standard testbed for image captioning. The dataset consists of 80,000 training images and 40,000 validation images, each annotated with 5 captions written by workers on Amazon Mechanical Turk.
To download the data, change to the cs231n/datasets directory and run the script get_coco_captioning.sh.
We have preprocessed the data and extracted features for you already. For all images we have extracted features from the fc7 layer of the VGG-16 network pretrained on ImageNet; these features are stored in the files train2014_vgg16_fc7.h5 and val2014_vgg16_fc7.h5 respectively. To cut down on processing time and memory requirements, we have reduced the dimensionality of the features from 4096 to 512; these features can be found in the files train2014_vgg16_fc7_pca.h5 and val2014_vgg16_fc7_pca.h5.
The raw images take up a lot of space (nearly 20GB) so we have not included them in the download. However all images are taken from Flickr, and URLs of the training and validation images are stored in the files train2014_urls.txt and val2014_urls.txt respectively. This allows you to download images on the fly for visualization. Since images are downloaded on-the-fly, you must be connected to the internet to view images.
Dealing with strings is inefficient, so we will work with an encoded version of the captions. Each word is assigned an integer ID, allowing us to represent a caption by a sequence of integers. The mapping between integer IDs and words is in the file coco2014_vocab.json, and you can use the function decode_captions from the file cs231n/coco_utils.py to convert numpy arrays of integer IDs back into strings.
There are a couple special tokens that we add to the vocabulary. We prepend a special <START> token and append an <END> token to the beginning and end of each caption respectively. Rare words are replaced with a special <UNK> token (for "unknown"). In addition, since we want to...
Can you write Python code for it?


I am working on the file "CS231n/reference/CS231n-master/assignment3/RNN_Captioning.ipynb".
The first task is:
Run the following cell to perform numeric gradient checking on the CaptioningRNN class; you should errors around 1e-7 or less.
Can you write Python code for it?

batch_size = 2
timesteps = 3
input_dim = 4
wordvec_dim = 5
hidden_dim = 6
word_to_idx = {'<NULL>': 0, 'cat': 2, 'dog': 3}
vocab_size = len(word_to_idx)

captions = np.random.randint(vocab_size, size=(batch_size, timesteps))
features = np.random.randn(batch_size, input_dim)

model = CaptioningRNN(word_to_idx,
          input_dim=input_dim,
          wordvec_dim=wordvec_dim,
          hidden_dim=hidden_dim,
          cell_type='rnn',
          dtype=np.float64,
        )

loss, grads = model.loss(features, captions)

for param_name in sorted(grads):
  f = lambda _: model.loss(features, captions)[0]
  param_grad_num = eval_numerical_gradient(f, model.params[param_name], verbose=False, h=1e-6)
  e = rel_error(param_grad_num, grads[param_name])
  print '%s relative error: %e' % (param_name, e)


I am working on the file "CS231n/reference/CS231n-master/assignment3/RNN_Captioning.ipynb".
The first task is:
Temporal Affine layer
At every timestep we use an affine function to transform the RNN hidden vector at that timestep into scores for each word in the vocabulary. Because this is very similar to the affine layer that you implemented in assignment 2, we have provided this function for you in the temporal_affine_forward and temporal_affine_backward functions in the file cs231n/rnn_layers.py. Run the following to perform numeric gradient checking on the implementation.
Can you write Python code for it?

# Gradient check for temporal affine layer
N, T, D, M = 2, 3, 4, 5

x = np.random.randn(N, T, D)
w = np.random.randn(D, M)
b = np.random.randn(M)

out, cache = temporal_affine_forward(x, w, b)

dout = np.random.randn(*out.shape)

fx = lambda x: temporal_affine_forward(x, w, b)[0]
fw = lambda w: temporal_affine_forward(x, w, b)[0]
fb = lambda b: temporal_affine_forward(x, w, b)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
dw_num = eval_numerical_gradient_array(fw, w, dout)
db_num = eval_numerical_gradient_array(fb, b, dout)

dx, dw, db = temporal_affine_backward(dout, cache)

print 'dx error: ', rel_error(dx_num, dx)
print 'dw error: ', rel_error(dw_num, dw)
print 'db error: ', rel_error(db_num, db)


I am working on the file "CS231n/reference/CS231n-master/assignment3/RNN_Captioning.ipynb".
The first task is:
Temporal Softmax loss
In an RNN language model, at every timestep we produce a score for each word in the vocabulary. We know the ground-truth word at each timestep, so we use a softmax loss function to compute loss and gradient at each timestep. We sum the losses over time and average them over the minibatch.
However there is one wrinke: since we operate over minibatches and different captions may have different lengths, we append <NULL> tokens to the end of each caption so they all have the same length. We don't want these <NULL> tokens to count toward the loss or gradient, so in addition to scores and ground-truth labels our loss function also accepts a mask array that tells it which elements of the scores count towards the loss.
Since this is very similar to the softmax loss function you implemented in assignment 1, we have implemented this loss function for you; look at the temporal_softmax_loss function in the file cs231n/rnn_layers.py.
Run the following cell to sanity check the loss and perform numeric gradient checking on the function.
Can you write Python code for it?

# Sanity check for temporal softmax loss
from cs231n.rnn_layers import temporal_softmax_loss

N, T, V = 100, 1, 10

def check_loss(N, T, V, p):
  x = 0.001 * np.random.randn(N, T, V)
  y = np.random.randint(V, size=(N, T))
  mask = np.random.rand(N, T) <= p
  print temporal_softmax_loss(x, y, mask)[0]

check_loss(100, 1, 10, 1.0)   # Should be about 2.3
check_loss(100, 10, 10, 1.0)  # Should be about 23
check_loss(5000, 10, 10, 0.1) # Should be about 2.3

# Gradient check for temporal softmax loss
N, T, V = 7, 8, 9

x = np.random.randn(N, T, V)
y = np.random.randint(V, size=(N, T))
mask = (np.random.rand(N, T) > 0.5)

loss, dx = temporal_softmax_loss(x, y, mask, verbose=False)

dx_num = eval_numerical_gradient(lambda x: temporal_softmax_loss(x, y, mask)[0], x, verbose=False)

print 'dx error: ', rel_error(dx, dx_num)


I am working on the file "CS231n/reference/CS231n-master/assignment3/RNN_Captioning.ipynb".
The first task is:
Test-time sampling
Unlike classification models, image captioning models behave very differently at training time and at test time. At training time, we have access to the ground-truth caption so we feed ground-truth words as input to the RNN at each timestep. At test time, we sample from the distribution over the vocabulary at each timestep, and feed the sample as input to the RNN at the next timestep.
In the file cs231n/classifiers/rnn.py, implement the sample method for test-time sampling. After doing so, run the following to sample from your overfit model on both training and validation data. The samples on training data should be very good; the samples on validation data probably won't make sense.
Can you write Python code for it?

for split in ['train', 'val']:
  minibatch = sample_coco_minibatch(small_data, split=split, batch_size=2)
  gt_captions, features, urls = minibatch
  gt_captions = decode_captions(gt_captions, data['idx_to_word'])

  sample_captions = small_rnn_model.sample(features)
  sample_captions = decode_captions(sample_captions, data['idx_to_word'])

  for gt_caption, sample_caption, url in zip(gt_captions, sample_captions, urls):
    plt.imshow(image_from_url(url))
    plt.title('%s\n%s\nGT:%s' % (split, sample_caption, gt_caption))
    plt.axis('off')
    plt.show()


I am working on the file "CS231n/reference/CS231n-master/assignment3/RNN_Captioning.ipynb".
The first task is:
Vanilla RNN: step backward
In the file cs231n/rnn_layers.py implement the rnn_step_backward function. After doing so run the following to numerically gradient check your implementation. You should see errors less than 1e-8.
Can you write Python code for it?

from cs231n.rnn_layers import rnn_step_forward, rnn_step_backward

N, D, H = 4, 5, 6
x = np.random.randn(N, D)
h = np.random.randn(N, H)
Wx = np.random.randn(D, H)
Wh = np.random.randn(H, H)
b = np.random.randn(H)

out, cache = rnn_step_forward(x, h, Wx, Wh, b)

dnext_h = np.random.randn(*out.shape)

fx = lambda x: rnn_step_forward(x, h, Wx, Wh, b)[0]
fh = lambda prev_h: rnn_step_forward(x, h, Wx, Wh, b)[0]
fWx = lambda Wx: rnn_step_forward(x, h, Wx, Wh, b)[0]
fWh = lambda Wh: rnn_step_forward(x, h, Wx, Wh, b)[0]
fb = lambda b: rnn_step_forward(x, h, Wx, Wh, b)[0]

dx_num = eval_numerical_gradient_array(fx, x, dnext_h)
dprev_h_num = eval_numerical_gradient_array(fh, h, dnext_h)
dWx_num = eval_numerical_gradient_array(fWx, Wx, dnext_h)
dWh_num = eval_numerical_gradient_array(fWh, Wh, dnext_h)
db_num = eval_numerical_gradient_array(fb, b, dnext_h)

dx, dprev_h, dWx, dWh, db = rnn_step_backward(dnext_h, cache)

print 'dx error: ', rel_error(dx_num, dx)
print 'dprev_h error: ', rel_error(dprev_h_num, dprev_h)
print 'dWx error: ', rel_error(dWx_num, dWx)
print 'dWh error: ', rel_error(dWh_num, dWh)
print 'db error: ', rel_error(db_num, db)


I am working on the file "CS231n/reference/CS231n-master/assignment3/RNN_Captioning.ipynb".
The first task is:
Word embedding: backward
Implement the backward pass for the word embedding function in the function word_embedding_backward. After doing so run the following to numerically gradient check your implementation. You should see errors less than 1e-11.
Can you write Python code for it?

N, T, V, D = 50, 3, 5, 6

x = np.random.randint(V, size=(N, T))
W = np.random.randn(V, D)

out, cache = word_embedding_forward(x, W)
dout = np.random.randn(*out.shape)
dW = word_embedding_backward(dout, cache)

f = lambda W: word_embedding_forward(x, W)[0]
dW_num = eval_numerical_gradient_array(f, W, dout)

print 'dW error: ', rel_error(dW, dW_num)


I am working on the file "CSE681/lab3/bicycle_rent.ipynb".
The first task is:
Bazı verileri (outlier) temizleyerek yeniden model oluşturacağız

Sadece 500 den büyük kiralamaları ve rüzgar hızı 0.4 den küçük olan verileri yeni bir dataframe de birleştiriniz. (bisikletKiraYeni)
Matplot kullanarak. bu verileri scatter plot ile çiziniz.
Can you write Python code for it?

filter1 = bisikletKiralama['cnt'] > 500
filter2 = bisikletKiralama['windspeed'] < 0.4
bisikletKiralama = bisikletKiralama[(filter1) & (filter2)]
count = bisikletKiralama.count()

plt.style.use('seaborn-talk')
plt.scatter(bisikletKiralama['windspeed'], bisikletKiralama['cnt'])
plt.xlabel('Rüzgar Hızı')
plt.ylabel('Kiralama Sayısı')
plt.show()

# Temizlenmiş verileri kullanarak model2 adında yeni bir model oluşturunuz.
model2=linear_model.LinearRegression()
count = len(bisikletKiralama.index)
model2.fit(bisikletKiralama['windspeed'].values.reshape(count, 1), bisikletKiralama['cnt'].values.reshape(count, 1))

plt.style.use('seaborn-talk')
plt.scatter(bisikletKiralama['windspeed'], bisikletKiralama['cnt'])
plt.plot(bisikletKiralama['windspeed'], model2.predict(bisikletKiralama['windspeed'].values.reshape(count,1)), color='red')
plt.xlabel('Rüzgar Hızı')
plt.ylabel('Kiralama Sayısı')
plt.show()


I am working on the file "CSE681/lab3/bicycle_rent.ipynb".
The first task is:
RSS i hesaplayınız
Aşağıda tek bir nokta için yardımcı olması amacı ile bir hesap verilmiştir.
siz tüm veri noktaları için bu hesabı yapınız ve toplam hatayı bulunuz.
Can you write Python code for it?

# Sadece anlamaya çalışınız bu satırlar toplam hatayı hesaplamanıza yardımcı olacak
#bisKiralamaClear[:1]
#lets show them how to predict single data
print(bisikletKiralama[:1]['windspeed'].values.reshape(-1,1),bisikletKiralama[:1]['cnt'])
#model2.predict(bisikletKiralama[:1]['windspeed'].values.reshape(-1,1))

rssSingle=(bisikletKiralama[:1]['cnt'].values.reshape(-1,1) - model2.predict(bisikletKiralama[:1]['windspeed'].values.reshape(-1,1))) ** 2

#RSS for single point
rssSingle

#Şimdi sizin sıranız. Toplam hatayı hesaplayınız.

RSS = ((bisikletKiralama[:]['cnt'].values.reshape(-1,1) - model2.predict(bisikletKiralama[:]['windspeed'].values.reshape(-1,1))) ** 2).sum()
RSS


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch02-variables-expressions-and-statements.ipynb".
The first task is:
Another exception is the * operator
It repeats a string
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch02-variables-expressions-and-statements.ipynb".
The first task is:
Comments

Comments are a way for you to add notes to your program
They improve understanding of your code by describing the why of the code
Code already provides the how, so don't have your comments repeat that
This assumes that your code is understandable an you have chosen appropriate variable names
Can you write Python code for it?

# Poor example
# Calculate it
y = 2
x = 2 * 3.14 * y

# Good example
# Calculate circumference of a circle
pi = 3.14
radius = 2
circumference = 2 * pi * radius


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch02-variables-expressions-and-statements.ipynb".
The first task is:
Expressions

An expression is a combination of values, variables and operators
A value or a variable is itself an expression
The interpreter evaluates expressions and uses the result for other expressions or simply displays it (if it is executed in the console)
A statement is comprised of one or more expressions

Order of operations

The order in which operators are evaluated in an expression depends on the rules of precedence
Python follows the standard mathematical convention
You may know it as PEMDAS
Parenthesis
Exponent
Multiplication
Division
Addition
Subtraction
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch02-variables-expressions-and-statements.ipynb".
The first task is:
If either operand is a float, the result is a float
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch02-variables-expressions-and-statements.ipynb".
The first task is:
One exception is the + operator
It performs concatenation with strings
This means that it joins strings together
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch02-variables-expressions-and-statements.ipynb".
The first task is:
String operations

In general, you can't use mathematical operators with string values or variables
Even if the strings look like numbers
Can you write Python code for it?

# Can't divide a string by an integer
# Uncomment to demonstrate
# '13' / 42


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch02-variables-expressions-and-statements.ipynb".
The first task is:
The print function displays the value of a variable
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch02-variables-expressions-and-statements.ipynb".
The first task is:
There is a special division operator // called floor division
It throws away the remainder
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch02-variables-expressions-and-statements.ipynb".
The first task is:
Variable names and keywords

Variables names need to be meaningful
Don't use x, y, and z as names unless they have real meaning in the problem (like 3-d points)
Variable names can be arbitrarily long and can contain:
letters
numbers
underscores
They have to begin with a letter
You can use uppercase letters, but start variables with a lowercase letter
Python is case sensitive so max and Max are different variables
An illegal variable name is a syntax error
Python has 30 keywords which are words that mean something specific in the language

Statements

A statement is a unit of code that the Python interpreter can execute
Usually a statement is a single line of code, but
It can be longer depending on the type of statement
A script (or program) usually contains a sequence of statements

Operators and operands

Operators are special symbols that represent computations like addition and multiplication
The value(s) to which the operator is applied is called an operand
Can you write Python code for it?

# Operator: + (addition)
# Operands: 3 and 4
3 + 4

# Operator: - (subtraction)
# Operands: 3 and 4
3 - 4

# Operator: *tiplication (mul)
# Operands: 3 and 4
3 * 4

# Operator: / (division)
# Operands: 3 and 4
3 / 4


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch02-variables-expressions-and-statements.ipynb".
The first task is:
Variables

A variable is a name, or label for a value
It has no inherent meaning
An assignment statement gives a variable a value
In Python, it even creates the variable if it hasn't been created before
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
Bob wants to know how much he should charge for his app on the App store.  He ran two price trials (results below) to gauge interest.  He also has some costs associated with selling his app (listed below).  Write a function called calculate_profit that takes a proposed price for his app and returns the predicted profit earned.  You may assume a simple linear relationship between the purchase price and the number of purchases.

| Purchase price | Number of purchases |
|---------------:|--------------------:|
| \$2.00 | 1,000 |
| \$3.00 | 700 |
| Known costs |  |
|:---|---:|
| Refund rate | 5\% |
| Transaction cost | \$0.50 |
Can you write Python code for it?

# Function to calculate the profit
def calculate_profit( app_price ):
    # ????

    # Profit!
    return 0


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
Composition

Building solutions to large problems using solutions to small problems as a building block is a fundamental skill in thinking like a computer science
The process of combining building blocks is called composition
Almost anywhere you can put a value in Python, you can put an expression
The exception is that only a variable can go on the left of an assignment statement

Adding new functions

You can create your own functions using a function definition
Can you write Python code for it?

def print_star_wars_greeting():
    print( 'May the force be with you' )
    print( 'And also with you' )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
Definitions and uses

Function definitions are executed like other statements, but they just create function objects
These store the statements in the body so they can be executed when the function is called
What happens if we try to call a function before it is defined?
Look back at the previous example
Move print_lyrics() to the top
Now, move print_we_will() in between the other two functions
What is it about Python that might cause this behavior?
You have to create a function before you can execute it

Flow of execution

The order in which statements are executed is called flow of execution
Execution starts at the top
Statements are executed one at a time
Think of a function as a detour to another part of code
When it is done, it comes right back to where it left
Don’t forget that functions can call other functions though

Parameters and arguments

Sometimes functions need additional information in order to execute properly
We have seen builtin functions using arguments
A good example was math.sin( radians )
You can do it with user-defined functions as well
Inside the function, arguments are referred to as parameters
Can you write Python code for it?

def greet( name ):
    print( 'Hello', name )
    print( 'How are you doing?' )
greet( 'Boba Fett' )
robot = 'Vincent'
greet( robot )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
Even if the variables share the same name, they are different variables
Think of this like telephone numbers and area codes.  The same number can exist as a local number in different area codes.

Stack diagrams

TODO Insert an image
Sometimes it is useful to visualize the execution of a program using a stack diagram
Each function is represented by a frame
The frame holds all the important information about a function's execution, like the variables and the current line of execution
The currently executing function is placed on the bottom
Whenever you encounter a runtime error, Python will display an error detailing the current line of code and the function in which it resides
It will also list the line of code that called that function and the function in which it resides
This continues until it gets the main entry point of the program __main__
This is called a stack trace (or sometimes even a trace back)
To visualize the execution of the previous code that prints song lyrics, use this website http://www.pythontutor.com/visualize.html

Fruitful functions and void functions

We have already used functions that return a value
The book refers to these as fruitful functions
Other functions that don't return anything are called void functions
When a function returns something, it is rare that you want to ignore it
If you don't use the value, it is lost
Can you write Python code for it?

def calculate_area_circle( radius ):
    return math.pi * radius ** 2

def calculate_area_donut( radius_outer, radius_inner ):
    area_outer = calculate_area_circle( radius_outer )
    area_inner = calculate_area_circle( radius_inner )
    return area_outer - area_inner

print( calculate_area_donut( 3, 2 ) )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
Exercises

Write a function that takes a radius of a circle as an argument and returns (not prints) the circumference of the circle.  Write a second function that also takes a raidus of a circle as an argument and returns the area of the circle.  Calculate, by hand, the correct answers for circles with radii of 2, 3, and 4.  Test the functions with these values and check for correctness.
Can you write Python code for it?

def calc_circle_circumference( radius ):
    # INSERT YOUR CODE HERE
    return 0

def calc_circle_area( radius ):
    # INSERT YOUR CODE HERE
    return 0


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
Math functions

Beyond builtin functions, Python has a number of modules that provide additional functionality
Think of modules like a library of related functions and variables
Most of the important math functions can be found in the math module
Before you can use it, you need to import it into your code
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
Note that converting from a float to an int can result in the loss of data
Numeric data is atomic
This means you can't split it up into different parts
However, string data is different
Can you write Python code for it?

# Convert an int to a string
str( 42 )

# Convert a float to a string
str( 3.14159 )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
Note that parameters are local as well
x and y don't exist outside of the function
Can you write Python code for it?

def do_something_else( x, y ):
    x = x + 2
    y = y - 1
    z = x * y
    print( 'Inside: x=[' + str( x ) + ']' )
    print( 'Inside: y=[' + str( y ) + ']' )
    print( 'Inside: z=[' + str(z) + ']' )

x = 3
y = 4
do_something_else( x, y )
print( 'Outside: x=[' + str( x ) + ']' )
print( 'Outside: y=[' + str( y ) + ']' )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
The name of this function is type
It is built in to the Python language
The expression in the parenthesis is called the function's argument
The function returns the datatype of the argument, in this case an int, which is short for integer

Type conversion functions

Datatypes are very important in programming languages
A value's type dictates how it can be used
Python has functions that convert a value from one datatype to another
The two basic numeric datatypes are:
int For integer data
float For data with fractional parts expressed with a decimal point
Can you write Python code for it?

# Convert an int to a float
float( 42 )

# Convert a float to an int
int( 3.14159 )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
To access a function or variable defined in the module, you need to specify the name of the module and function using dot notation
Think of this as using an area code to dial a phone number
There could be multiple functions with the same name, but the module name tells Python where to find the one you want
Can you write Python code for it?

radians = math.pi / 2
height = math.sin( radians )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
Variables and parameters are local

Variables created in a function are local to that function
This means that they only exist while that function is executing
They can't be accessed from outside the function and when the function ends, they are thrown away
For example, if we try to use the variable z outside of the function, Python will fail to compile the code and generate an error
Can you write Python code for it?

def do_something( x, y ):
    z = x * y
    print( 'z=[' + str(z) + ']' )

do_something( 3, 4 )
# print( z ) # Fails since z doesn't exist


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
You can find all the functions and variables in the module in the Python documentation https://docs.python.org/3/library/math.html
As an example, let's write code to calculate the circumference of a circle <br />
$c = 2 r \pi$
Can you write Python code for it?

# Define the radius
radius = 3
# Calculate the circumference
circumference = 2 * radius * math.pi
print( circumference )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch03-functions.ipynb".
The first task is:
def is a keyword that tells Python you are defininig a function
The name of the function immediately follows def
The naming rules for functions are the same as for variables
A verb is usually in a function name since it does something
For the moment, leave the parentheses empty
We will add parameters later
The header is the first line of the definition
The body is the rest of the function and is indicated by the :
Python uses the indentation to know where the body of the function starts and stops
The indentation is 4 spaces, not a TAB
Defining functions actually creates a variable with the name of the function and a value referred to as a function object
The function isn't executed until you call it in a __function call_
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch04-interface-design.ipynb".
The first task is:
Interface design

When we generalize a function, we need to think hard about how a user will call the function
We want to make it "as simple as possible, but not simpler"
This part of the function is referred to as its interface
It is:
The name of the function
What the function is supposed to do
The function arguments
The function's return value
We interact with interfaces everyday
Some examples of interfaces are:
TV remote control
ATM
Steering wheel and dashboard in a car
What are some other real-world examples of interfaces?

Refactoring

Sometimes you will discover that you can generalize a function even further after you have already written it
When you rearrange a program (or function) to improve functionality, sometimes by improving function interfaces, it is called refactoring
This happens very often in software development
Programmers rarely, if ever, get functions and programs right the first time
Often, you learn something about the problem only after you start implementing your solution
What sort of things can you learn?  Why do you learn them only after you start on the solution?
Software development is an iterative process
What does this mean?  Have you experienced this in your programs?

A development plan

There are a variety of processes used to develop software
One of the simplest process for novice programmers is encapsulation and generalization
It assumes that you have already decomposed the problem into subproblems
The following are the steps in the process:
Design an algorithm for a small subproblem with no abstraction
Encapsulate and generalize the algorithm
Repeat the previous two steps for additional subproblems
Combine subproblem solutions where appropriate into more general algorithms
This is referred to as a bottom-up approach
Another process goes the other way:
Design a high-level algorithm for the overall problem
Identify subproblems using abstract portions of the algorithm
Expand the abstract portions of the algorithm into more specific algorithms
Repeat the previous two steps...
Can you write Python code for it?

def polyline( t, length, n, angle ):
    """Draw n line segments with the given length and
    angle (in degreees) between them.  t is a turtle.
    """
    for i in range( n ):
        t.fd( length )
        t.lt( angle )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch04-interface-design.ipynb".
The first task is:
Simple repetition

When designing algorithms, we often need to repeat some action over and over
One way to implement this in Python is the for loop
Can you write Python code for it?

for i in range(4):
    print( 'Hello!' )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch04-interface-design.ipynb".
The first task is:
To practice loops and turtle graphics, complete the following exercises from the textbook:
1. Write a function called square that takes a parameter named t, which is a turtle. It should use the turtle to draw a square.  Write a function call that passes bob as an argument to square, and then run the program again.
1. Add another parameter, named length, to square. Modify the body so length of the sides is length, and then modify the function call to provide a second argument. Run the program again. Test your program with a range of values for length.
1. The functions lt and rt make 90-degree turns by default, but you can provide a second argument that specifies the number of degrees. For example, lt(bob, 45) turns bob 45 degrees to the left.  Make a copy of square and change the name to polygon. Add another parameter named n and modify the body so it draws an n-sided regular polygon. Hint: The exterior angles of an n-sided regular polygon are 360.0/n degrees.
1. Write a function called circle that takes a turtle, t, and radius, r, as parameters and that draws an approximate circle by invoking polygon with an appropriate length and number of sides. Test your function with a range of values of r.  Hint: figure out the circumference of the circle and make sure that length * n = circumference.  Another hint: if bob is too slow for you, you can speed him up by changing bob.delay, which is the time between moves, in seconds. bob.delay = 0.01 ought to get him moving.
1. Make a more general version of circle called arc that takes an additional parameter angle, which determines what fraction of a circle to draw. angle is in units of degrees, so when angle=360, arc should draw a complete circle.
If you would like more of a challenge, try these additional exercises:
1. Write a function called drawSpiral that takes a parameter named degrees that specifies the amount in degrees to draw a spiral.
2. Write a function called drawStar that takes a parameter named length that draws a five-sided star with the specified side...
Can you write Python code for it?

def square( t ):
    for i in range( 4 ):
        t.fd( 100 )
        t.lt( 90 )

#square( bob )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch04-interface-design.ipynb".
The first task is:
We can now draw a square of any size
We can even generalize it further to draw a polygon with any number of sides
Can you write Python code for it?

def polygon( t, n, length ):
    # Calculate the angle to turn for each side
    angle = 360.0 / n
    # Draw each side
    for i in range( n ):
        t.fd( length )
        t.lt( angle )

#polygon( bob, 7, 70 )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch04-interface-design.ipynb".
The first task is:
Write a function called draw_circle that takes a parameter t, which is a turtle, and a parameter r, which is a radius.  The function should draw a circle with the specified radius.
Can you write Python code for it?

def draw_circle( t, r ):
    # INSERT YOUR CODE HERE
    print( 'Remove this line' ) # Jupyter needs a statement to compile


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch04-interface-design.ipynb".
The first task is:
Write a function called draw_polygon that takes a parameter t, which is a turtle, a parameter n, which is the number of sides, and a parameter length, which is the length of a side.  The function should draw a polygon with the specified number of sides of the specified length.
Can you write Python code for it?

def draw_polygon( t, n, length ):
    # INSERT YOUR CODE HERE
    print( 'Remove this line' ) # Jupyter needs a statement to compile


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch04-interface-design.ipynb".
The first task is:
Write a function called draw_variable_square that takes a parameter t, which is a turtle, and a parameter length, which is the length of a side.  The function should use the turtle to draw a square with sides of the specified length.
Can you write Python code for it?

def draw_variable_square( t, length ):
    # INSERT YOUR CODE HERE
    print( 'Remove this line' ) # Jupyter needs a statement to compile


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
All operands in a boolean expression should be boolean expressions or boolean variables/values
However, Python can bend the rules
For example, nonzero numbers are considered True
But try to avoid this behavior

Conditional execution

When writing programs, we often need the ability to change the behavior of a program depending on the situation
Conditional statements give us this ability
The simplest version is the if statement
Can you write Python code for it?

x = 1
if( x > 0 ):
    print( 'x is positive' )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
Boolean expressions

A boolean (or bool in Python) is a True or False value
A boolean expression is an expression that results in either a True or False
We have already seen a simple operator that produces a boolean: ==
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
It has been modified so that the conditional is an equality test
What would happen if a non-integer number was passed to the function?
Can you write Python code for it?

# Uncomment this to see what will happen
# countdown_unsafe( 3.5 )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
Recursion

We have seen how one function can call another
It is also legal for a function to call itself
Such functions are called recursive and use a process called recursion
Can you write Python code for it?

def countdown( n ):
    if( n <= 0 ):
        print( 'Blastoff!' )
    else:
        print( n )
        # Recursive call
        countdown( n - 1 )

countdown( 5 )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
The possible paths the code can follow are referred to as branches

Chained conditionals

If you have more than two possibilities, you can build a chained conditional
This is done by adding one or more elif statements
Can you write Python code for it?

y = 0
if( x < y ):
    print( 'x is less than y' )
elif( x > y ):
    print( 'x is greater than y' )
else:
    print( 'x and y are equal' )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
This problem-solving approach is called divide and conquer
It breaks a problem in to smaller chunks that are easier to solve
In this case, the smaller problem is identical in nature to the previous one, just smaller
The function calls itself until n <= 0
This is called the base case
Recursive functions need at least one base case to function properly, but you can have more
You can always convert a recursive function into an iterative function that uses a loop
You can also do the reverse and go from an iterative function to a recursive function
For some problems it is easier to write a recursive function than an iterative one

Infinite recursion

If you forget a base case or have an error in your program so that you never reach a base case, the program will never stop running
The recursive function keeps calling itself, but the problem is never smaller
This is called infinite recursion and is generally considered bad
Due to the way programs work inside a computer, they don't really run forever
You usually get a runtime error like a stack overflow or max recursion depth exceeded
The exact error depends on the programming language
Take care to write your base case conditions carefully to avoid these situations
Consider the countdown function from above
Can you write Python code for it?

def countdown_unsafe( n ):
    if( n == 0 ):
        print( 'Blastoff!' )
    else:
        print( n )
        # Recursive call
        countdown( n - 1 )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
True and False are not strings, nor are the equivalent to strings
They belong to the bool datatype
There are other relational operators besides ==:
!= Not equal
> Greater than
< Less than
>= Greater than or equal to
<= Less than or equal to
Remember that = is an assignment operator and == is a relational operator that tests equality

Logical operators

There are 3 logical operators:
and
or
not
They mean the same thing in Python that they do in English
Can you write Python code for it?

True and True

True and False

True or True

True or False

False or False


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
Using <= acts as a failsafe if an incorrect value is passed to the function
Always assume the worst can happen and write your code accordingly
Think of it as defensive programming (just like defensive driving)

Keyboard input

Up to this point we have primarily relied upon hard-coded input for functions
Python provides the input function that gets input from the keyboard
It returns the user's input as a string value
You can pass a prompt as an argument to the function
If you want the user's input to start below the prompt add a \n newline character
If you need the user's input as something other than a string, convert it
Work some examples
You can even use the eval function to evaluate it as if it where a Python expression

Debugging

Note that when Python indicates that an error was discovered on a particular line, that is only where the problem was detected
It does not mean that is the line where the problem actually occurs
You may have to look at previous lines of code to find the actual error

Exercises

You are given the number of hours in a semester broken down by grade.  For example, of Bob's total 16 hours, he got an A in 6 hours (2 classes), a B in 7 hours (2 classes and a lab), and a C in 3 hours (1 class).  Write a function calculate_gpa that takes the number of hours for each letter grade A through F and computes the GPA.  Test your function with at least three different sets of letter grade hours.
Can you write Python code for it?

def calculate_gpa( a_hours, b_hours, c_hours, d_hours, f_hours ):
    # INSERT YOUR CODE HERE
    return 0


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
When combining multiple expresssions, don't be afraid to use parenthesis
They help make the code more readable and ensure it works as intended
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
Write a function called calculate_fibonacci that takes a parameter n, denoting the $n$-th fibonacci number, and returns the fibonacci number.
Can you write Python code for it?

def calculate_fibonacci( n ):
    # INSERT YOUR CODE HERE
    return 0


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch05-conditionals-and-recursion.ipynb".
The first task is:
elif is an abbreviation for "else if"
Using chained conditionals, only one branch will be taken
Each condition is checked in order from first to last
When a condition is met, the branch is executed and the remainder are ignored, even if others would be true as well
There isn't a limit on the number of elif statements, butdon't go overboard
An else statement isn't required so don't use one if it doesn't make sense
Can you write Python code for it?

# TODO - calculate a tax rate, tip or GPA example


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch06-fruitful-functions.ipynb".
The first task is:
While both functions do the same thing, the second is easier to debug
The first function has three (3) potential paths to the function, meaning there are three paths that need to be checked in case of an error
The second function has only one way in and out of the function, meaning it is easier to track down an error

Incremental development

As you write more and longer functions, you will find yourself debugging more often
As the complexity increases, so does the probability of errors and the need for debugging
Incremental development is a way of dealing with this complexity and reducing the chance of errrors
The goal is to only add small portions of code at a time and test them before moving on
The key aspects of the process are:
Start with a working program and make small, incremental changes.  At any point, if there is an error, you should have a good idea as to where it is.
Use temporary variables to hold intermediate values so you can display and check them.
Once the program is working, remove some of the scaffolding or consolidate multiple statements into compound expressions, but only if it doesn't make the program hard to read.
Displaying the algorithm's current progress is a simple form of logging
Adding print statements for crucial values as you write code is a form of scaffolding
This type of code helps you write the software, but isn't useful in the final version
TODO Work hypotenuse example from the textbook (Ex. 6.2)

Composition

Building larger components from smaller components is called composition
Here, we see it as calling functions from within another function
We did this with the area of a donut function discussed in a previous chapter
Although the book says that we can remove the temporary variable radius and result in its circle_area function once we are done with development, I disagree
It makes the code much more readable and easier to debug/refactor in the future
The idea of composition is fundamental in programming and we will use it consistently

Boolean functions

Functions can...
Can you write Python code for it?

def is_between( x, y, z ):
    return (x <= y) and (y <= z)

print( is_between( 1, 2, 3) )
print( is_between( 1, 0, 3) )

# Do this
if( is_between( 1, 2, 3 ) ):
    print( 'It is between the values' )

# Don't do this
if( is_between( 1, 2, 3 ) == True ):
    print( 'It is between the values' )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch06-fruitful-functions.ipynb".
The first task is:
Write a function that implements multiplication using only addition using both an iterative and a recursive approach.  Which was easier?  Why?
Can you write Python code for it?

def multiply_iterative( n ):
    # INSERT YOUR CODE HERE
    return 0

def multiply_recursive( n ):
    # INSERT YOUR CODE HERE
    return 0


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch07-iteration.ipynb".
The first task is:
Adding 1 to a variable is referred to as incrementing the value
Subtracting 1 from a variable is referred to as decrementing the value
Other languages have operators that do this in one step, but Python doesn't

The while statement

We have previously discussed using a for loop for iteration
Python provides other options for iteration
The while loop can also be used
Can you write Python code for it?

def countdown( n ):
    while( n > 0 ):
        print( n )
        n = n - 1
    print( 'Blastoff!' )

countdown( 5 )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch07-iteration.ipynb".
The first task is:
The loop continues while the conditional test n > 0 is true
Note that this can result in an infinite loop
While loops are often controlled via a single boolean variable
It is referred to as a sentinel variable
While you can have iteration with either a for or while loop, there are generally differences in how they are used
A for loop can be referred to as a definite loop
It is used when you know the number of times the loop should repeat before the loop is ever started
A while loop can be referred to as an indefinite loop
It is used when you don't know how many times a loop should repeat
Notice the difference between an infinite loop and an indefinite loop

Break statement

Sometimes you don't know you need to end a particular iteration of a loop until you are already inside it
In those situations, a break statement will allow you to break out of the loop
Can you write Python code for it?

def echo():
    while( True ):
        line = input( '> ' )
        if( 'done' == line ):
            break
        print( line )
    print( 'Done!' )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch07-iteration.ipynb".
The first task is:
This loop will continue to execute until the user enters the appropriate text
Although break statements can be useful, be careful
Overuse can lead to sloppy and confusing code
There are frequently better options if you take the time to think through the problem

Exercises

Build a program that implements a number guessing game.  The program should be passed a high and low number.  It should then generate a random number in between the high and low values.  The program should then repeatedly prompt the user for a value, noting if the value was too high, too low, or correct.  When the correct value is guessed, the program should terminate.
Can you write Python code for it?

def start_guessing( low, high ):
    # INSERT YOUR CODE HERE
    print( 'Correct!' )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch07-iteration.ipynb".
The first task is:
Updating variables

As we have seen, multiple assignment usually occurs when you update a variable's value
Rmember, you have to initialize the variable before you use it
Can you write Python code for it?

#x = x + 1    # Generates an error

x = 0
x = x + 1    # Works properly now


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch08-strings.ipynb".
The first task is:
As we have previously discussed, uppercase letters come before lowercase letters with respect to encoding
This means that uppercase are considered less than lowercase letters
If you need to compare strings, it is common to convert them to all lowercase (or uppercase) before comparing unless you specifically want to compare using case

Debugging

When you traverse a sequence of items, it is sometimes challenging to get the beginning and ending indices correct
These are frequently referred to as off-by-one errors since you stop one early or one late
To aid in debugging, it is helpful to display not only the index, but the value at the index as well
Can you write Python code for it?

word = 'banana'
index = 2
print( 'index=[', index, '] value=[', word[ index ], ']' )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch08-strings.ipynb".
The first task is:
If the first index is greater than or equal to the second, an empty string is returned

Strings are immutable

Strings are immutable
This means you can't change a string once it has been built
Can you write Python code for it?

greeting = 'Hello world!'
# Uncomment to see the error generated
# greeting[0] = 'J'


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch08-strings.ipynb".
The first task is:
In the error, the object is the string and the item is the character
Don't worry about the idea of an object right now
We will discuss it later
For now, think of it as a variable's value
Since strings are immutable, the only way to "change" one is to create a new one with the changes you want
Can you write Python code for it?

greeting = 'Hello world!'
new_greeting = 'J' + greeting[1:]
print( new_greeting )
print( greeting )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch08-strings.ipynb".
The first task is:
Notice that if the letter isn't found in the word, the function returns -1
It is common to return a special value indicating failure of some kind
However, it needs to be noted in the comments so it is actually useful

String methods

A method is similar to a function
It takes arguments and can return a value, but the syntax to use it is different
It operates on a variable that is an object (like a string)
All the TurtleWorld code used methods on a turtle
Can you write Python code for it?

word = 'banana'
new_word = word.upper()
print( new_word )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch08-strings.ipynb".
The first task is:
Remember that indices are zero-based, while counting is one-based
There are multiple ways to get the last letter
Can you write Python code for it?

last = fruit[ length - 1 ]
last = fruit[ -1 ]


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch08-strings.ipynb".
The first task is:
String slices

A segment of a string is called a slice
The method of selecting a slice is similar to accessing a single character
Can you write Python code for it?

s = 'Monty Python'
print( s[0:5] )
print( s[6:12] )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch08-strings.ipynb".
The first task is:
The operator [n:m] returns a portion of the string
The n-th item is included, but the m-th item is not
TODO insert image
If you omit an index from the slice, it either uses the beginning or the end of the string depending on which index is omitted
Can you write Python code for it?

fruit = 'banana'
print( fruit[:3] )
print( fruit[3:] )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch08-strings.ipynb".
The first task is:
The second approach can be simpler, but you don't know the index of the value
Loops can also be used to build strings
Can you write Python code for it?

prefixes = 'JKLMNOPQ'
suffix = 'ack'

for letter in prefixes:
    print( letter + suffix )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch08-strings.ipynb".
The first task is:
Write a function that prompts a user to enter their first and last names.  The function should then print a computer username consisting of the first letter of their first name and the first seven letters of their last name.
Can you write Python code for it?

def create_username():
    username = ''
    # YOUR CODE HERE
    print( username )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch09-word-play.ipynb".
The first task is:
The book says fin is an acceptable name, but I opt for a more descriptive name
There are a number of methods for reading and writing files, including:
read( size ) Reads size bytes of data.  If size is omitted or negative, the entire file is readn and return.  Returns an empty string if the end of the file (EOF) is reached.
readline() Reads a single line from the file
write( a_string ) Writes a string to the file
close() Closes the file object and frees up any system resources
You can also use a for loop to read each line of the file
Can you write Python code for it?

for line in input_file:
    word = line.strip()
    print( word )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch09-word-play.ipynb".
The first task is:
The for loop traverses each letter in the word looking for an e
In fact, if you paid very good attention, you will see that the uses_all and uses_only functions in the book are the same
In computer science, we frequently encounter problems that are essentially the same as ones we have already solved, but are just worded differently
When you find one (called problem recognition), you can apply a previously developed solution
How much work you need to do to apply it is dependent on how general your solution is
This is an essential skill for problem-solving in general and not just programming

Looping with indices

The previous code didn't have a need to use the indices of characters so the simple for ... in loop was used
There are a number of ways to traverse a string while maintaining a current index
Use a for loop across the range of the length of the string
Use recursion
Use a while loop and maintain the current index
I recommend the first option as it lets the for loop maintain the index
Recursion is more complex than necessary for this problem
A while loop can be used, but isn't as well suited since we know exactly how many times we need to run through the loop
Examples of all three options are below
Can you write Python code for it?

fruit = 'banana'

# For loop
for i in range( len( fruit ) ):
    print( 'For: [',i,']=[',fruit[i],']' )

# Recursive function
def recurse_through_string( word, i ):
    print( 'Recursive: [',i,']=[',fruit[i],']' )
    if( (i + 1) < len( word ) ):
        recurse_through_string( word, i + 1 )

recurse_through_string( fruit, 0 )

# While loop
i = 0
while( i < len( fruit ) ):
    print( 'While: [',i,']=[',fruit[i],']' )
    i = i + 1


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
A for loop over an empty list never executes the body of the loop
If you have a nested list, the nested list still counts as a single element
Can you write Python code for it?

nested_list = ['spam', 1, [ 'Brie', 'Roquefort' , 'Pol le Veq'], [1, 2, 3] ]
len( nested_list )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
A function that combines a sequence of elements into a single value is sometimes called a reduce function
Sometimes you want to build a new list from an existing list
Can you write Python code for it?

def capitalize_all( a_list ):
    result = []
    for a_string in a_list:
        result.append( a_string.capitalize() )
    return result

words = [ 'life', 'the', 'universe', 'and', 'everything' ]
capitalized_words = capitalize_all( words )
print( capitalized_words )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
As you can see, lists can be nested
A list containing no elements is an empty list
An empty list is created using empty brackets []
You can assign a list to a variable
Can you write Python code for it?

cheeses = ['Chedder', 'Pepper Jack', 'Queso Fresca']
grades = [99, 84, 91]
empty = []


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
Be careful because this can cause problems if you aren’t paying attention
For immutable objects (like strings), it isn’t a problem

List arguments

When you pass a list as an argument to a function, it gets a reference to the list
What does this mean?
If the list modifies the list, it is the same list as in the calling function
Can you write Python code for it?

def delete_head( a_list )
    del a_list[0]

letters = [ 'a', 'b', 'c' ]
delete_head( letters )
print( letters )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
By default, the delimiter is a space
You can change that by passing it as an argument
Can you write Python code for it?

a_string = 'spam-spam-spam'
delimiter = '-'
a_string.split( delimiter )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
However, if you need the index, you need to use the range and len functions
For example, you may want to update the values in the list as you traverse it
Can you write Python code for it?

numbers = [1, 2, 3]

for i in range( len( numbers ) ):
    numbers[i] = numbers[i] * 2

print( numbers )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
In the first example, Python only create one string object
In the second, Python created two lists that are equivalent, but not identical
Why the difference?

Aliasing

A variable doesn’t actually hold an object, it has a reference to it
When two variables refer to the same object, it is called aliasing
If an object is mutable, changes made to one affect the other
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
List operations

The + operator concatenates lists
This means that it links them together into a single list
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
List slices

The slice operator also works on lists:
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
Lists and strings

A list of characters is not the same as a string
To convert a string to a list, use the list operator
Can you write Python code for it?

a_string = 'spam'
a_list = list( a_string )
print( a_list )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
Note that like strings, indices start at 0
In fact, all the things you can use for indices and use indices for in strings, you can do in lists
This includes the in operator
One way to think of the relationship between indices and elements is that it is a mapping
Each index maps to one of the elements
Much like a word maps to a definition in a dictionary
TODO State diagram for some lists

Traversing a list

The most common way to traverse a list is with a for loop
It is the same approach we used with strings
Can you write Python code for it?

for cheese in cheeses:
    print( cheese )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
Note that list3 is unmodified
sort arranges the elements of a list from low to high
Can you write Python code for it?

list4 = [ 'd', 'c', 'e', 'a', 'b' ]
list4.sort()
print( list4 )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
Objects and values

To check whether two variables refer to the same object, use the is operator
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
See the stack diagram on pg. 97 of the textbook
Make sure you know if the function you are using modifies the list argument or returns a new list
Can you write Python code for it?

list1 = [ 1, 2 ]
list2 = list1.append( 3 )
print( list1 )
print( list2 )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
Similarly, the * operator repeats a list a specified number of times
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
Since it is an operator, don’t use list as a variable name
Don’t use l either since it looks like a 1
Use a_list or something more descriptive detailing a value in your algorithm
If you want to split a string into words and not letters, use split
Can you write Python code for it?

a_string = 'pining for the fjords'
a_list = a_string.split()
print( a_list )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
The elements of a list don't have to be the same data type in Python (in other languages they do)
However, if you want a sequence with different data types there are better alternatives
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
This is sometimes called a map function since it "maps" a function onto each of the elements in a sequence
Note that a map is more commonly used to refer to a specific data structure (e.g., a dictionary)
Another common operation is to filter elements from one list to another
Can you write Python code for it?

def only_upper( a_list ):
    result = []
    for a_string in a_list:
        if( a_string.isupper() ):
            result.append( a_string )
    return result

words = [ 'LIFE', 'The', 'uNiverse', 'AND', 'everything' ]
some_words = only_upper( words )
print( some_words )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch10-lists.ipynb".
The first task is:
join is the inverse of split
Since it is a string method, you have to invoke it on a string
In this case, invoke it on the delimiter
Can you write Python code for it?

a_list = [ 'pining', 'for', 'the', 'fjords' ]
delimiter = ' '
delimiter.join( a_list )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch11-dictionaries.ipynb".
The first task is:
Note that order isn't preserved in a dictionary (unlike a list)
The values can be retrieved using the same notation
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch11-dictionaries.ipynb".
The first task is:
Reverse lookup

Dictionaries are designed to find a value given a key
This is called a lookup
What if you want to do the reverse? (Note: this is referred to as a reverse lookup)
For example, what if you want to lookup a word by its definition?
Unfortunately there is no simple function to do that
Why might that be?
The book presents a simple function that implements this functionality
Can you write Python code for it?

def reverse_lookup( a_dict, value ):
    for key in a_dict:
        if a_dict[key] == value:
            return key
    raise ValueError


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch11-dictionaries.ipynb".
The first task is:
Sometimes you may want to invert a dictionary
This means that you turn the keys into values and values into keys
However, remember that keys are unique, but values aren't necessarily unique
This means that the original dictionary can have multiple keys with the same value
The inversion process is therefore more complex than simply switching things around.
Can you write Python code for it?

def invert_dict( a_dict ):
    inverted_dict = dict()
    for key in a_dict:
        value = a_dict[key]
        if value not in inverted_dict:
            inverted_dict[value] = [ key ]
        else:
            inverted_dict[value].append( key )
    return inverted_dict


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch11-dictionaries.ipynb".
The first task is:
The Python documentation for dictionaries is quite extensive: https://docs.python.org/3/tutorial/datastructures.html#dictionaries
Dictionaries have a number of methods available
Can you write Python code for it?

# Get the number of key-value pairs
print( len( birthdays ) )

# Get the values in the dictionary
print( birthdays.values() )

# Get the keys in the dictionary
print( birthdays.keys() )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch11-dictionaries.ipynb".
The first task is:
To add an item to the dictionary, use square brackets like a list
Can you write Python code for it?

birthdays['0704'] = 'Steve'
birthdays['0529'] = 'Tony'
print( birthdays )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch11-dictionaries.ipynb".
The first task is:
Why is this approach inefficient?
Note the raise keyword
It causes an exception (specifically a ValueError) if the value isn't in the dictionary
The raise statement also takes an optional argument that is a detailed error message

Dictionaries and lists

Lists can appear as values in a dictionary
For example, people often have the same birthday
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch12-tuples.ipynb".
The first task is:
Essentially, it returns an iterator over a list of tuples
If the sequences aren't the same length, the result has the length of the shorter one
Can you write Python code for it?

for element in zip( 'Peter', 'Tony' ):
    print( element )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch12-tuples.ipynb".
The first task is:
If you want to create a tuple with a single value, add a comma (,) after the value, but don’t add parenthesis
You can also use the built in function tuple
Can you write Python code for it?

a_tuple = tuple()
print( a_tuple )
a_tuple = tuple( 'lupins' )
print( a_tuple )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch12-tuples.ipynb".
The first task is:
Lists and tuples

The zip function takes two or more sequences and "zips" them into a list of tuples
Can you write Python code for it?

a_string = 'abc'
a_list = [ 0, 1, 2 ]
for element in zip( a_string, a_list ):
    print( element )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch12-tuples.ipynb".
The first task is:
Note that min and max are built-in functions

Variable-length argument tuples

All the functions we have built and used required a specific number of arguments
You can use tuples to build functions that accept a variable number of arguments
Prepend the argument’s variable name with an * to do this
It is referred to as the gather operator
Can you write Python code for it?

def printall( *args ):
    print( args )

printall( 1 , 2.0 , '3' )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch12-tuples.ipynb".
The first task is:
Python allows you to do it using a tuple assignment
Can you write Python code for it?


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch12-tuples.ipynb".
The first task is:
The complement is the scatter operator
It allows you to pass a sequence of values as individual arguments to the function
Can you write Python code for it?

a_tuple = ( 7, 3 )
# divmod( a_tuple ) # Uncomment to see error
divmod( *a_tuple )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch12-tuples.ipynb".
The first task is:
Tuples as return values

A function can only return one value
However, if we make that value a tuple, we can effectively return multiple values
For example, the divmod function takes two (2) arguments and retunrs a tuple of two (2) values, the quotient and remainder
Can you write Python code for it?

quotient, remainder = divmod( 7, 3 )
print( quotient )
print( remainder )


I am working on the file "CSNE2444-Intro-to-CS-I/jupyter-notebooks/ch12-tuples.ipynb".
The first task is:
You can combine zip, for and tuple assignment to traverse two (or more) sequences at the same time
Can you write Python code for it?

def has_match( tuple1, tuple2 ):
    result = False
    for x, y in zip( tuple1, tuple2 ):
        if( x == y ):
            result = True
    return result


I am working on the file "CSW/.ipynb_checkpoints/cinergi_csw-checkpoint.ipynb".
The first task is:
Add ServiceType constraint.
Can you write Python code for it?

val = 'opendap'
filter2 = fes.PropertyIsLike(propertyname='apiso:ServiceType',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?')
filter_list = [fes.And([filter1, filter2, bbox_filter])]
csw.getrecords2(constraints=filter_list, maxrecords=1000)

print(len(csw.records.keys()))
for rec in list(csw.records.keys()):
    print('title:'+csw.records[rec].title)
    print('identifier:'+csw.records[rec].identifier)
    print('modified:'+csw.records[rec].modified)
    print(' ')


I am working on the file "CSW/.ipynb_checkpoints/cinergi_csw-checkpoint.ipynb".
The first task is:
Print out the references.   Expecting to see the opendap link here:
Can you write Python code for it?


I am working on the file "CSW/CSW_ISO_Queryables-IOOS.ipynb".
The first task is:
Also on the GEO-IDE Wiki we find the list of UUIDs for each region/provider, which we turn into a dictionary here:
Can you write Python code for it?

regionids = {'AOOS':	'{1E96581F-6B73-45AD-9F9F-2CC3FED76EE6}',
'CENCOOS':	'{BE483F24-52E7-4DDE-909F-EE8D4FF118EA}',
'CARICOOS':	'{0C4CA8A6-5967-4590-BFE0-B8A21CD8BB01}',
'GCOOS':	'{E77E250D-2D65-463C-B201-535775D222C9}',
'GLOS':	'{E4A9E4F4-78A4-4BA0-B653-F548D74F68FA}',
'MARACOOS':	'{A26F8553-798B-4B1C-8755-1031D752F7C2}',
'NANOOS':	'{C6F4754B-30DC-459E-883A-2AC79DA977AB}',
'NAVY':	'{FB160233-7C3B-4841-AD4B-EB5AD843E743}',
'NDBC':	'{B3F50F38-3DE4-4EC9-ABF8-955887829FCC}',
'NERACOOS':	'{E13C88D9-3FF3-4232-A379-84B6A1D7083E}',
'NOS/CO-OPS':	'{2F58127E-A139-4A45-83F2-9695FB704306}',
'PacIOOS':	'{78C0463E-2FCE-4AB2-A9C9-6A34BF261F52}',
'SCCOOS':	'{20A3408F-9EC4-4B36-8E10-BBCDB1E81BDF}',
'SECOORA':	'{E796C954-B248-4118-896C-42E6FAA6EDE9}',
'USACE':	'{4C080A33-F3C3-4F27-AF16-F85BF3095C41}',
'USGS/CMGP': '{275DFB94-E58A-4157-8C31-C72F372E72E}'}

[op.name for op in csw.operations]

def dateRange(start_date='1900-01-01',stop_date='2100-01-01',constraint='overlaps'):
    if constraint == 'overlaps':
        start = fes.PropertyIsLessThanOrEqualTo(propertyname='startDate', literal=stop_date)
        stop = fes.PropertyIsGreaterThanOrEqualTo(propertyname='endDate', literal=start_date)
    elif constraint == 'within':
        start = fes.PropertyIsGreaterThanOrEqualTo(propertyname='startDate', literal=start_date)
        stop = fes.PropertyIsLessThanOrEqualTo(propertyname='endDate', literal=stop_date)
    return start,stop

# get specific ServiceType URL from records
def service_urls(records,service_string='urn:x-esri:specification:ServiceType:odp:url'):
    urls=[]
    for key,rec in records.iteritems():
        #create a generator object, and iterate through it until the match is found
        #if not found, gets the default value (here "none")
        url = next((d['url'] for d in rec.references if d['scheme'] == service_string), None)
        if url is not None:
            urls.append(url)
    return urls

# Perform the CSW query, using Kyle's cool new filters on ISO queryables
# find all datasets in a bounding box...


I am working on the file "CSW/CSW_ISO_Queryables-IOOS.ipynb".
The first task is:
On the GeoIDE Wiki they give some example CSW examples to illustrate the range possibilities.  Here's one where to search for PACIOOS WMS services:
Can you write Python code for it?


I am working on the file "CSW/CSW_ServiceType_query.ipynb".
The first task is:
Query for all COAWST datasets
Can you write Python code for it?

val = 'COAWST'
filter1 = fes.PropertyIsLike(propertyname='apiso:AnyText',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?')
filter_list = [ filter1 ]

csw.getrecords2(constraints=filter_list,maxrecords=100,esn='full')
print len(csw.records.keys())
for rec in list(csw.records.keys()):
    print csw.records[rec].title

choice=np.random.choice(list(csw.records.keys()))
print(csw.records[choice].title)
csw.records[choice].references


I am working on the file "CSW/CSW_ServiceType_query.ipynb".
The first task is:
Query for all COAWST datsets that also contain WMS endpoints
Since all COAWST datasets contain WMS endpoints, this should return the same number of dataset records
Can you write Python code for it?

val = 'COAWST'
filter1 = fes.PropertyIsLike(propertyname='apiso:AnyText',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?',matchCase=True)
val = 'WMS'
filter2 = fes.PropertyIsLike(propertyname='apiso:ServiceType',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?',matchCase=False)
filter_list = [ [filter1, filter2] ]
csw.getrecords2(constraints=filter_list, maxrecords=1000)
print(len(csw.records.keys()))

for rec in list(csw.records.keys()):
    print('title:'+csw.records[rec].title)
    print('identifier:'+csw.records[rec].identifier)
    print('modified:'+csw.records[rec].modified)

val = 'wms'
filter2 = fes.PropertyIsLike(propertyname='apiso:ServiceType',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?',matchCase=True)
filter_list = [  filter2 ]
csw.getrecords2(constraints=filter_list, maxrecords=1000)
print(len(csw.records.keys()))


I am working on the file "CSW/CSW_test-NGDC.ipynb".
The first task is:
Define a function that will return the endpoint for a specified service type
Can you write Python code for it?

# get specific ServiceType URL from records
def service_urls(records,service_string='urn:x-esri:specification:ServiceType:odp:url'):
    urls=[]
    for key,rec in records.iteritems():
        #create a generator object, and iterate through it until the match is found
        #if not found, gets the default value (here "none")
        url = next((d['url'] for d in rec.references if d['scheme'] == service_string), None)
        if url is not None:
            urls.append(url)
    return urls


I am working on the file "CSW/CSW_test-NGDC.ipynb".
The first task is:
In the next cell we take a look at the references for a random record.  The ServiceType for the OPeNDAP Data URL below is ServiceType:odp:url, which seems correct!
Can you write Python code for it?

choice=random.choice(list(csw.records.keys()))
print choice
csw.records[choice].references


I am working on the file "CSW/CSW_test-NGDC.ipynb".
The first task is:
In the next cell we try using a vertical extent range, which doesn't seem to be working, as we are looking for z values between [-5000, 0] and we see in the ISO record for 'NS10agg' here:

http://oos.soest.hawaii.edu/thredds/iso/hioos/nss/ns10agg?catalog=http%3A%2F%2Foos.soest.hawaii.edu%2Fthredds%2Fidd%2Fnss_hioos.html&dataset=NS10agg

that the vertical extent is from [-2, -2]
Can you write Python code for it?


I am working on the file "CSW/CSW_test-NGDC.ipynb".
The first task is:
Now print out some titles
Can you write Python code for it?

for rec,item in csw.records.iteritems():
    print item.title


I am working on the file "CSW/CSW_test-NGDC.ipynb".
The first task is:
Print out all the OPeNDAP Data URL endpoints
Can you write Python code for it?

dap_urls = service_urls(csw.records,service_string='urn:x-esri:specification:ServiceType:odp:url')
print "\n".join(dap_urls)

def standard_names(ncv):
    '''
    get dictionary of variables with standard_names
    '''
    d={}
    for k,v in ncv.iteritems():
        try:
            standard_name=v.getncattr('standard_name')
            try:
                d[standard_name]=[d[standard_name],[k]]
            except:
                d[standard_name]=[k]
        except:
            pass
    return d

for url in dap_urls:
    ncv = netCDF4.Dataset(url).variables
    print ncv.keys()
    lat = ncv['lon'][:]
    lon = ncv['lat'][:]
    tvar = ncv['time']
    istart = netCDF4.date2index(dt.datetime.strptime(start_date,'%Y-%m-%d'),tvar,select='nearest')
    istop = netCDF4.date2index(dt.datetime.strptime(stop_date,'%Y-%m-%d'),tvar,select='nearest')
    if istart != istop:
        dtime = netCDF4.num2date(tvar[istart:istop],tvar.units)
        # make a dictionary containing all data from variables that matched the standard_name
        # find list of variables for each standard_name
        d = standard_names(ncv)
        # find all the variables matching standard_name=std_name
        print d[std_name]
        # read all the data into a dictionary
        data_dict={}
        lev=0
        for v in d[std_name]:
            data_dict[v]=ncv[v][istart:istop].flatten()
        # Create Pandas data frame, with time index
        ts = pd.DataFrame.from_dict(data_dict)
        ts.index=dtime
        ts.plot(figsize=(12,4));
        title(std_name)


I am working on the file "CSW/CSW_test-NGDC.ipynb".
The first task is:
These are tags NGDC would like tested:
apiso:ResponsiblePartyName
apiso:Keywords
apiso:KeywordType
apiso:VertExtent_min
apiso:VertExtent_max
apiso:ServiceType
apiso:ServiceTypeVersion
apiso:OperatesOn
resource.url
website.url
Can you write Python code for it?

#from IPython.core.display import HTML
#HTML('<iframe src=http://www.ngdc.noaa.gov/geoportal/ width=950 height=400></iframe>')

# connect to CSW, explore it's properties

endpoint = 'http://www.ngdc.noaa.gov/geoportal/csw' # NGDC Geoportal

#endpoint = 'http://www.nodc.noaa.gov/geoportal/csw'   # NODC Geoportal: granule level
#endpoint = 'http://data.nodc.noaa.gov/geoportal/csw'  # NODC Geoportal: collection level
#endpoint = 'http://geodiscover.cgdi.ca/wes/serviceManagerCSW/csw'  # NRCAN CUSTOM
#endpoint = 'http://geoport.whoi.edu/gi-cat/services/cswiso' # USGS Woods Hole GI_CAT
#endpoint = 'http://cida.usgs.gov/gdp/geonetwork/srv/en/csw' # USGS CIDA Geonetwork
#endpoint = 'http://cmgds.marine.usgs.gov/geonetwork/srv/en/csw' # USGS Coastal and Marine Program
#endpoint = 'http://geoport.whoi.edu/geoportal/csw' # USGS Woods Hole Geoportal
#endpoint = 'http://geo.gov.ckan.org/csw'  # CKAN testing site for new Data.gov
#endpoint = 'https://edg.epa.gov/metadata/csw'  # EPA
#endpoint = 'http://cwic.csiss.gmu.edu/cwicv1/discovery'  # CWIC

csw = CatalogueServiceWeb(endpoint,timeout=60)
csw.version

[op.name for op in csw.operations]

# hopefully something like this will be implemented in fes soon
def dateRange(start_date='1900-01-01',stop_date='2100-01-01',constraint='overlaps'):
    if constraint == 'overlaps':
        start = fes.PropertyIsLessThanOrEqualTo(propertyname='apiso:TempExtent_begin', literal=stop_date)
        stop = fes.PropertyIsGreaterThanOrEqualTo(propertyname='apiso:TempExtent_end', literal=start_date)
    elif constraint == 'within':
        start = fes.PropertyIsGreaterThanOrEqualTo(propertyname='apiso:TempExtent_begin', literal=start_date)
        stop = fes.PropertyIsLessThanOrEqualTo(propertyname='apiso:TempExtent_end', literal=stop_date)
    return start,stop

# hopefully something like this will be implemented in fes soon
def zRange(min_z='-5000', max_z='0', constraint='overlaps'):
    if constraint == 'overlaps':
        zmin =...


I am working on the file "CSW/cinergi_csw.ipynb".
The first task is:
Add bounding box constraint.
To specify lon,lat order for bbox (which we want to do so that we can use the same bbox with either geoportal server or pycsw requests), we need to request the bounding box specifying the CRS84 coordinate reference system.   The CRS84 option is available in pycsw 1.1.10+. The ability to specify the crs in the bounding box request is available in owslib 0.8.12+.  For more info on the bounding box problem and how it was solved, see this pycsw issue, this geoportal server issue, and this owslib issue
Can you write Python code for it?

#bbox = [-87.40, 34.25, -63.70, 66.70]    # [lon_min, lat_min, lon_max, lat_max]      maine
bbox = [-162., 19.0, -154., 23.0]   # Hawaii
bbox_filter = fes.BBox(bbox,crs='urn:ogc:def:crs:OGC:1.3:CRS84')
filter_list = [fes.And([filter1, bbox_filter])]
csw.getrecords2(constraints=filter_list, maxrecords=1000)

print(len(csw.records.keys()))
for rec in list(csw.records.keys()):
    print('title:'+csw.records[rec].title)
    print('identifier:'+csw.records[rec].identifier)
    print('modified:'+csw.records[rec].modified)
    print(' ')


I am working on the file "CSW/cinergi_csw.ipynb".
The first task is:
Try a simple query first using just apiso:AnyText
Can you write Python code for it?

val = 'roms'
#val = 'modflow'
filter1 = fes.PropertyIsLike(propertyname='apiso:AnyText',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?')
filter_list = [ filter1 ]

csw.getrecords2(constraints=filter_list,maxrecords=100,esn='full')
print len(csw.records.keys())
for rec in list(csw.records.keys()):
    print csw.records[rec].title


I am working on the file "CSW/csw_post_request-Copy1.ipynb".
The first task is:
BBOX query on NGDC Geoportal Server CSW
Can you write Python code for it?

#endpoint = 'http://catalog.data.gov/csw-all/csw'
endpoint = 'http://geoport.whoi.edu/csw'

input='''
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" version="2.0.2" service="CSW"
    resultType="results" startPosition="1" maxRecords="100">
    <csw:Query typeNames="csw:Record" xmlns:ogc="http://www.opengis.net/ogc"
        xmlns:gml="http://www.opengis.net/gml">
        <csw:ElementSetName>full</csw:ElementSetName>
        <csw:Constraint version="1.1.0">
            <ogc:Filter>
                <ogc:PropertyIsLike wildCard="*" singleChar="#" escapeChar="!">
                    <ogc:PropertyName>apiso:ServiceType</ogc:PropertyName>
                    <ogc:Literal>*WMS*</ogc:Literal>
                </ogc:PropertyIsLike>
            </ogc:Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>
''';

print input

xml_string=requests.post(endpoint, data=input, headers=headers).text
print xml_string[:2000]

input = '''
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" version="2.0.2" service="CSW" resultType="results" startPosition="31" maxRecords="100"><csw:Query typeNames="csw:Record" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gml="http://www.opengis.net/gml"><csw:ElementSetName>full</csw:ElementSetName><csw:Constraint version="1.1.0"><ogc:Filter><ogc:PropertyIsLike wildCard="*" singleChar="#" escapeChar="!"><ogc:PropertyName>apiso:ServiceType</ogc:PropertyName><ogc:Literal>*wms*</ogc:Literal></ogc:PropertyIsLike></ogc:Filter></csw:Constraint></csw:Query></csw:GetRecords>
'''

print(input)

xml_string=requests.post(endpoint, data=input, headers=headers).text
print xml_string[:2000]

endpoint='http://oos.soest.hawaii.edu/pacioos/ogc/csw.py'

input='''
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ogc="http://www.opengis.net/ogc"
    xmlns:gml="http://www.opengis.net/gml" outputSchema="http://www.opengis.net/cat/csw/2.0.2"
    outputFormat="application/xml" version="2.0.2"...


I am working on the file "CSW/csw_post_request-Copy1.ipynb".
The first task is:
Query COMT pycsw
Try (lat,lon) order of bounding box with srsName=EPSG:4326
Can you write Python code for it?

input='''
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ogc="http://www.opengis.net/ogc"
    xmlns:gml="http://www.opengis.net/gml" outputSchema="http://www.opengis.net/cat/csw/2.0.2"
    outputFormat="application/xml" version="2.0.2" service="CSW" resultType="results"
    maxRecords="1000"
    xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
    <csw:Query typeNames="csw:Record">
        <csw:ElementSetName>full</csw:ElementSetName>
        <csw:Constraint version="1.1.0">
            <ogc:Filter>
                <ogc:And>
                    <ogc:BBOX>
                        <ogc:PropertyName>ows:BoundingBox</ogc:PropertyName>
                        <gml:Envelope srsName="urn:x-ogc:def:crs:EPSG:6.11:4326">
                            <gml:lowerCorner> 27 -100</gml:lowerCorner>
                            <gml:upperCorner> 30 -97</gml:upperCorner>
                        </gml:Envelope>
                    </ogc:BBOX>
                    <ogc:PropertyIsLessThanOrEqualTo>
                        <ogc:PropertyName>apiso:TempExtent_begin</ogc:PropertyName>
                        <ogc:Literal>2008-12-01T16:43:00Z</ogc:Literal>
                    </ogc:PropertyIsLessThanOrEqualTo>
                    <ogc:PropertyIsGreaterThanOrEqualTo>
                        <ogc:PropertyName>apiso:TempExtent_end</ogc:PropertyName>
                        <ogc:Literal>2008-06-01T16:43:00Z</ogc:Literal>
                    </ogc:PropertyIsGreaterThanOrEqualTo>
                    <ogc:PropertyIsLike wildCard="*" singleChar="?" escapeChar="\\">
                        <ogc:PropertyName>apiso:AnyText</ogc:PropertyName>
                        <ogc:Literal>*FVCOM*</ogc:Literal>
                    </ogc:PropertyIsLike>
                </ogc:And>
            </ogc:Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>
''';

xml_string=requests.post(endpoint, data=input,...


I am working on the file "CSW/csw_post_request-Copy1.ipynb".
The first task is:
Try (lon,lat) order of bounding box with srsName=CRS84
Can you write Python code for it?

input='''
<csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ogc="http://www.opengis.net/ogc"
    xmlns:gml="http://www.opengis.net/gml" outputSchema="http://www.opengis.net/cat/csw/2.0.2"
    outputFormat="application/xml" version="2.0.2" service="CSW" resultType="results"
    maxRecords="1000"
    xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
    <csw:Query typeNames="csw:Record">
        <csw:ElementSetName>full</csw:ElementSetName>
        <csw:Constraint version="1.1.0">
            <ogc:Filter>
                <ogc:And>
                    <ogc:BBOX>
                        <ogc:PropertyName>ows:BoundingBox</ogc:PropertyName>
                        <gml:Envelope srsName="urn:ogc:def:crs:OGC:1.3:CRS84">
                            <gml:lowerCorner>-100 27</gml:lowerCorner>
                            <gml:upperCorner> -97 30</gml:upperCorner>
                        </gml:Envelope>
                    </ogc:BBOX>
                    <ogc:PropertyIsLessThanOrEqualTo>
                        <ogc:PropertyName>apiso:TempExtent_begin</ogc:PropertyName>
                        <ogc:Literal>2008-12-01T16:43:00Z</ogc:Literal>
                    </ogc:PropertyIsLessThanOrEqualTo>
                    <ogc:PropertyIsGreaterThanOrEqualTo>
                        <ogc:PropertyName>apiso:TempExtent_end</ogc:PropertyName>
                        <ogc:Literal>2008-06-01T16:43:00Z</ogc:Literal>
                    </ogc:PropertyIsGreaterThanOrEqualTo>
                    <ogc:PropertyIsLike wildCard="*" singleChar="?" escapeChar="\\">
                        <ogc:PropertyName>apiso:AnyText</ogc:PropertyName>
                        <ogc:Literal>*FVCOM*</ogc:Literal>
                    </ogc:PropertyIsLike>
                </ogc:And>
            </ogc:Filter>
        </csw:Constraint>
    </csw:Query>
</csw:GetRecords>
''';

xml_string=requests.post(endpoint, data=input,...


I am working on the file "CSW/csw_roms_hawaii.ipynb".
The first task is:
Finally, it is possible to download the data directly into a data pandas data frame and plot it.
Can you write Python code for it?

import io
import requests
import pandas as pd

r = requests.get(url)

df = pd.read_csv(io.StringIO(r.content.decode('utf-8')),
                 index_col='date_time', parse_dates=True)

%matplotlib inline
import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(11, 2.75))
ax = df['sea_water_temperature (C)'].plot(ax=ax)
ax.set_xlabel('')
ax.set_ylabel(r'Sea water temperature ($^\circ$C)')
ax.set_title(value.title)


I am working on the file "CSW/csw_roms_hawaii.ipynb".
The first task is:
Let's start by creating the search filters.
The filter used here constraints the search on a certain geographical region (bounding box), a time span (last week), and some CF variable standard names that represent sea surface temperature.
Can you write Python code for it?

#Bounding Box
min_lon, min_lat = -166.0, 19.0
max_lon, max_lat = -157.0, 23.0

bbox = [min_lon, min_lat, max_lon, max_lat]
crs = 'urn:ogc:def:crs:OGC:1.3:CRS84'

# Temporal range: Last week.
now = datetime.utcnow()
start, stop = now - timedelta(days=(7)), now

# Temporal range:  Specified time range
start = dateutil.parser.parse('2017-03-01T00:00:00Z')
stop  = dateutil.parser.parse('2017-04-01T00:00:00Z')

# Find any of these Ocean Model Names
model_names = ['ROMS', 'FVCOM', 'SELFE', 'ADCIRC', 'Delft3D', 'DelftFM', 'HyCOM', 'NCOM']

# ServiceType
service_type = 'WMS'


I am working on the file "CSW/csw_roms_hawaii.ipynb".
The first task is:
The easiest way to get more information is to explorer the individual records.
Here is the abstract and subjects from the station in Astoria, OR.
Can you write Python code for it?

k

import textwrap

print('\n'.join(textwrap.wrap(value.abstract)))

print('\n'.join(value.subjects))


I am working on the file "CSW/csw_roms_hawaii.ipynb".
The first task is:
There are many direct links to Comma Separated Value (CSV) and
eXtensible Markup Language (XML) responses to the various variables available in that station.
In addition to those links, there are three very interesting links for more information: 1.) the QC document, 2.) the station photo, 3.) the station home page.
For a detailed description of what those geolink results mean check the lookup table.

The original search was focused on sea water temperature,
so there is the need to extract only the endpoint for that variable.
PS: see also the pyoos example for fetching data from CO-OPS.
Can you write Python code for it?

start, stop

for ref in value.references:
    url = ref['url']
    if 'csv' in url and 'sea' in url and 'temperature' in url:
        print(msg(geolink=sniff_link(url), **ref))
        break


I am working on the file "CSW/csw_roms_hawaii.ipynb".
The first task is:
With these 3 elements it is possible to assemble a OGC Filter Encoding (FE) using the owslib.fes* module.
* OWSLib is a Python package for client programming with Open Geospatial Consortium (OGC) web service (hence OWS) interface standards, and their related content models.
Can you write Python code for it?

from owslib import fes
from ioos_tools.ioos import fes_date_filter

kw = dict(wildCard='*', escapeChar='\\',
          singleChar='?', propertyname='apiso:AnyText')

or_filt = fes.Or([fes.PropertyIsLike(literal=('*%s*' % val), **kw)
                  for val in model_names])

kw = dict(wildCard='*', escapeChar='\\',
          singleChar='?', propertyname='apiso:ServiceType')

serviceType = fes.PropertyIsLike(literal=('*%s*' % service_type), **kw)

begin, end = fes_date_filter(start, stop)
bbox_crs = fes.BBox(bbox, crs=crs)

filter_list = [
    fes.And(
        [
            bbox_crs,  # bounding box
            begin, end,  # start and end date
            or_filt,  # or conditions (CF variable names)
            serviceType  # search only for datasets that have WMS services
        ]
    )
]

from owslib.csw import CatalogueServiceWeb

endpoint = 'https://data.ioos.us/csw'

csw = CatalogueServiceWeb(endpoint, timeout=60)


I am working on the file "CSW/csw_roms_hawaii.ipynb".
The first task is:
look for all modeling records
Can you write Python code for it?

filter_list = [ or_filt ]

get_csw_records(csw, filter_list, pagesize=10, maxrecords=1000)

records = '\n'.join(csw.records.keys())
print('Found {} records.\n'.format(len(csw.records.keys())))
k=0
for key, value in list(csw.records.items()):
    print('[{}]\n{}\n'.format(value.title, key))
    [print(d['scheme']) for d in value.references]
    print('\n')
    if any('None' in d['scheme'] for d in value.references):
        k = k+1

[print(d['scheme']) for d in value.references]


I am working on the file "CSW/csw_unidata.ipynb".
The first task is:
Let's start by creating the search filters.
The filter used here constraints the search on a certain geographical region (bounding box), a time span (last week), and some CF variable standard names that represent sea surface temperature.
Can you write Python code for it?

from datetime import datetime, timedelta
import dateutil.parser

service_type = 'WMS'

min_lon, min_lat = -90.0, 30.0
max_lon, max_lat = -80.0, 40.0

bbox = [min_lon, min_lat, max_lon, max_lat]
crs = 'urn:ogc:def:crs:OGC:1.3:CRS84'

# Temporal range: Last week.
now = datetime.utcnow()
start, stop = now - timedelta(days=(7)), now

start = dateutil.parser.parse('2017-03-01T00:00:00Z')
stop  = dateutil.parser.parse('2017-04-01T00:00:00Z')

# Ocean Model Names
model_names = ['NAM', 'GFS']