I am working on the file "LSTM_Timeseries_predict.ipynb".
The first task is:
Training the network
We are ready to train. 100 epochs should yield acceptable results.
Can you write Python code for it?

# Training parameters
EPOCHS = 500

# train
loss_summary = []
start = time.time()
for epoch in range(0, EPOCHS):
    for x1, y1 in next_batch(X, Y, "train", BATCH_SIZE):
        trainer.train_minibatch({x: x1, y: y1})
    if epoch % (EPOCHS / 20) == 0:
        training_loss = cntk.utils.get_train_loss(trainer)
        loss_summary.append((epoch, training_loss))
        print("epoch: {:3d}/{}, loss: {:.5f}".format(epoch, EPOCHS, training_loss))

loss_summary = np.array(loss_summary)

print("training took {0:.1f} sec".format(time.time() - start))


I am working on the file "LSTM_Timeseries_predict.ipynb".
The first task is:
We check that the errors are roughly the same for train, validation and test sets. We also plot the expected output (Y) and the prediction our model made to shows how well the simple LSTM approach worked.
Can you write Python code for it?

# predict
f, a = plt.subplots(3, 1, figsize = (12, 8))
for j, ds in enumerate(["train", "val", "test"]):
    results = []
    for x1, y1 in next_batch(X, Y, ds, BATCH_SIZE):
        pred = z.eval({x: x1})
        results.extend(pred[:, 0])
    a[j].plot(Y[ds], label = ds + ' raw')
    a[j].plot(results, label = ds + ' predicted')
[i.legend() for i in a];


I am working on the file "LSTM_Timeseries_predict.ipynb".
The first task is:
generate_RNN_data()
The RNN will be trained on sequences of length $N$ of single values (scalars), meaning that each training sample is a $N\times1$ matrix. CNTK requires us to shape our input data as an array with each element being an observation. So, for inputs, $X$, we need to create a tensor or 3-D array with dimensions $[M, N, 1]$ where $M$ is the number of training samples.<br/>
As output we want our network to predict the next value of the sequence, so our target, $Y$, is a $[M, 1]$ array containing the next day's value of the stock price after the sequence presented in $X$.</br/>
We do this by sampling from the sequence of stock prices and creating numpy ndarrays.
Can you write Python code for it?

def generate_RNN_data(x, time_steps=10):
    """
    Generate sequences to feed to rnn
    x: DataFrame, daily close
    time_steps: int, number of days in sequences used to train the RNN
    """

    rnn_x = []
    for i in range(len(x) - (time_steps+1)):
        # Each training sample is a sequence of length time_steps
        xi = x[i: i + time_steps].astype(np.float32).as_matrix()
        # We need to reshape as a column vector as the model expects
        # 1 float per time point
        xi = xi.reshape(-1,1)
        rnn_x.append(xi)
    rnn_x = np.array(rnn_x)

    # The target values are a single float per training sequence
    rnn_y = np.array(x[time_steps+1:].astype(np.float32)).reshape(-1,1)

    return split_data(rnn_x, 0.2, 0.2), split_data(rnn_y, 0.2, 0.2)


I am working on the file "LSTM_Timeseries_predict.ipynb".
The first task is:
split_data()
This function will split the data into training, validation and test sets and return a list with those elements, each containing a ndarray as described above.
Can you write Python code for it?

def split_data(data, val_size=0.1, test_size=0.1):
    """
    splits np.array into training, validation and test
    """
    pos_test = int(len(data) * (1 - test_size))
    pos_val = int(len(data[:pos_test]) * (1 - val_size))

    train, val, test = data[:pos_val], data[pos_val:pos_test], data[pos_test:]

    return {"train": train, "val": val, "test": test}


I am working on the file "LaTeX/.ipynb_checkpoints/DisplayEx01-checkpoint.ipynb".
The first task is:
Basic rich display
Find a Physics related image on the internet and display it in this notebook using the Image object.

Load it using the url argument to Image (don't upload the image to this server).
Make sure the set the embed flag so the image is embedded in the notebook data.
Set the width and height to 600px.
Can you write Python code for it?

Image(url='https://funtaqa.files.wordpress.com/2013/10/physics.jpg',embed=True,width=600,height=600)

assert True # leave this to grade the image display


I am working on the file "LaTeX/.ipynb_checkpoints/DisplayEx01-checkpoint.ipynb".
The first task is:
Use the HTML object to display HTML in the notebook that reproduces the table of Quarks on this page. This will require you to learn about how to create HTML tables and then pass that to the HTML object for display. Don't worry about styling and formatting the table, but you should use LaTeX where appropriate.
Can you write Python code for it?

%%html

Name
Symbol
Antiparticle
Charge $(e)$
Mass $MeV/c^2$

Up
u
$\bar{u}$
$+\frac{2}{3}$
1.5-3.3

Down
d
$\bar{d}$
$-\frac{1}{3}$
3.5-6.0

Charm
c
$\bar{c}$
$+\frac{2}{3}$
1,160–1,340

Strange
s
$\bar{s}$
$-\frac{1}{3}$
70–130

Top
t
$\bar{t}$
$+\frac{2}{3}$
169,100–173,300

Bottom
b
$\bar{b}$
$-\frac{1}{3}$
4,130–4,370

assert True # leave this here to grade the quark table


I am working on the file "LaTeX/.ipynb_checkpoints/InteractEx05-checkpoint.ipynb".
The first task is:
Use interactive to build a user interface for exploing the draw_circle function:

width: a fixed value of 300px
height: a fixed value of 300px
cx/cy: a slider in the range [0,300]
r: a slider in the range [0,50]
fill: a text area in which you can type a color's name

Save the return value of interactive to a variable named w.
Can you write Python code for it?

w = interactive(draw_circle,width=fixed(300),height=fixed(300),cx=(0,300),cy=(0,300),r=(0,50))

c = w.children
assert c[0].min==0 and c[0].max==300
assert c[1].min==0 and c[1].max==300
assert c[2].min==0 and c[2].max==50
assert c[3].value=='red'


I am working on the file "LaTeX/.ipynb_checkpoints/InteractEx05-checkpoint.ipynb".
The first task is:
Use the display function to show the widgets created by interactive:
Can you write Python code for it?

display(w)

assert True # leave this to grade the display of the widget


I am working on the file "LaTeX/.ipynb_checkpoints/InteractEx05-checkpoint.ipynb".
The first task is:
Write a function named draw_circle that draws a circle using SVG. Your function should take the parameters of the circle  as function arguments and have defaults as shown. You will have to write the raw SVG code as a Python string and then use the IPython.display.SVG object and IPython.display.display function.
Can you write Python code for it?

def draw_circle(width=100, height=100, cx=25, cy=25, r=5, fill='red'):
    """Draw an SVG circle.

    Parameters
    ----------
    width : int
        The width of the svg drawing area in px.
    height : int
        The height of the svg drawing area in px.
    cx : int
        The x position of the center of the circle in px.
    cy : int
        The y position of the center of the circle in px.
    r : int
        The radius of the circle in px.
    fill : str
        The fill color of the circle.
    """
    s = '<svg width="%d" height="%d" > <circle cx="%d" cy="%d" r="%d" fill="%s" /> </svg>'  %(width,height,cx,cy,r,fill)
    display(SVG(s))

draw_circle(cx=10, cy=10, r=10, fill='blue')

assert True # leave this to grade the draw_circle function


I am working on the file "LaTeX/.ipynb_checkpoints/ProjectEuler17-checkpoint.ipynb".
The first task is:
Now define a count_letters(n) that returns the number of letters used to write out the words for all of the the numbers 1 to n inclusive.
Can you write Python code for it?

def count_letters(n):
    """Count the number of letters used to write out the words for 1-n inclusive."""
    lst2 = []
    for entry in number_to_words(n):
        count = 0
        for char in entry:
            if char != ' ' and char != '-':
                count = count + 1
        lst2.append(count)
    return lst2


I am working on the file "LaTeX/.ipynb_checkpoints/ProjectEuler17-checkpoint.ipynb".
The first task is:
Now write a set of assert tests for your count_letters function that verifies that it is working as expected.
Can you write Python code for it?

assert count_letters(1) == [3]
assert len(count_letters(342)) == 342
assert count_letters(5) == [3,3,5,4,4]

assert True # use this for grading the count_letters tests.


I am working on the file "LaTeX/.ipynb_checkpoints/ProjectEuler17-checkpoint.ipynb".
The first task is:
Now write a set of assert tests for your number_to_words function that verifies that it is working as expected.
Can you write Python code for it?

assert len(number_to_words(5))==5
assert len(number_to_words(900))==900
assert number_to_words(50)[-1]=='fifty'

assert True # use this for grading the number_to_words tests.


I am working on the file "LaTeX/Display.ipynb".
The first task is:
A few points:

Calling display on an object will send all possible representations to the Notebook.
These representations are stored in the Notebook document.
In general the Notebook will use the richest available representation.

If you want to display a particular representation, there are specific functions for that:
Can you write Python code for it?

from IPython.display import (
    display_pretty, display_html, display_jpeg,
    display_png, display_json, display_latex, display_svg
)


I am working on the file "LaTeX/Display.ipynb".
The first task is:
An image can also be displayed from raw data or a URL.
Can you write Python code for it?


I am working on the file "LaTeX/Display.ipynb".
The first task is:
Audio
IPython makes it easy to work with sounds interactively. The Audio display class allows you to create an audio control that is embedded in the Notebook. The interface is analogous to the interface of the Image display class. All audio formats supported by the browser can be used. Note that no single format is presently supported in all browsers.
Can you write Python code for it?

from IPython.display import Audio
Audio("./scrubjay.mp3")


I am working on the file "LaTeX/Display.ipynb".
The first task is:
HTML
Python objects can declare HTML representations that will be displayed in the Notebook. If you have some HTML you want to display, simply use the HTML class.
Can you write Python code for it?

from IPython.display import HTML

s = """

Header 1
Header 2

row 1, cell 1
row 1, cell 2

row 2, cell 1
row 2, cell 2

"""

h = HTML(s)

display(h)


I am working on the file "LaTeX/Display.ipynb".
The first task is:
Here is a more complicated example that loads d3.js from a CDN, uses the %%html magic to load CSS styles onto the page and then runs ones of the d3.js examples.
Can you write Python code for it?

Javascript(
    """$.getScript('https://cdnjs.cloudflare.com/ajax/libs/d3/3.2.2/d3.v3.min.js')"""
)

%%html
<style type="text/css">

circle {
  fill: rgb(31, 119, 180);
  fill-opacity: .25;
  stroke: rgb(31, 119, 180);
  stroke-width: 1px;
}

.leaf circle {
  fill: #ff7f0e;
  fill-opacity: 1;
}

text {
  font: 10px sans-serif;
}

</style>

%%javascript

// element is the jQuery element we will append to
var e = element.get(0);

var diameter = 600,
    format = d3.format(",d");

var pack = d3.layout.pack()
    .size([diameter - 4, diameter - 4])
    .value(function(d) { return d.size; });

var svg = d3.select(e).append("svg")
    .attr("width", diameter)
    .attr("height", diameter)
  .append("g")
    .attr("transform", "translate(2,2)");

d3.json("./flare.json", function(error, root) {
  var node = svg.datum(root).selectAll(".node")
      .data(pack.nodes)
    .enter().append("g")
      .attr("class", function(d) { return d.children ? "node" : "leaf node"; })
      .attr("transform", function(d) { return "translate(" + d.x + "," + d.y + ")"; });

  node.append("title")
      .text(function(d) { return d.name + (d.children ? "" : ": " + format(d.size)); });

  node.append("circle")
      .attr("r", function(d) { return d.r; });

  node.filter(function(d) { return !d.children; }).append("text")
      .attr("dy", ".3em")
      .style("text-anchor", "middle")
      .text(function(d) { return d.name.substring(0, d.r / 3); });
});

d3.select(self.frameElement).style("height", diameter + "px");


I am working on the file "LaTeX/Display.ipynb".
The first task is:
IPython expands on this idea and allows objects to declare other, rich representations including:

HTML
JSON
PNG
JPEG
SVG
LaTeX

A single object can declare some or all of these representations; all of them are handled by IPython's display system. .
Basic display imports
The display function is a general purpose tool for displaying different representations of objects. Think of it as print for these rich representations.
Can you write Python code for it?


I am working on the file "LaTeX/Display.ipynb".
The first task is:
Images
To work with images (JPEG, PNG) use the Image class.
Can you write Python code for it?

from IPython.display import Image

i = Image(filename='./ipython-image.png')
display(i)


I am working on the file "LaTeX/Display.ipynb".
The first task is:
Links to local files
IPython provides builtin display classes for generating links to local files. Create a link to a single file using the FileLink object:
Can you write Python code for it?

from IPython.display import FileLink, FileLinks
FileLink('../Visualization/Matplotlib.ipynb')


I am working on the file "LaTeX/Display.ipynb".
The first task is:
Overriding the __repr__ method:
Can you write Python code for it?

class Ball(object):
    def __repr__(self):
        return 'TEST'

b = Ball()

print(b)


I am working on the file "LaTeX/Display.ipynb".
The first task is:
Returning an Image object from an expression will automatically display it:
Can you write Python code for it?


I am working on the file "LaTeX/Display.ipynb".
The first task is:
The same thing can be accomplished using the %%javascript cell magic:
Can you write Python code for it?


I am working on the file "LaTeX/Display.ipynb".
The first task is:
Video
More exotic objects can also be displayed, as long as their representation supports the IPython display protocol.  For example, videos hosted externally on YouTube are easy to load:
Can you write Python code for it?

from IPython.display import YouTubeVideo
YouTubeVideo('sjfsUzECqK0')


I am working on the file "LaTeX/Display.ipynb".
The first task is:
You can also use the %%html cell magic to accomplish the same thing.
Can you write Python code for it?

%%html

Header 1
Header 2

row 1, cell 1
row 1, cell 2

row 2, cell 1
row 2, cell 2

%%html


I am working on the file "LaTeX/Display.ipynb".
The first task is:
You can remove the abvove styling by using "Cell"$\rightarrow$"Current Output"$\rightarrow$"Clear" with that cell selected.
JavaScript
The Notebook also enables objects to declare a JavaScript representation. At first, this may seem odd as  output is inherently visual and JavaScript is a programming language. However, this opens the door for rich output that leverages the full power of JavaScript and associated libraries such as d3.js for output.
Can you write Python code for it?


I am working on the file "LaTeX/DisplayEx01.ipynb".
The first task is:
Use the HTML object to display HTML in the notebook that reproduces the table of Quarks on this page. This will require you to learn about how to create HTML tables and then pass that to the HTML object for display. Don't worry about styling and formatting the table, but you should use LaTeX where appropriate.
Can you write Python code for it?


I am working on the file "Laav_Lane_Lines_Detection.ipynb".
The first task is:
Build a Lane Finding Pipeline
Build the pipeline and run your solution on all test_images. Make copies into the test_images_output directory, and you can use the images in your writeup report.
Try tuning the various parameters, especially the low and high Canny thresholds as well as the Hough lines parameters.
Can you write Python code for it?

# TODO: Build your pipeline that will draw lane lines on the test_images
# then save them to the test_images directory.

for im in os.listdir("test_images/"):

    if im=='output':
        continue

    image = mpimg.imread('test_images/' + im)

    #Copy the image to prevent unecessary changes
    image_cp = np.copy(image)

    # Convert image to grey scale
    gray = grayscale(image_cp)

    # Define a kernel size and apply Gaussian smoothing
    kernel_size = 5
    blur_gray = gaussian_blur(gray, kernel_size)

    # Define our parameters for Canny and apply
    low_threshold = 50
    high_threshold = 150
    edges = canny(blur_gray, low_threshold, high_threshold)

    # Define a polygon region of interest
    # Keep in mind the origin (x=0, y=0) is in the upper left in image processing

    # Coordinates of vertices
    left_upper = [int(xsize/2)-50, lane_limit_height]
    right_upper = [int(xsize/2)+50, lane_limit_height]
    right_bottom = [xsize-50, ysize]
    left_bottom = [50, ysize]

    vertices = np.array( [[ left_upper,right_upper,right_bottom,left_bottom]], dtype=np.int32 )

    region_canny_image = region_of_interest(edges, vertices)

    # ````````````````` Test region of interest```````````````````````````````
    # mpimg.imsave('test_images/output/' + 'test_region_area.png', region_image)
    # plt.imshow(region_image)

    # Define the Hough transform parameters
    # Make a blank the same size as our image to draw on
    rho = 2
    theta = np.pi/180
    threshold = 32
    min_line_len = 10
    max_line_gap = 200
    # As you increase above longer the line connecting the left lane lines

    # Run Hough on edge detected image
    line_image = hough_lines(region_canny_image, rho, theta, threshold, min_line_len, max_line_gap)
    # plt.imshow(line_image)

    # Combine the processed image after selecting the required region with the original img
    comb_hough_orig = weighted_img(line_image, image)
    # plt.imshow(comb_hough_orig)

    #saving the resultant image
  ...


I am working on the file "Laav_Lane_Lines_Detection.ipynb".
The first task is:
Improve the draw_lines() function
At this point, if you were successful with making the pipeline and tuning parameters, you probably have the Hough line segments drawn onto the road, but what about identifying the full extent of the lane and marking it clearly as in the example video (P1_example.mp4)?  Think about defining a line to run the full length of the visible lane based on the line segments you identified with the Hough Transform. As mentioned previously, try to average and/or extrapolate the line segments you've detected to map out the full extent of the lane lines. You can see an example of the result you're going for in the video "P1_example.mp4".
Go back and modify your draw_lines function accordingly and try re-running your pipeline. The new output should draw a single, solid line over the left lane line and a single, solid line over the right lane line. The lines should start from the bottom of the image and extend out to the top of the region of interest.
Now for the one with the solid yellow lane on the left. This one's more tricky!
Can you write Python code for it?

yellow_output = 'test_videos_output/solidYellowLeft.mp4'
## To speed up the testing process you may want to try your pipeline on a shorter subclip of the video
## To do so add .subclip(start_second,end_second) to the end of the line below
## Where start_second and end_second are integer values representing the start and end of the subclip
## You may also uncomment the following line for a subclip of the first 5 seconds
##clip2 = VideoFileClip('test_videos/solidYellowLeft.mp4').subclip(0,5)
clip2 = VideoFileClip('test_videos/solidYellowLeft.mp4')
yellow_clip = clip2.fl_image(process_image)
%time yellow_clip.write_videofile(yellow_output, audio=False)

HTML("""
<video width="960" height="540" controls>
  <source src="{0}">
</video>
""".format(yellow_output))


I am working on the file "Laav_Lane_Lines_Detection.ipynb".
The first task is:
Let's try the one with the solid white lane on the right first ...
Can you write Python code for it?

white_output = 'test_videos_output/solidWhiteRight.mp4'
## To speed up the testing process you may want to try your pipeline on a shorter subclip of the video
## To do so add .subclip(start_second,end_second) to the end of the line below
## Where start_second and end_second are integer values representing the start and end of the subclip
## You may also uncomment the following line for a subclip of the first 5 seconds
##clip1 = VideoFileClip("test_videos/solidWhiteRight.mp4").subclip(0,5)
clip1 = VideoFileClip("test_videos/solidWhiteRight.mp4")
white_clip = clip1.fl_image(process_image) #NOTE: this function expects color images!!
%time white_clip.write_videofile(white_output, audio=False)


I am working on the file "Laav_Lane_Lines_Detection.ipynb".
The first task is:
Play the video inline, or if you prefer find the video in your filesystem (should be in the same directory) and play it in your video player of choice.
Can you write Python code for it?

HTML("""
<video width="960" height="540" controls>
  <source src="{0}">
</video>
""".format(white_output))


I am working on the file "Laav_Lane_Lines_Detection.ipynb".
The first task is:
Test Images
Build your pipeline to work on the images in the directory "test_images"
You should make sure your pipeline works well on these images before you try the videos.
Can you write Python code for it?


I am working on the file "Laav_Lane_Lines_Detection.ipynb".
The first task is:
Test on Videos
You know what's cooler than drawing lanes over images? Drawing lanes over video!
We can test our solution on two provided videos:
solidWhiteRight.mp4
solidYellowLeft.mp4
Note: if you get an import error when you run the next cell, try changing your kernel (select the Kernel menu above --> Change Kernel). Still have problems? Try relaunching Jupyter Notebook from the terminal prompt. Also, consult the forums for more troubleshooting tips.
If you get an error that looks like this:
NeedDownloadError: Need ffmpeg exe.
You can download it by calling:
imageio.plugins.ffmpeg.download()
Follow the instructions in the error message and check out this forum post for more troubleshooting tips across operating systems.
Can you write Python code for it?

# Import everything needed to edit/save/watch video clips
from moviepy.editor import VideoFileClip
from IPython.display import HTML

def process_image(image):
    # NOTE: The output you return should be a color image (3 channel) for processing video below
    # TODO: put your pipeline here,
    # you should return the final output (image where lines are drawn on lanes)

    #Copy the image to prevent unecessary changes
    image_cp = np.copy(image)

    # Convert image to grey scale
    gray = grayscale(image_cp)

    # Define a kernel size and apply Gaussian smoothing
    kernel_size = 5
    blur_gray = gaussian_blur(gray, kernel_size)

    # Define our parameters for Canny and apply
    low_threshold = 50
    high_threshold = 150
    edges = canny(blur_gray, low_threshold, high_threshold)

    # Define a polygon region of interest
    # Keep in mind the origin (x=0, y=0) is in the upper left in image processing

    # Coordinates of vertices
    left_upper = [int(xsize/2)-50, lane_limit_height]
    right_upper = [int(xsize/2)+50, lane_limit_height]
    right_bottom = [xsize-50, ysize]
    left_bottom = [50, ysize]

    vertices = np.array( [[ left_upper,right_upper,right_bottom,left_bottom]], dtype=np.int32 )

    region_canny_image = region_of_interest(edges, vertices)

    # ````````````````` Test region of interest```````````````````````````````
    # mpimg.imsave('test_images/output/' + 'test_region_area.png', region_image)
    # plt.imshow(region_image)

    # Define the Hough transform parameters
    # Make a blank the same size as our image to draw on
    rho = 2
    theta = np.pi/180
    threshold = 32
    min_line_len = 10
    max_line_gap = 200
    # As you increase above longer the line connecting the left lane lines

    # Run Hough on edge detected image
    line_image = hough_lines(region_canny_image, rho, theta, threshold, min_line_len, max_line_gap)
    # plt.imshow(line_image)

    # Combine the processed image after selecting the required region with the original img
   ...


I am working on the file "Lab 1 - Esercizi su semplici algoritmi numerici.ipynb".
The first task is:
Esercizio 3: Bisection Search
Si implementi ora una funzione che calcoli, come nell'esercizio precedente, un valore approsimato della radice quadrata di un numero dato, ma che trovi tale valore con un numero ridotto di iterazioni.
Suggerimento: L'algoritmo dovrebbe usare lo stesso metodo che usiamo per cercare un vocabolo in un dizionario: ad ogni iterazione dovremmo dividere lo spazio di ricerca in due metà e decidere in quale delle due continuare a cercare. Lo spazio di ricerca iniziale dovrebbe essere $[0..x]$ se $x>1$, oppure $[0..1]$ se $x \leq1 $.
Can you write Python code for it?

# COMPLETARE LA FUNZIONE SEGUENTE
def BS_ApxRadiceQuadrata(x, epsilon=1e-02):
    # DA COMPLETARE
    # DA COMPLETARE
    # DA COMPLETARE
    return "failed", "", 0

# Funzione di test per la funzione che dovete implementare
def UnitTest():
    Xs = [25, 0.25, 123456, 123456789]
    for i,x in enumerate(Xs):
        print('----------- Start Test ', i+1, ' -------------')
        msg, y, iter = BS_ApxRadiceQuadrata(x)
        if msg == 'ok':
            print('TEST ',i+1,' ok, result: ', y, ' | iterations:', iter)
        else:
            print('TEST ',i+1,' failed! | iterations:', iter)
        print('----------- Test ', i+1, ' Completed -------------')

# Potete scegliere tra
UnitTest()


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
In the next step, we will split all the entries in the RDD on the commas ","

Type:

z_str_rdd_split = z_str_rdd.map(lambda line: line.split(","))

z_str_rdd_split.collect()
Can you write Python code for it?

#Step 2.10 - Perform a map transformation to split all entries in the RDD on the commas ",".

#Check out the entries in the new RDD

#Notice how the entries in the new RDD are now ARRAYs with elements, where the original
#strings have been split using the comma delimiter.


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
In this step, we will augment each entry in the previous RDD with the number "1" to create pairs (or tuples). The first element of the tuple will be the keyword and the second elements of the tuple will be the digit "1".

This is a common technic used to count elements using Spark.

Type:

countWords = z_str_rdd_split_flatmap.map(lambda word:(word,1))

countWords.collect()
Can you write Python code for it?

#Step 2.12 - Learn the difference between two transformations: map and flatMap.
#Go back to the RDD z_str_rdd_split defined above using a map transformation from z_str_rdd
#and use this time a flatmap.


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
Let us now filter out the RDD and only keep the entries that contain the token "Spark". This will be achieved using the "filter" transformation, combined with the Python syntax for figuring out whether a particular substring is present within a larger string: substring in string.

We will also take a look at the first line in the newly filtered RDD.

Type:

Spark_lines = textfile_rdd.filter(lambda line: "Spark" in line)

Spark_lines.first()

Can you write Python code for it?

#Step 3.3 - Filter for only lines with word Spark


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
Let's now create a new RDD with one string "Hello Spark" and take a look at it.

Type:

y = ["Hello Spark!"]

y_str_rdd = sc.parallelize(y)

y_str_rdd.first()

Can you write Python code for it?

#Step 2.6 - Create String RDD, Extract first line


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
Now we have above what is known as a PAIR RDD. Each entry in the RDD has a KEY and a VALUE.

The KEY is the word (First, Line, etc...) and the value is the number "1"

We can now AGGREGATE this RDD by summing up all the values BY KEY

Type:

from operator import add

countWords2 = countWords.reduceByKey(add)

countWords2.collect()

Can you write Python code for it?

#Step 2.13 - Check out the results of the aggregation

#You just created an RDD countWords2 which contains the counts for each token...


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
Now we will point Spark to the text file stored in the local filesystem and use the "textFile" method to create an RDD named "textfile_rdd" which will contain one entry for each line in the original text file.

We will also count the number of lines in the RDD (which would be as well the number of lines in the text file.

Type:

textfile_rdd = sc.textFile("README.md")

textfile_rdd.count()

Can you write Python code for it?


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
Perform a first map transformation and rpelace each element X in the RDD with X+1.

Type:

x_nbr_rdd_2 = x_nbr_rdd.map(lambda x: x+1)
Can you write Python code for it?

#Step 2.4 - Perform your first map transformation. Replace each element X in the RDD with X+1.
#Remember that RDDs are IMMUTABLE, so it is not possible to UPDATE an RDD. You need to create
#a NEW RDD


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
Step 2 - Working with Resilient Distributed Datasets
Step 2 - Create RDD with numbers 1 to 10,

Extract first line,

Extract first 5 lines,

Create RDD with string "Hello Spark",

Extract first line.

Type:

x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

x_nbr_rdd = sc.parallelize(x)

Can you write Python code for it?


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
Take a look at the elements of the new RDD.

Type:

x_nbr_rdd_2.collect()
Can you write Python code for it?

#Step 2.5 - Check out the elements of the new RDD. Warning: Be careful with this in real life !! As you
#will be bringing all elements of the RDD (from all partitions) to the driver...


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
Take a look at the elements of this RDD.

Type:

z_str_rdd.collect()
Can you write Python code for it?

#Step 2.9 - Show all the entries in the RDD. Warning: Be careful with this in real life !!
#As you will be bringing all elements of the RDD (from all partitions) to the driver...


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
Type: <br>
x_nbr_rdd.first()
Can you write Python code for it?


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
Using your knowledge from the previous exercises, you will now count the number of times the substring "Spark" appears in the original text.

Instructions:

Looking back at previous exercises, you will need to:

1- Execute a flatMap transformation on the original RDD Spark_lines and split on white space.

2- Augment each token with the digit "1", so as to obtain a PAIR RDD where the first element of the tuple is the token and the second element is the digit "1".

3- Execute a reduceByKey with the addition to count the number of instances of each token.

4- Filter the resulting RDD from Step 3- above to only keep entries which start with "Spark".
 In Python, the syntax to decide whether a string starts with a token is string.startswith("token").

5- Display the resulting list of tokens which start with "Spark".
Can you write Python code for it?

#Step 3.5 - count the number of instances of tokens starting with "Spark"


I am working on the file "Lab 1 - IntroSpark - Student.ipynb".
The first task is:
We will now count the number of entries in this filtered RDD and present the result as a concatenated string.

Type:

print "The file README.md has " + str(Spark_lines.count()) + \

" of " + str(textfile_rdd.count()) + \

" Lines with the word Spark in it."

Can you write Python code for it?


I am working on the file "Lab 1 - Vectors and Matrices Solutions.ipynb".
The first task is:
2) Use the function outer(vector1, vector2) to find the outer product of rv and cv. Does the order of the arguments matter?
Can you write Python code for it?


I am working on the file "Lab 1 - Vectors and Matrices Solutions.ipynb".
The first task is:
3) Use dot() and .conjugate() to find the dot product of v1 and it's own conjugate:
Can you write Python code for it?


I am working on the file "Lab 1 - Vectors and Matrices Solutions.ipynb".
The first task is:
Can also use the matrix type which is like array but restricts to 2D. Also, matrix adds .H and .I methods for hermitian and inverse, respectively. For more information, see Stack Overflow question #4151128
Can you write Python code for it?

m2 = matrix( [[2,1],[2,1]])

m2.H

eig(m2)

# use a question mark to get help on a command
eig?


I am working on the file "Lab 1 - Vectors and Matrices Solutions.ipynb".
The first task is:
III. Matrices
Can you write Python code for it?

# a two-dimensional array
m1 = array([[2,1],[2,1]])
m1

# can find transpose with the T method:
m1.T

# find the eigenvalues and eigenvectors of a matrix:
eig(m1)


I am working on the file "Lab 1 - Vectors and Matrices Solutions.ipynb".
The first task is:
Matrix in QuTiP
Can you write Python code for it?

qm = Qobj([[1,2],[2,1]])
qm

qm.eigenenergies()  # in quantum (as we will learn) eigenvalues often correspond to energy levels

evals, evecs = qm.eigenstates()

evecs

evecs[0]


I am working on the file "Lab 1 - Vectors and Matrices Solutions.ipynb".
The first task is:
Part 2: Using QuTiP
Keeping track of row and column vectors in Ipython is somewhat artificial and tedious. The QuTiP library is designed to take care of many of these headaches
Can you write Python code for it?

from qutip import *

# Create a row vector:
qv = Qobj([[1,2]])
qv

# Find the corresponding column vector
qv.dag()

qv2 = Qobj([[1+2j,4-1j]])
qv2

qv2.dag()


I am working on the file "Lab 1 - Vectors and Matrices Solutions.ipynb".
The first task is:
Practice:
Problem 1.2 using the hist() function.
Can you write Python code for it?


I am working on the file "Lab 1 - Vectors and Matrices Solutions.ipynb".
The first task is:
Problem 1.8
Hint: using sympy, we can calculate the relevant integral. The conds='none' asks the solver to ignore any strange conditions on the variables in the integral. This is fine for most of our integrals. Usually the variables are real and well-behaved numbers.
Can you write Python code for it?

# Solution
from sympy import *
c,a,x = symbols("c a x")
Q.positive((c,a))
first = integrate(c*exp(-a*x),(x,0,oo),conds='none')
print("first = ",first)
second = integrate(a*exp(-a*x),(x,0,oo),conds='none')
print("second = ",second)


I am working on the file "Lab 1 - Vectors and Matrices Solutions.ipynb".
The first task is:
Signs are opposite compared to the book, but it turns out that (-) doesn't matter in the interpretation of eigenvectors: only "direction" matters (the relative size of the entries).
Example: Problem 1.16 using Ipython functions
Can you write Python code for it?

M16 = array([[0,-1j],[1j,0]])

evals, evecs = eig(M16)

evecs

evecs[:,0]

evecs[:,1]

dot(evecs[:,0].conjugate(),evecs[:,1])


I am working on the file "Lab 1 - Vectors and Matrices Solutions.ipynb".
The first task is:
Two kinds of vector products we'll see: inner product (dot product) and outer product
1) Use the function dot(vector1, vector2) to find the dot product of rv and cv. Does the order of the arguments matter?
Can you write Python code for it?


I am working on the file "Lab 1 - Vectors and Matrices.ipynb".
The first task is:
2) Use the function outer(vector1, vector2) to find the outer product of rv and cv. Does the order of the arguments matter?
Can you write Python code for it?


I am working on the file "Lab 1 - Vectors and Matrices.ipynb".
The first task is:
3) Use dot() and .conjugate() to find the dot product of v1 and it's own conjugate:
Can you write Python code for it?


I am working on the file "Lab 1 - Vectors and Matrices.ipynb".
The first task is:
Example from class:
Can you write Python code for it?

qv = Qobj([[0,1],[-2,3]])
qv

evals, evecs = qv.eigenstates()

evals

evecs

qv*evecs[0]

qv*evecs[0] == evals[0]*evecs[0]

qv*evecs[1] == evals[1]*evecs[1]


I am working on the file "Lab 1 - Vectors and Matrices.ipynb".
The first task is:
Matrix in QuTiP
Can you write Python code for it?


I am working on the file "Lab 1 - Vectors and Matrices.ipynb".
The first task is:
Part 2: Using QuTiP
Keeping track of row and column vectors in Ipython is somewhat artificial and tedious. The QuTiP library is designed to take care of many of these headaches
Can you write Python code for it?


I am working on the file "Lab 1 - Vectors and Matrices.ipynb".
The first task is:
Problem 1.8
Find the constant $c$ that normalizes the probability density:
$$p(x) =
\begin{cases}
ce^{-ax},  & x \geq 0 \[2ex]
0 & x < 0
\end{cases}
$$
Hint: using sympy, we can calculate the relevant integral. The conds='none' asks the solver to ignore any strange conditions on the variables in the integral. This is fine for most of our integrals. Usually the variables are real and well-behaved numbers.
Can you write Python code for it?

# Partial Solution:
from sympy import *
c,a,x = symbols("c a x")
Q.positive((c,a))
first = integrate( c*exp(-a*x) ,(x,0,oo),conds='none')
first

check = integrate( a*exp(-a*x) ,(x,0,oo),conds='none')
check


I am working on the file "Lab 1 - Vectors and Matrices.ipynb".
The first task is:
Signs are opposite compared to the book, but it turns out that (-) doesn't matter in the interpretation of eigenvectors: only "direction" matters (the relative size of the entries).
Example: Problem 1.16 using Python functions
Can you write Python code for it?


I am working on the file "Lab 1 - Vectors and Matrices.ipynb".
The first task is:
Some additional examples we have already done by hand:
Problem 1.2 using the hist() function.
Create a histogram of the data and from it estimate the probability distribution. Use min and max values for the range of 5 and 14, and use 5 bins (5 and 6 go in the first bin).
Can you write Python code for it?

# Solution
data = [10,13,14,14,6,8,7,9,12,14,13,11,10,7,7]
# Fill in the hist() function:
n, bins, patches = hist(data, bins=9, range=(5,14))


I am working on the file "Lab 1 - Vectors and Matrices.ipynb".
The first task is:
Two kinds of vector products we'll see: inner product (dot product) and outer product
1) Use the function dot(vector1, vector2) to find the dot product of rv and cv. Does the order of the arguments matter?
Can you write Python code for it?


I am working on the file "Lab 2 - Cenni di Programmazione Funzionale - Parte prima.ipynb".
The first task is:
2. Funzioni ricorsive
ESERCIZIO: Si scrivano le seguenti funzioni usando delle definizioni ricorsive.

Fattoriale(n)
Fibonacci(n)
IsPalindrome(stringa)
Can you write Python code for it?

# Fattoriale
def Fattoriale(n):
    if n == 0:
        return 1
    else:
        return n*Fattoriale(n-1)

print(Fattoriale(4))

# Fibonacci
def Fib(n):
    if n == 0 or n == 1:
        return 1
    else:
        return Fib(n-1) + Fib(n-2)

print(Fib(6))

# Test if a list (e.g. string) is a palindrome
def IsPalindrome(Ls):
    if len(Ls) <= 1:
        return True
    else:
        return (Ls[0] == Ls[-1]) and IsPalindrome(Ls[1:-1])

print(IsPalindrome("abcdedcba"))
print(IsPalindrome("abcdeecba"))


I am working on the file "Lab 2 - Cenni di Programmazione Funzionale - Parte prima.ipynb".
The first task is:
3. List processing
Esempi da fare:

Sommatoria degli elementi di una lista:
$$\mbox{sum}(Ls) = \sum_{e \in Ls} e$$
Produttoria degli elementi di una lista:
$$\mbox{prod}(Ls) = \prod_{e \in Ls} e$$
Generalizzare le due funzioni precedenti in una singola funzione Fold:
$$\mbox{fold}(f,v_0,[v_1,v_2,v_3,...,v_n]) = f(v_1, f(v_2, f(v_3, ... f(v_n,v_0))))$$
Can you write Python code for it?

def Sum(Ls):
    if Ls == []:
        return 0
    else:
        return Ls[0] + Sum(Ls[1:]) # Notazione postfix per l'addizione

# Meglio se si usa operator.add dalla libreria operator
# https://docs.python.org/3.6/library/operator.html
def Add(x,y):
    return x+y

def Sum2(Ls):
    if Ls == []:
        return 0
    else:
        return Add(Ls[0], Sum(Ls[1:])) # Notazione prefix per l'addizione

def Mul(x,y):
    return x*y

def Prod(Ls):
    if Ls == []:
        return 1
    else:
        return Mul(Ls[0], Prod(Ls[1:]))

def Fold(F, v, Ls):
    if Ls == []:
        return v
    else:
        return F(Ls[0], Fold(F, v, Ls[1:]))

def FoldSum(Ls):
    return Fold(Add, 0, Ls)

def FoldProd(Ls):
    return Fold(Mul, 1, Ls)


I am working on the file "Lab 2 - Cenni di Programmazione Funzionale - Parte prima.ipynb".
The first task is:
ESERCIZIO: Si scriva la funzione reverse(Ls) utilizzando la FolfLeft invece della Fold (se non altrimenti specificato, per fold si intende la FolfRight).
Can you write Python code for it?

def ReverseFoldL(Ls):
    # DA COMPLETARE
    pass


I am working on the file "Lab 2 - Cenni di Programmazione Funzionale - Parte prima.ipynb".
The first task is:
Espressività della funzione fold
Con la funzione fold appena vista, si possono scrivere diverse funzioni classiche che operano su delle liste. Si chiede di sviluppare come esercizio, le seguenti funzioni in termini di fold:

And(Ls): viene valutata a True se tutti gli elementi della lista sono True
Or(Ls): viene valutata a True se almeno uno degli elementi della lista è True
Length(Ls): determina la lughezza di una lista
Reverse(Ls): inverte il contenuto di una lista (esempio: [1,2,3,4] diventa [4,3,2,1])
FoldFactorial(n): una funzione che calcola il fattoriale di n
SumLength(Ls): restituisce una coppia di valori, data dalla somma degli elementi nella lista, e la lunghezza della stringa (esempio: SumLength([1,2,3,4]) = (10,4))
Map(F, Ls): una funzione map equivalente alla builtin di python
Filter(P, Ls): una funzione filter equivalente alla builtin di python
Can you write Python code for it?

# DA SCRIVERE LE 8 FUNZIONI RICHIESTE SOPRA


I am working on the file "Lab 2 - Quantum States - Blank.ipynb".
The first task is:
Devices:
HWP - Half-wave plate axis at $\theta$ to the horizontal
LP - Linear polarizer, axis at $\theta$
QWP - Quarter-wave plate, axis at $\theta$
Note, these are functions so you need to call them with a specific value of theta.
Can you write Python code for it?

def HWP(theta):
    return Qobj([[cos(2*theta),sin(2*theta)],[sin(2*theta),-cos(2*theta)]]).tidyup()

def LP(theta):
    return Qobj([[cos(theta)**2,cos(theta)*sin(theta)],[sin(theta)*cos(theta),sin(theta)**2]]).tidyup()

def QWP(theta):
    return Qobj([[cos(theta)**2 + 1j*sin(theta)**2,
                 (1-1j)*sin(theta)*cos(theta)],
                [(1-1j)*sin(theta)*cos(theta),
                 sin(theta)**2 + 1j*cos(theta)**2]]).tidyup()


I am working on the file "Lab 2 - Quantum States - Blank.ipynb".
The first task is:
These are the polarization states:
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States - Solutions.ipynb".
The first task is:
1) verify that the $|V\rangle$ state is normalized
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States - Solutions.ipynb".
The first task is:
2) Verify that the $|H\rangle$ and $|V\rangle$ states are orthogonal. Repeat for the other pairs of states.
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States - Solutions.ipynb".
The first task is:
3) Find the horizontal component $c_H$ of the state $\psi = \frac{1}{\sqrt{5}}|H\rangle + \frac{2}{\sqrt{5}}|V\rangle$
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States - Solutions.ipynb".
The first task is:
These are the polarization states:
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
2) Verify that the $|V\rangle$ state is normalized
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
3) Verify that the $|H\rangle$ and $|V\rangle$ states are orthogonal. Repeat for the other pairs of states.
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
4) Calculate the horizontal component $c_H$ of the state $\psi = \frac{1}{\sqrt{5}}|H\rangle + \frac{2}{\sqrt{5}}|V\rangle$
Can you write Python code for it?

psi = 1/np.sqrt(5)*H + 2/np.sqrt(5)*V
psi

psi2 = Qobj([[1/np.sqrt(5)],[2/np.sqrt(5)]])
psi2

H.dag()*psi


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
5) Verify Eq. (3.18), $P(H||45\rangle)=\frac{1}{2},$ (which states "The probability that a photon prepared in the +45 state will leave a PA_HV in the Horizontal state is one half.")
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
6) Demonstrate that a half-wave plate at 45-degrees converts $|H\rangle$ to $|V\rangle$
Can you write Python code for it?

HWP(np.radians(45)) * H

HWP(np.pi/4) * H == V


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
7) Re-create Figure 3.9 by plotting the probability P(+45) vs phase φ
Can you write Python code for it?

import matplotlib.pyplot as plt

plt.plot([1,2,3,2,3,4])

phi_list = np.linspace(0,8*np.pi,num=100)

plt.plot(phi,np.sin(phi))

sin_list = [np.sin(p) for p in phi] # notes


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
Example 1) Check that the $|H\rangle$ state is normalized
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
Example 2) Converting from ket to bra:
Can you write Python code for it?

psi = Qobj([[1+1j],[2-1j]])
psi

psi.dag()

psi.dag().dag()


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
These are the polarization states:
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
add text
Can you write Python code for it?

plt.plot(phi,sin_list)

def psi(phi):
    return 1/np.sqrt(2)*(H + np.exp(1j*phi)*V)

answer = [(M45.dag()*psi(phi)).norm()**2 for phi in phi_list]

plt.plot(phi_list/np.pi,answer,"-o")
plt.xlabel("$\phi$")
plt.ylabel("P(-45)")


I am working on the file "Lab 2 - Quantum States Solutions.ipynb".
The first task is:
the .dag() python method computes the "daggar" or the complex transpose.
1) Is psi normalized? If not, find the normalization constant and confirm that constant normalizes psi.
Can you write Python code for it?

psi.dag()*psi

psi_norm = psi*np.sqrt(1/7)

psi_norm.dag() * psi_norm


I am working on the file "Lab 2 - Quantum States.ipynb".
The first task is:
Devices:
HWP - Half-wave plate axis at $\theta$ to the horizontal
LP - Linear polarizer, axis at $\theta$
QWP - Quarter-wave plate, axis at $\theta$
Note, these are functions so you need to call them with a specific value of theta.
Can you write Python code for it?

def HWP(theta):
    return Qobj([[np.cos(2*theta),np.sin(2*theta)],[np.sin(2*theta),-np.cos(2*theta)]]).tidyup()

def LP(theta):
    return Qobj([[np.cos(theta)**2,np.cos(theta)*np.sin(theta)],[np.sin(theta)*np.cos(theta),np.sin(theta)**2]]).tidyup()

def QWP(theta):
    return Qobj([[np.cos(theta)**2 + 1j*np.sin(theta)**2,
                 (1-1j)*np.sin(theta)*np.cos(theta)],
                [(1-1j)*np.sin(theta)*np.cos(theta),
                 np.sin(theta)**2 + 1j*np.cos(theta)**2]]).tidyup()

QWP(np.pi/4)


I am working on the file "Lab 2 - Quantum States.ipynb".
The first task is:
Example 2) Converting from ket to bra:
Can you write Python code for it?


I am working on the file "Lab 2 - Quantum States.ipynb".
The first task is:
These are the polarization states:
Can you write Python code for it?


I am working on the file "Lab 2 - Spark SQL - Student.ipynb".
The first task is:

Dataframes work like RDDs, you can map, reduce, groupby, etc.

Take a look at the first two rows of data using "take"

Can you write Python code for it?

#Use take on the dataframe to pull out 2 rows


I am working on the file "Lab 2 - Spark SQL - Student.ipynb".
The first task is:

We can look at the schema with this command:

example1_df.printSchema()
Can you write Python code for it?


I am working on the file "Lab 2 - Spark SQL - Student.ipynb".
The first task is:
Step 2

Dowload a JSON Recordset to work with

Let's download the data, we can run commands on the console of the server (or docker image) that the notebook enviroment is using. To do so we simply put a "!" in front of the command that we want to run. For example:
!pwd
To get the data we will download a file to the enviroment. Simple run these two commands, the first just ensures that the file is removed if it exists:
!rm world_bank.json.gz -f

!wget https://raw.githubusercontent.com/bradenrc/sparksql_pot/master/world_bank.json.gz
Can you write Python code for it?

#enter the commands to remove and download file here


I am working on the file "Lab 2 - Spark SQL - Student.ipynb".
The first task is:
Step 4

Register a table

Using
DataframeObject.registerTempTable("name_of_table")
Create a table named "world_bank"
Can you write Python code for it?

#Create the table to be referenced via SQL


I am working on the file "Lab 2 - Spark SQL - Student.ipynb".
The first task is:
Step 5

Writing SQL Statements

Using SQL Get the first 2 records
sqlContext.sql("SQL Statement") will return a Dataframe with the records
Can you write Python code for it?

#Use SQL to select from table limit 2 and print the output

#Extra credit, take the Dataframe you created with the two records and convert it into Pandas

#Now Calculate a Simple count based on a group, for example "regionname"

# With JSON data you can reference the nested data
# If you look at Schema above you can see that Sector.Name is a nested column
# Select that column and limit to reasonable output (like 2)


I am working on the file "Lab 2 - Spark SQL - Student.ipynb".
The first task is:
Step 6

Creating simple graphs

Using Pandas we can do create some simple visualizations.
First create a SQL statement that is a resonable number if items
For example, you can count the number of projects (rows) by countryname

or in anothe words:

count(*), countryname from table group by countryname
Can you write Python code for it?

# we need to tell the charting library (matplotlib) to display charts inline
# just run this paragraph
%matplotlib inline
import matplotlib.pyplot as plt, numpy as np

# first write the sql statment and look at the data, remember to add .toPandas() to have it look nice
# an even easier option is to create a variable and set it to the SQL statement
# for example:
# query = "select count(*) as Count, countryname from world_bank group by countryname"
# chart1_df = sqlContext.sql(query).toPandas()
# print chart1_df

# now take the variable (or same sql statement) and use the method:
# .plot(kind='bar', x='countryname', y='Count', figsize=(12, 5))


I am working on the file "Lab 2 - Spark SQL - Student.ipynb".
The first task is:
Step 7

Creating a dataframe "manually" by adding a schema to an RDD

First, we need to create an RDD of pairs or triplets. This can be done using code (for loop) as
seen in the instructor's example, or more simply by assigning values to an array.
Can you write Python code for it?

# Default array defined below. Feel free to change as desired.
array=[[1,1,1],[2,2,2],[3,3,3],[4,4,4],[5,5,5]]
my_rdd = sc.parallelize(array)
my_rdd.collect()


I am working on the file "Lab 3 - Cenni di Programmazione Funzionale - Parte seconda.ipynb".
The first task is:
4. Lazy Evaluation e Liste infinite
Negli esempi sopra, le funzioni map e filter erano funzioni che restituivano delle lista che venivano elaborate in maniera LAZY: ovvero, l'elemento i-esimo della lista non sono calcolato sino a quando non è effettivamente necessario per effettuare qualche calcolo. Per questo motivo, si dice che la lista viene valutata in modalità lazy. Questo è un concetto molto generale, che in Python viene implementato attraverso degli oggetti chiamati generators e iterators. I primi sono gli oggetti che generano le liste, mentre i secondi sono gli oggetti che permettono di valutare tali liste in modalità lazy, valutando un elemento alla volta solo quando viene effettivamente richiesto. Per poter definire una funzione che genera una lista infinita si usa la parola chiave yield invece del solito return. Per richiedere un elemento alla volta di un iteratore si utilizza la builtin next() (vedere l'esempio sotto).
Si noti, che sfruttando una valutazione delle liste di tipo LAZY è quindi possibile definire delle liste di lunghezza infinita, che però vengono valutate solamente quando uno dei suoi elementi è veramente necessario per effettuare dei conti.
ESEMPI:
1. Funzione Counter() che restituisce la lista infinita dei numeri natuali a partire da 1.
2. Funzione builtin Enumerate(Ls) che restituisce la coppia (i,l) dove l è l'i-esimo elemento della lista Ls.
Can you write Python code for it?

# Primo esempio di lista "infinita"
def Counter():
    c = 1
    while True:
        yield c
        c += 1

cnt = Counter()
print(type(cnt))
print([next(cnt) for _ in range(10)])

# Implementazione naive in Python della funzione builtin `enumerate(Ls)`
def Enumerate(Ls):
    i = 0
    for l in Ls:
        yield i,l
        i += 1

print(Enumerate("CiaoBella"))
print(list(Enumerate("CiaoBella")))


I am working on the file "Lab 3 - Cenni di Programmazione Funzionale - Parte seconda.ipynb".
The first task is:
ESERCIZIO 3: Scrivere una funzione che calcoli la norma di un vettore $\sqrt{\sum_{i=1,..,n} x_i^2}$. Suggerimento: la funzione sqrt è una funzione della libreria math e deve essere importata con ìl comando from math import sqrt.
Can you write Python code for it?

from math import sqrt

# SVOLGERE ESERCIZIO


I am working on the file "Lab 3 - Cenni di Programmazione Funzionale - Parte seconda.ipynb".
The first task is:
ESERCIZIO 4: Implementare una funzione che restituisce lista infinita dei numeri primi primi.
Can you write Python code for it?

# Terzo esempio di lista infinita
def NumeriPrimi():
    # DA COMPLETARE

crivello = NumeriPrimi()
print([next(crivello) for _ in range(10)])


I am working on the file "Lab 3 - Cenni di Programmazione Funzionale - Parte seconda.ipynb".
The first task is:
ESERCIZIO 5 (FACOLTATIVO): Implementare una funzione che restituisca la lista infinita dei numeri dati dalla successione di Fibonacci.
Can you write Python code for it?

def Fibonacci():
    # DA COMPLETARE
    pass


I am working on the file "Lab 3 - Cenni di Programmazione Funzionale - Parte seconda.ipynb".
The first task is:
List comprehensions
Usare la notazione di list comprehensions può rendere talvolta il codice più compatto e può spostare il focus di chi lo legge dal come si stan facendo i conti al cosa si sta calcolando. Si consideri l'esempio seguente:
collection = list()
for datum in data_set:
    if condition(datum):
        collection.append(datum)
    else:
        new = modify(datum)
        collection.append(new)
che usando la notazione di list comprehensions si può riscrivere come:
collection = [d if condition(d) else modify(d) for d in data_set]
Funzione reduce
La funzione reduce letteralmente "riduce" una sequenza di elementi ad uno scalare. In programmazione funzionale viene chiamata anche fold. La funzione reduce prende in input una funzione, chiamata "combinante", una sequenza iterabile di elementi e un valore iniziale (facoltativo). In output viene restituito un valore che risulta dall'applicare in sequenza agli elementi della lista la funzione data:

`reduce(function, sequence[, initial]) -> value`

Per esempio, se viene passata in input la funzione f(x,y), la lista [1,2,3,4] e il valore iniziale 0, la funzione reduce calcola il valore:
$$v = f(f(f(f(0,1), 2), 3), 4)$$
Se la funzione data è la somma, questo risulta equivalente a calcolare:
$$((((0+1)+2)+3)+4) = 10$$
La funzione reduce non è una builtin di Python e deve essere importata con il comando:
from functools import reduce
ESEMPIO 4: Scrivere il codice per l'esempio precedente
Can you write Python code for it?

from functools import reduce

reduce(lambda x,y: x+y, [1,2,3,4])


I am working on the file "Lab 3 - Operators - Solutions (old).ipynb".
The first task is:
4) Represent $\hat{P}_H$ in the ±45 basis.
Check your answer against Eqns. 4.A.17 and 4.72
Can you write Python code for it?


I am working on the file "Lab 3 - Operators - Solutions (old).ipynb".
The first task is:
Same with the $\hat{P}_V$ operator:
Can you write Python code for it?


I am working on the file "Lab 3 - Operators Solutions.ipynb".
The first task is:
1) Using the $\hat{R}_p(\theta)$ operator, verify the operator properties described in Sections 4.1 and 4.2. Specifically, verify Eqns. 4.6, 4.7, 4.16, 4.18, 4.22, and 4.27
Can you write Python code for it?


I am working on the file "Lab 3 - Operators Solutions.ipynb".
The first task is:
4) Use the similarity transform to represent $|V\rangle$ in the ±45 basis
Can you write Python code for it?


I am working on the file "Lab 3 - Operators Solutions.ipynb".
The first task is:
6) Represent $\hat{P}_V$ in the ±45 basis.
Can you write Python code for it?


I am working on the file "Lab 3 - Operators Solutions.ipynb".
The first task is:
Example 2: Verify Eq. 4.38 for the HV basis states. Repeat for the ±45, and LR basis
Can you write Python code for it?

identity(2)

Ph + Pv == identity(2)

P45*P45.dag()

M45*M45.dag()

P45*P45.dag() + M45*M45.dag()

L*L.dag()

R*R.dag()

L*L.dag() + R*R.dag()


I am working on the file "Lab 3 - Operators Solutions.ipynb".
The first task is:
Example 3: Represent the $\hat{R}_p(\theta)$ operator in the HV basis and verify your representation by operating on $|H\rangle$ and $|V\rangle$ states. Use the following template function definition.
Can you write Python code for it?

def Rp(theta):
    return Qobj([[cos(theta),-sin(theta)],[sin(theta),cos(theta)]]).tidyup()

Rp(pi/2)

V==Rp(pi/2)*H

# Solution Goes Here
H==Rp(-pi/2)*V


I am working on the file "Lab 3 - Operators Solutions.ipynb".
The first task is:
Example: the similarity transform
The following defines a function that creates a similarity transform matrix. It takes the two old basis vectors and the two new basis vectors as arguments. To apply the transform, simply multiply the matrix onto the state vector or ooperator matrix. Following the examples below, explore this transform.
Can you write Python code for it?

def sim_transform(o_basis1, o_basis2, n_basis1, n_basis2):
    a = n_basis1.dag()*o_basis1
    b = n_basis1.dag()*o_basis2
    c = n_basis2.dag()*o_basis1
    d = n_basis2.dag()*o_basis2
    return Qobj([[a.data[0,0],b.data[0,0]],[c.data[0,0],d.data[0,0]]])


I am working on the file "Lab 3 - Operators Solutions.ipynb".
The first task is:
Same with the $\hat{P}_V$ operator:
Can you write Python code for it?


I am working on the file "Lab 3 - Operators.ipynb".
The first task is:
Example 1: the outer product and the projection operator
We already have the $|H\rangle$ state represented as a vector in the HV basis, so the $\hat{P}_H$ operator is the outer product $|H\rangle\langle H|$ (a ket then a bra):
Can you write Python code for it?


I am working on the file "Lab 3 - Operators.ipynb".
The first task is:
Example 3: Represent the $\hat{R}_p(\theta)$ operator in the HV basis and verify your representation by operating on $|H\rangle$ and $|V\rangle$ states. Use the following template function definition.
Can you write Python code for it?

def Rp(theta):
    return Qobj([[cos(theta),-sin(theta)],[sin(theta),cos(theta)]]).tidyup()

Rp(pi/2)

V==Rp(pi/2)*H

# Solution Goes Here


I am working on the file "Lab 3 - Operators.ipynb".
The first task is:
Example: the similarity transform
The following defines a function that creates a similarity transform matrix. It takes the two old basis vectors and the two new basis vectors as arguments. To apply the transform, simply multiply the matrix onto the state vector or ooperator matrix. Following the examples below, explore this transform.
Can you write Python code for it?


I am working on the file "Lab 3 - Operators.ipynb".
The first task is:
Same with the $\hat{P}_V$ operator:
Can you write Python code for it?


I am working on the file "Lab 3 - Operators.ipynb".
The first task is:
We can define a similarity transform that converts from $HV\rightarrow \pm 45$
Can you write Python code for it?

Shv45 = sim_transform(H,V,P45,M45)  # as found in Example 4.A.1, Eq. 4.A.10.
Shv45

Shv45 * H  # compare to Eq. 4.A.12


I am working on the file "Lab 4 - Matrici, Immagini e Frattali.ipynb".
The first task is:
ESERCIZIO: Provare ad usare diversi valori per la colormap, come indicato nel relativo tutorial.
Un frattale semplice: il Julia Set
Usando le matrici, si può facilmente arrivare a rappresentare immagini frattali in 2D.
Prendiamo per esempio un semplice insieme di Julia. Per darne una rappresentazione grafica, usiamo il piano complesso tramite una matrice come nell'esempio precedente, e per ogni punto del piano complesso $(r,c)$, corrispondente all'elemento $a_{(n-r)(n-c)}$ della matrice quadrata $n \times n$, controlliamo se la successione dei punti:
$$z_{k+1} = z_k^2 + c$$
converge in un numero finito di iterazioni, oppure diverge. La successione si considera divergere quando $\mid z_{k+1} \mid > 2$, mentre si considera a convergenza se dopo un numero massimo di iterazioni la condizione precedente non si è ancora verificata.
ESERCIZIO 1: Si completi la funzione JuliaSetRec(z, c, k) seguente in modo che controlli se la successione $z_{k+1} = z_k^2 + c$ sia convergente o meno; il parametro z corrisponde al numero complesso, c è la costante che appare nella successione, k è il numero di ricorsione corrente. Se la successione converge la funzione ritorna il valore 0; se la successione diverge la funzione ritorna il numero di ricorsione k in cui si è dichiarata la successione come divergente. Si implementi la funzione in maniera RICORSIVA.
Can you write Python code for it?

# Per informazioni di base sui Julia Set, vedere per esempio:
# https://en.wikipedia.org/wiki/Julia_set
def JuliaSetRec(z, c, k, max_k=64):
    # DA COMPLETARE

    # FAR TORNARE IL VALORE COME SPECIFICATO SOPRA
    return 0

def JuliaSet(z, c=-0.413):
    return JuliaSetRec(z, c, 0)

# Alcuni parametri
DrawImage(JuliaSet, 200, 0.01)


I am working on the file "Lab 4 - Matrici, Immagini e Frattali.ipynb".
The first task is:
Un gradiente circolare
Per rendersi idea di come vengono gestiti i colori, si consideri il codice seguente, in cui per comodità si usa il piano complesso, e si fa corrispondere ad ogni elemento $a_{rc}$ di una matrice quadrata $n \times n$ il valore:
$$a_{rc} = \mid complex(r-n, c-n) \mid$$
in cui complex(a,b) è la primitiva di Python che crea il numero complesso $a+b\,i$ (i unità immaginaria).
Can you write Python code for it?

# Calcola il modulo del numero complesso
# che viene usato per stabilire il colore nell'immagine
def ComplexModulo(z):
    return abs(z)

# Costruisci una matrice n x n dove ciascun elemento
# corrisponde ad un pixel dell'immagine; il valore numerico
# dell'elemento ne determina il colore
def MakeImage(F, n, scale=0.01):
    data = [scale*i for i in range(-n,n)]
    return np.matrix([[F(complex(a, b)) for a in data] for b in data])

# Disegna l'immagine della matrice creata dalla funzione F(z) : Z -> R
def DrawImage(F, n, scale):
    # Restituisce una matrice n*n
    m = MakeImage(F, n, scale)
    # Crea una figura, disegna l'immagine data dalla matrice, aggiungi la colorbar sulla destra
    plt.figure(figsize=(6,6))
    img = plt.imshow(m, extent=(scale*n, -scale*n, scale*n, -scale*n), cmap='jet')
    plt.colorbar()
    plt.show()

# Alcuni parametri
DrawImage(ComplexModulo, 200, 0.01)


I am working on the file "Lab 4 - Measurements Solutions.ipynb".
The first task is:
10,000 does pretty well for getting to the predictions. "There is no substitute for an adequate sample size."
Q: Answer problems 5.11, 5.12, 5.13, 5.14, 5.17, 5.18, 5.19 from the textbook. These are an opportunity to practice with a new operator $\hat{\mathscr{P}}_{C}$
Can you write Python code for it?


I am working on the file "Lab 4 - Measurements Solutions.ipynb".
The first task is:
5.19 - this one is tricky, but is easier with our custom function
Can you write Python code for it?

psi = 1/sqrt(3)*H + sqrt(2/3.0)*exp(1j*pi/3.0)*V
psi.norm()

Delta(psi,P_45)*Delta(psi,Phv)

1/2j*(psi.dag()*(Phv*P_45 - P_45*Phv)*psi).data[0,0]


I am working on the file "Lab 4 - Measurements Solutions.ipynb".
The first task is:
Ex: Use the random function to generate a mock data set for the state $|\psi\rangle$.
random.choice([1,-1],size=10,p=[0.2,0.8])

gives a list of 10 numbers, either 1 or -1 with the associated probability p:
Can you write Python code for it?

data = random.choice([1, -1],size=20,p=[0.2,0.8])

data.mean()

data.var()


I am working on the file "Lab 4 - Measurements Solutions.ipynb".
The first task is:
Q: Define the $\hat{\mathscr{P}}_{HV}$ operator
Can you write Python code for it?


I am working on the file "Lab 4 - Measurements Solutions.ipynb".
The first task is:
Q: Verify the mean and variance of the mock data set match your QM predictions. How big does the set need to be for you to get ±5% agreement?
Can you write Python code for it?

data = random.choice([1, -1],size=10000,p=[0.2,0.8])

data.mean()

data.var()


I am working on the file "Lab 4 - Measurements Solutions.ipynb".
The first task is:
Q: What is the expectation value $\langle \hat{\mathscr{P}}_{HV}\rangle$ for state $|\psi\rangle = \frac{1}{\sqrt{5}}|H\rangle + \frac{2}{\sqrt{5}}|V\rangle$? Interpret this result given the amplitudes in the state.
Can you write Python code for it?

psi = 1/sqrt(5)*H + 2/sqrt(5)*V

psi.dag()*Phv*psi


I am working on the file "Lab 4 - Measurements.ipynb".
The first task is:
Q: Define the $\hat{\mathscr{P}}_{HV}$ operator
Can you write Python code for it?


I am working on the file "Lab 4 - Measurements.ipynb".
The first task is:
Q: Verify the mean and variance of the mock data set match your QM predictions. How big does the set need to be for you to get ±5% agreement?
Can you write Python code for it?


I am working on the file "Lab 4 - Measurements.ipynb".
The first task is:
Q: What is the expectation value $\langle \hat{\mathscr{P}}_{HV}\rangle$ for state $|\psi\rangle = \frac{1}{\sqrt{5}}|H\rangle + \frac{2}{\sqrt{5}}|V\rangle$? Interpret this result given the amplitudes in the state.
Can you write Python code for it?


I am working on the file "Lab 4 - Measurements.ipynb".
The first task is:
Q: What is the variance of $\mathscr{P}_{HV}$?
Can you write Python code for it?


I am working on the file "Lab 5 - Quick Look at Machine Learning.ipynb".
The first task is:
Predizione tramite i k campioni più simili
Per una rapida descrizione del metodo si rimanda alla documentazione ufficiale:

Introduzione: Nearest Neighbors
Metodo da usare: KNeighborsClassifier

IMPORTANTE: Per imparare a programmare, si deve imparare a leggere e studiare la DOCUMENTAZIONE delle librerie che si utilizzano!
ESERCIZIO 1: Provare il classificatore seguente provando a cambiare il numero di vicini utilizzati (parametro n_neighbors). Provare a usare due algoritmi diversi: ball_tree e kd_tree (leggere la documentazione per capire come fare). Commentare la differenza di risultati ottenuti con i due metodi.
Can you write Python code for it?

from sklearn import neighbors
knn = neighbors.KNeighborsClassifier(n_neighbors=5)
# Input to this function must be "DataFrames"
knn.fit(x_train, y_train)

y_pred = knn.predict(x_test)
print(mean_absolute_error(y_test, y_pred), mean_squared_error(y_test, y_pred), r2_score(y_test, y_pred))

from sklearn.metrics import confusion_matrix, accuracy_score, classification_report
print('ACCURACY:',accuracy_score(y_test, y_pred))
print('REPORT:',classification_report(y_test, y_pred))
print('Confusion Matrix:', confusion_matrix(y_test, y_pred))


I am working on the file "Lab 5 - Quick Look at Machine Learning.ipynb".
The first task is:
Rappresentazioni grafiche con Seaborn
Can you write Python code for it?

import seaborn as sns

sns.jointplot(data=bm, x='Age', y='Official Time', kind='reg', color='g')
sns.plt.show()

bm_female = bm[bm['M/F'] == 'F']
sns.jointplot(data=bm_female, x='Age', y='Official Time', kind='reg', color='r')
sns.plt.show()

bm_male = bm[bm['M/F'] == 'M']
sns.jointplot(data=bm_male, x='Age', y='Official Time', kind='reg', color='b')
sns.plt.show()


I am working on the file "Lab 5 - Two-particle systems.ipynb".
The first task is:
$P(H_s|H_i) = \frac{P(H_s,H_i)}{P(H_i)}$
Can you write Python code for it?


I am working on the file "Lab 5 - Two-particle systems.ipynb".
The first task is:
Example 8.2 prob. of measuring vertical signal and horizontal idler if $|\psi\rangle = |R,+45\rangle$
Can you write Python code for it?

# First, form the prepared state:
psi = tensor(R,P45)

# Then create the projection operator for the state we are asking about:
projection = VH*VH.dag()

# Finally, calculate the probability by computing the bra-ket:
psi.dag()*projection*psi


I am working on the file "Lab 5 - Two-particle systems.ipynb".
The first task is:
Example: find the probability of measuring a horizontal idler photon in the state $|\psi\rangle = |H,+45\rangle$
Can you write Python code for it?

psi = tensor(H,P45)  # the prepared state

psi.dag()*Ph_i*psi


I am working on the file "Lab 5 - Two-particle systems.ipynb".
The first task is:
Finally, to really drive this odd point home, show that they are never found in the $\big|+45,-45\big\rangle$ state:
Can you write Python code for it?

# Solution

# Probability that they are in different 45 states:
Pp45m45 = tensor(P45,M45) * tensor(P45,M45).dag()

phiPlus.dag()*Pp45m45*phiPlus


I am working on the file "Lab 5 - Two-particle systems.ipynb".
The first task is:
Guaranteed to measure a horizontal signal photon whenever a horizontal idler photon is measured. What about vertical? Find the conditional probability of measuring a vertical signal photon if the idler photon is found to be vertical:
Now, measure a different basis (use the +45 states) to show that the photons are always found in the same polarization even when measured at a different angle:
Can you write Python code for it?

# Solution

# Probability that signal is +45 and idler +45
Pp45p45 = tensor(P45,P45) * tensor(P45,P45).dag()
phiPlus.dag()*Pp45p45*phiPlus

# Solution

# Probability that the idler is +45 regardless of the signal
Pp45i = tensor(qeye(2),P45) * tensor(qeye(2),P45).dag()
phiPlus.dag()*Pp45i*phiPlus


I am working on the file "Lab 5 - Two-particle systems.ipynb".
The first task is:
It can be hard to interpret these values visually but remember it was constructed by multiplying all the terms between two matrices with only diagonal elements. It makes sense that the result is also diagonal. Also, the sign of the diagonal depends on the state of the signal photon (the first one listed). Recall the states are in the order: HH, HV, VH, VV so the first two states have H signal photons and are therefore 1, and the second two states are V signal photons so -1 for those diagonals.
Now construct the two-photon operator that measures the idler photon:
Can you write Python code for it?


I am working on the file "Lab 5 - Two-particle systems.ipynb".
The first task is:
So we interpret the state $|HH\rangle$ as the vector (1,0,0,0) in a four-dimensional space.
Recall: The polarization measurement operator (for one photon):
Can you write Python code for it?


I am working on the file "Lab 5 - Two-particle systems.ipynb".
The first task is:
So we see that the tensor product has the following elements: 1*3 = 3, 1*4 = 4, 2*3 = 6, 2*4 = 8. Essentially, we distributed the multiplication of the first vector through the second vector. Using the technical terms of vector spaces, the tensor product exists in a larger Hilbert space (the number of dimensions is the product of the dimensions of the original states). See this with larger initial states: two 3-dim vectors have a tensor product in 9-dim space:
Can you write Python code for it?


I am working on the file "Lab 5 - Two-particle systems.ipynb".
The first task is:
The polarization states (in the HV-basis):
Can you write Python code for it?


I am working on the file "Lab 5 - Two-particle systems.ipynb".
The first task is:
This is expected, because the HH state has 50% of the probability amplitude. Same for a horizontal signal photon:
Can you write Python code for it?

phiPlus.dag()*Ph_s*phiPlus  # probability of measuring a horizontal signal photon


I am working on the file "Lab 6 - Spin.ipynb".
The first task is:
Ex: find $\langle \hat{S}_x\rangle$ for the state $|\psi\rangle=|+Z\rangle$.
Can you write Python code for it?


I am working on the file "Lab 6 - Spin.ipynb".
The first task is:
Example: determine $P(S_x = \frac{\hbar}{2} ||-y\rangle)$
Can you write Python code for it?


I am working on the file "Lab 6 - Spin.ipynb".
The first task is:
Example: verify the commutation relation: $\left[\hat{S}_x,\hat{S}_z\right] = -i\hbar\hat{S}_y$
Can you write Python code for it?

Sx*Sz - Sz*Sx == -1j*Sy  # remember, h = 1


I am working on the file "Lab 6 - Spin.ipynb".
The first task is:
This makes sense given that $S_x$ can be either $\frac{+\hbar}{2}$ or $\frac{-\hbar}{2}$ with equal probability. Similarly, if the state is $|\psi\rangle=|+x\rangle$.
Can you write Python code for it?


I am working on the file "Lab 6/Lab 6.ipynb".
The first task is:
Task 1: Capture an unstructured light field (20 points)
You will capture a video as you wave your camera around in front of a static scene. Here are some guidelines.

You need to avoid tilting and rotating the camera as much as possible. You should just shift the camera in a plane (see Figure 1). You may need to try this several times before you capture a good video.

You may want to press the phone against a flat object to help guide the planar motion

We might want to use a zig-zag motion In general, the more you cover the plane, the better your results in the next section will be.

Make sure your video is not too long. You should capture a few seconds of video.
Capture a scene with a few objects at different depths one object with enough depth. See Stanford LF repository for examples. In the next section you will create narrow synthetic aperture images that are focused on these objects
Make sure that all of the objects in the scene are in focus. As long as objects are about a meter in size and 1-2 meters away, they will more or less be in focus.

Task 2: Process video and extract frames (30 points)
Once you have captured a video, we need to register frames of captured video and generate our LF.
You should be able to use imageio to load a video and convert it to frames. Have a look to Examples. The idea is to select a number of frames to create a LF. Anything between 7x7 and 11x11 will be fine.
When you save the frames, you might want to reduce the resolution too, and maybe crop to the area of interest, to make it easier to handle later. Arround 400 pixels will be enough.
Here an example of an actual lightfield capture with Lytro (GIL, University of Zaragoza)

Note that if you recorded about 1 minute of video, this means 1800 frames, which is not nice to select frames manually!! So you might want to look at your video before in VLC and pre-select the images before saving them.
YOU DON'T NEED TO CREATE A SINGLE IMAGE FOR YOUR LF
Can you write Python code for it?

reader = imageio.get_reader('iPhone/IMG_8189.MOV')

data=[]
data_full = []
for im in reader:
    im1 = im[0::3,0::3]
    data_full += [im1]
    data+=[im1[200:550,0:350]]
data=data[:81]
# obraz = data[20]
show_image(data[21])
# print obraz, obraz[0::2], obraz[0::2,0::2]
# show_image(obraz[0::3,0::3])
print(len(data))

size=9
gray_data = []
for x in range(0, len(data)):
#     imageio.imwrite("data/{0}-{1}.png".format(x//9+1,x%9+1), data[x])
    r, g, b = data[x][:,:,0], data[x][:,:,1], data[x][:,:,2]
    gray = (0.2989 * r + 0.5870 * g + 0.1140 * b)/255.
    gray_data += [gray]
    gray2 = [make_frame(gray_data[0],sizex = 8,coords = (150,70), bold = 1, color = 0)]
#     imageio.imwrite("gray_data/{0}-{1}.png".format(x//9+1,x%9+1), gray)
#     imageio.imwrite("test/{0}-{1}.png".format(x//9+1,x%9+1), gray2)

size=9
big_frame = (80,220,50,190)


I am working on the file "Lab 7 - Implementazione di regressioni.ipynb".
The first task is:
Esercizio 1: Regressione Lineare
Si usino le slide usate a lezione.
Confrontare i coefficienti $w$ trovati dalla vostra soluzione con quelli che trova la classe LinearRegression della libreria Scikit Learn.
Can you write Python code for it?

class RegressioneLineare(object):
    def fit(self, x, y):
        # Build the matrix with vector (1, x) as rows
        X = np.matrix(list(map(lambda row: np.append([1], row), x)))
        # Solve the normal equation (what if X is not invertible?)
        self.w = matmul(matmul(inv(matmul(transpose(X), X)), transpose(X)), y)

    def predict(self, x):
        # Build the matrix with vector (1, x) as rows
        X = np.matrix(list(map(lambda row: np.append([1], row), x)))
        # Predict values
        return matmul(transpose(X), self.w)

from sklearn.linear_model import LinearRegression

lr = LinearRegression(normalize=False)
lr.fit(X, Y)
print('Scikit LinearRegression, pesi trovati:', lr.intercept_, lr.coef_)

my = RegressioneLineare()
my.fit(X,Y)
print('My Regressione lineare, pesi trovati:', my.w)


I am working on the file "Lab 7 - Time Evolution.ipynb".
The first task is:
Define the Hamiltonian:
$$H= - \mathbf{\mu}\cdot \mathbf{B} =-\gamma S_z B$$
$$\hat{H} = -\Omega \hat{S}_z$$
Can you write Python code for it?

Omega = 5
H = -Omega*Sz

t = arange(0,4*pi/Omega,0.05)


I am working on the file "Lab 7 - Time Evolution.ipynb".
The first task is:
Hamiltonian is the same for both particles so we use the tensor to form $\hat{H}$ from individual operators
Can you write Python code for it?

omega = 5
H = -omega*tensor(Sz,Sz)

expect_op = tensor(mz,pz)*tensor(mz,pz).dag()
result3 = sesolve(H, psi, t, expect_op)

for r,l in zip(result3.expect,labels):
    plt.plot(result3.times*omega/pi, r, label="$\langle -z,+z\\rangle$")

plt.xlabel("Time ($\Omega t/\pi$)", size=18)
plt.legend(fontsize=16) #TODO fix legend text size


I am working on the file "Lab 7 - Time Evolution.ipynb".
The first task is:
Harder to interpret, so we'll use the Bloch sphere:
Can you write Python code for it?

sx, sy, sz = result4.expect

b = Bloch()
b.add_points([sx,sy,sz])
b.zlabel = ['$\\left|+z\\right>$', '$\\left|-z\\right>$']
b.view = [-45,20]
b.add_vectors([1/sqrt(2),0,1/sqrt(2)])
b.show()


I am working on the file "Lab 7 - Time Evolution.ipynb".
The first task is:
Make a plot analogous to Fig 9.A.1:
Can you write Python code for it?

plt.tick_params(labelsize=18)
plt.plot(result5.times*omegaR/pi,Pmz)
plt.xlabel("Time ($\Omega_R t/\pi$)", size=18)
plt.ylabel("$P(-z, t)$", size=18)


I am working on the file "Lab 7 - Time Evolution.ipynb".
The first task is:
The next line calls a Schrödinger equation solver (sesolve). The arguments are the Hamiltonian, the starting state $\lvert+z\rangle$ (pz), the time values, a list of operators, and the arguments to the function H_t. sesolve returns many things, but the expect method is most useful, it gives the expectation values of the four operators in the operator list. Notice the fourth operator is the $\lvert-z\rangle$ projection operator. It's expectation value is $P(\lvert-z\rangle,t)$
Can you write Python code for it?

result5 = sesolve(H, pz, t, [Sx, Sy, Sz, mz*mz.dag()],args)
sx, sy, sz, Pmz = result5.expect


I am working on the file "Lab 7 - Time Evolution.ipynb".
The first task is:
Time-dependent Hamiltonian:
We'll explore the parameters of a spin in a time-varying magnetic field. This system is relevant to nuclear magnetic resonance (NMR) which is used in chemistry and as Magnetic Resonance Imaging (MRI) in medicine.
Following Compliment 9.A the Hamiltonian is:
$$\hat{H}= - \Omega_0 \hat{S}_z - \Omega_1 cos(\omega t)\hat{S}_x$$
We then solve for a certain amount of time after the state starts in $|\psi(0)\rangle = |+z\rangle$
We also use the definition of the Rabi frequency: $\Omega_R = \sqrt{(\omega - \Omega_0)^2 + (\Omega_1/2)^2}$ as in (9.A.28)
Note that the time span is 3 units of $\frac{2\pi}{\Omega_R}$. Leave the scaling in place, but to plot a longer time period, change 3.0 to something larger. This lets us match the units in Fig. 9.A.1.
Can you write Python code for it?

omega0 = 2.0 * 2 * pi   # pick a nice value for a frequency, note this is 1 Hz
omega1 = 0.25 * 2 * pi  # 25% of omega0
w = 2.0 * 2 * pi        # the driving frequency

H0 = - omega0 * Sz      # the first term in H
H1 = - omega1 * Sx      # the second term in H

omegaR = sqrt((w - omega0)**2 + (omega1/2.0)**2)
t = arange(0,3.0 * 2 * pi / omegaR,0.02)   # scale the time by omegaR, plot 3 units of 2pi/omegaR

args = [H0, H1, w]      # parts of the Hamiltonian

def H1_coeff(t, args):
    return cos(w * t)

H = [H0, [H1, H1_coeff]]


I am working on the file "Lab 8 - SHO Solutions.ipynb".
The first task is:
Alternatively, we can find the indeterminacy bound for ΔX and ΔP (the unitless operators): $$\Delta X \Delta P \geq \frac{1}{2}\left|\left\langle \left[\hat{X},\hat{P}\right] \right\rangle\right|$$
Can you write Python code for it?


I am working on the file "Lab 8 - SHO Solutions.ipynb".
The first task is:
Problem 12.2:
Can you write Python code for it?

n*a.dag() - a.dag()*n

n*a.dag() - a.dag()*n == a.dag()


I am working on the file "Lab 8 - SHO Solutions.ipynb".
The first task is:
Problem 12.3 (use n=2 as a test-case):
To define $|2\rangle$ use the basis(N,n) command where N is the dimension of the vector, and n is the quantum number.
Can you write Python code for it?

psi = basis(N,2)

psi

a.dag()*psi

a.dag()*basis(N,2) == sqrt(3)*basis(N,3)


I am working on the file "Lab 8 - SHO Solutions.ipynb".
The first task is:
Problem 12.7:
First, define $\hat{X}$ and $\hat{P}$ operators
Can you write Python code for it?

X = 1/2 * (a + a.dag())
P = 1/2j * (a - a.dag())

psi = 1/sqrt(2)*(basis(N,1)+basis(N,2))
ex = psi.dag()*X*psi
exq = psi.dag()*X*X*psi
ep = psi.dag()*P*psi
epq = psi.dag()*P*P*psi

deltaX = sqrt(exq[0][0][0] - ex[0][0][0]**2)
deltaP = sqrt(epq[0][0][0] - ep[0][0][0]**2)

deltaX * deltaP * 2 # compare to uncertainty relation (ΔxΔp >= 1/2)
# the factor of two is to convert from the unitless version of the operator


I am working on the file "Lab 8 - SHO Solutions.ipynb".
The first task is:
Which is also satisfied by the calculated value (1.41 > 0.25)
Problem 12.8:
Can you write Python code for it?

psi = 1/sqrt(2)*(basis(N,2)+basis(N,4))
ex = psi.dag()*X*psi
exq = psi.dag()*X*X*psi
ep = psi.dag()*P*psi
epq = psi.dag()*P*P*psi

deltaX = sqrt(exq[0][0][0] - ex[0][0][0]**2)
deltaP = sqrt(epq[0][0][0] - ep[0][0][0]**2)

deltaX * deltaP * 2 # to compare to book solution which uses the full x and p operators with units


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
Also, the identity is defined as qeye(n) for n dimensions in qutip:
Can you write Python code for it?


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
And the same but for the signal photon:
Can you write Python code for it?

Ph_s = tensor(Ph,qeye(2))  # Ph for signal photon


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
Define two-particle states using the tensor() function:
Mathematically, we are taking the tensor product of two vectors. That product is a larger vector with twice as many entries as the individual state vectors. As long as we take the tensor products in the right order (i.e. always talking about photon 1 and photon 2 in that order) we can also make operators that act on two-photon states). In order to keep a consistent naming scheme, we'll call the first photon the signal photon and the second photon the idler photon. The names aren't particularly important but they come from the process we use in the lab:  Spontaneous Parametric Down Conversion
First, look at a generic pair of vectors and their tensor product:
Can you write Python code for it?

A = Qobj([[1],[2]])
B = Qobj([[3],[4]])
print(A)
print(B)
print(tensor(A,B))


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
Entangled states:
A very interesting system can be set up where there are paired photons being created with unknown but correlated polarization. In this case, we can say the state is in a combination of $|HH\rangle$ and $|VV\rangle$. If either two-photon state is allowed, then the normalized state is $$\big|\phi^+\big\rangle = \frac{1}{\sqrt{2}}\big( \big|HH\big\rangle + \big|VV\big\rangle \big)$$
Can you write Python code for it?

phiPlus = 1/sqrt(2)*(HH + VV)

phiPlus.dag()*Ph_i*phiPlus  # probability of measuring a horizontal idler photon:


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
Example 8.2 prob. of measuring vertical signal and horizontal idler if $|\psi\rangle = |R,+45\rangle$
Can you write Python code for it?

# First, form the prepared state:
psi = tensor(R,P45)

# Then create the projection operator for the state we are asking about:
projection = VH*VH.dag()

# Finally, calculate the probability by computing the bra-ket:
psi.dag()*projection*psi


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
Finally, to really drive this odd point home, show that they are never found in the $\big|+45,-45\big\rangle$ state:
Can you write Python code for it?

# Solution

# Probability that they are in different 45 states:
Pp45m45 = tensor(P45,M45) * tensor(P45,M45).dag()

phiPlus.dag()*Pp45m45*phiPlus


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
Guaranteed to measure a horizontal signal photon whenever a horizontal idler photon is measured. What about vertical? Find the conditional probability of measuring a vertical signal photon if the idler photon is found to be vertical:
Now, measure a different basis (use the +45 states) to show that the photons are always found in the same polarization even when measured at a different angle:
Can you write Python code for it?

# Solution

# Probability that signal is +45 and idler +45
Pp45p45 = tensor(P45,P45) * tensor(P45,P45).dag()
phiPlus.dag()*Pp45p45*phiPlus

# Solution

# Probability that the idler is +45 regardless of the signal
Pp45i = tensor(qeye(2),P45) * tensor(qeye(2),P45).dag()
phiPlus.dag()*Pp45i*phiPlus


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
It can be hard to interpret these values visually but remember it was constructed by multiplying all the terms between two matrices with only diagonal elements. It makes sense that the result is also diagonal. Also, the sign of the diagonal depends on the state of the signal photon (the first one listed). Recall the states are in the order: HH, HV, VH, VV so the first two states have H signal photons and are therefore 1, and the second two states are V signal photons so -1 for those diagonals.
Now construct the two-photon operator that measures the idler photon:
Can you write Python code for it?


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
Next, construct a projection operator that projects the idler photon to H:
Can you write Python code for it?

Ph = H*H.dag()
Ph_i = tensor(qeye(2),Ph)  # Ph for idler photon


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
Now, back to the quantum mechanics. Form the four different combinations of two photons:
Can you write Python code for it?

HH = tensor(H,H)
HV = tensor(H,V)
VH = tensor(V,H)
VV = tensor(V,V)

# How do we represent HH? It is a vector with four elements.
HH


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
Now, find $P(H_s|H_i)$ (Example 8.5)
Can you write Python code for it?

# Projection operator for H idler and H signal:
phh = HH*HH.dag()
phiPlus.dag()*phh*phiPlus

# Projection operator for H idler
Pih = tensor(qeye(2),H*H.dag())
phiPlus.dag()*Pih*phiPlus


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
So we interpret the state $|HH\rangle$ as the vector (1,0,0,0) in a four-dimensional space.
Recall: The polarization measurement operator (for one photon):
Can you write Python code for it?


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
The two-photon operator, measuring the signal photon, is formed with the tensor() function. It is the tensor product of the projection operator Phv and the 2-dimensional identity operator qeye(2). The trick is putting them in the correct order. The first element in the tensor product acts on the signal photon, the second acts on the idler photon. So to act on only the signal photon, we create a tensor product with the projection operator first, and the identity second:
Can you write Python code for it?


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
This is expected, because the HH state has 50% of the probability amplitude. Same for a horizontal signal photon:
Can you write Python code for it?

phiPlus.dag()*Ph_s*phiPlus  # probability of measuring a horizontal signal photon


I am working on the file "Lab 8 - Two-particle systems.ipynb".
The first task is:
You start to see the pattern. Build these up from our earlier operators, just apply them to the specific particle by including them in the tensor product at that position.
Next we will do some example calculations.
Example: find the probability of measuring a horizontal idler photon if the system is prepared in the state $|HH\rangle$
Can you write Python code for it?


I am working on the file "Lab01/ML_lab1_review_student.ipynb".
The first task is:
(1b) Element-wise multiplication: vectors
In this exercise, you will calculate the element-wise multiplication of two vectors by hand and enter the result in the code cell below.  You'll later see that element-wise multiplication is the default method when two NumPy arrays are multiplied together.  Note we won't be performing element-wise multiplication in future labs, but we are introducing it here to distinguish it from other vector operators, and to because it is a common operations in NumPy, as we will discuss in Part (2b).
The element-wise calculation is as follows: $$ \mathbf{x} \odot \mathbf{y} =  \begin{bmatrix} x_1 y_1 \\  x_2 y_2 \\ \vdots \\ x_n y_n \end{bmatrix} $$
Calculate the value of $ \mathbf{z} $: $$ \mathbf{z} = \begin{bmatrix} 1 \\  2 \\ 3 \end{bmatrix} \odot \begin{bmatrix} 4 \\  5 \\ 6 \end{bmatrix} $$
Can you write Python code for it?

# TODO: Replace <FILL IN> with appropriate code
# Manually calculate your answer and represent the vector as a list of integers values.
z = [4, 10, 18]

# TEST Element-wise multiplication: vectors (1b)
Test.assertEqualsHashed(z, '4b5fe28ee2d274d7e0378bf993e28400f66205c2',
                        'incorrect value for vector z')


I am working on the file "Lab01/ML_lab1_review_student.ipynb".
The first task is:
(1d) Matrix multiplication
In this exercise, you will calculate the result of multiplying two matrices together by hand and enter the result in the code cell below.
Below, you'll find the calculation for multiplying two matrices together.  Note that the number of columns for the first matrix and the number of rows for the second matrix have to be equal and are are represented by $ n $:
$$ [\mathbf{X} \mathbf{Y}]{i,j} = \sum{r=1}^n \mathbf{X}{i,r} \mathbf{Y}{r,j} $$
First, you'll calculate the value for $ \mathbf{X} $.
$$ \mathbf{X} = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} \begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{bmatrix} $$
Next, you'll perform an outer product and calculate the value for $ \mathbf{Y} $.  Note that outer product is just a special case of general matrix multiplication and follows the same rules as normal matrix multiplication.
$$ \mathbf{Y} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} \begin{bmatrix} 1 & 2 & 3 \end{bmatrix} $$
Can you write Python code for it?

# TODO: Replace <FILL IN> with appropriate code
# Represent matrices as lists within lists. For example, [[1,2,3], [4,5,6]] represents a matrix with
# two rows and three columns. Use integer values.
X = [[22, 28], [49, 64]]
Y = [[1, 2, 3], [2, 4, 6], [3, 6, 9]]

# TEST Matrix multiplication (1d)
Test.assertEqualsHashed(X, 'c2ada2598d8a499e5dfb66f27a24f444483cba13',
                        'incorrect value for matrix X')
Test.assertEqualsHashed(Y, 'f985daf651531b7d776523836f3068d4c12e4519',
                        'incorrect value for matrix Y')


I am working on the file "Lab01/ML_lab1_review_student.ipynb".
The first task is:
(2c) Matrix math
With NumPy it is very easy to perform matrix math.  You can use np.matrix() to generate a NumPy matrix.  Just pass a two-dimensional ndarray or a list of lists to the function.  You can perform matrix math on NumPy matrices using *.
You can transpose a matrix by calling numpy.matrix.transpose() or by using .T on the matrix object (e.g. myMatrix.T).  Transposing a matrix produces a matrix where the new rows are the columns from the old matrix. For example: $$  \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}^\mathbf{\top} = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix} $$
Inverting a matrix can be done using numpy.linalg.inv().  Note that only square matrices can be inverted, and square matrices are not guaranteed to have an inverse.  If the inverse exists, then multiplying a matrix by its inverse will produce the identity matrix.  $ \scriptsize ( \mathbf{A}^{-1} \mathbf{A} = \mathbf{I_n} ) $  The identity matrix $ \scriptsize \mathbf{I_n} $ has ones along its diagonal and zero elsewhere. $$ \mathbf{I_n} = \begin{bmatrix} 1 & 0 & 0 & \dots & 0 \\ 0 & 1 & 0 & \dots & 0 \\ 0 & 0 & 1 & \dots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \dots & 1 \end{bmatrix} $$
For this exercise, multiply $ \mathbf{A} $ times its transpose $ ( \mathbf{A}^\top ) $ and then calculate the inverse of the result $ (  [ \mathbf{A} \mathbf{A}^\top ]^{-1}  ) $.
Can you write Python code for it?

# TODO: Replace <FILL IN> with appropriate code
from numpy.linalg import inv

A = np.matrix([[1,2,3,4],[5,6,7,8]])
print 'A:\n{0}'.format(A)
# Print A transpose
print '\nA transpose:\n{0}'.format(A.T)

# Multiply A by A transpose
AAt = np.array([[ 30,  70],
        [ 70, 174]])
print '\nAAt:\n{0}'.format(AAt)

# Invert AAt with np.linalg.inv()
AAtInv = np.array([[ 0.54375, -0.21875],
        [-0.21875,  0.09375]])
print '\nAAtInv:\n{0}'.format(AAtInv)

# Show inverse times matrix equals identity
# We round due to numerical precision
print '\nAAtInv * AAt:\n{0}'.format((AAtInv * AAt).round(4))

# TEST Matrix math (2c)
Test.assertTrue(np.all(AAt == np.matrix([[30, 70], [70, 174]])), 'incorrect value for AAt')
Test.assertTrue(np.allclose(AAtInv, np.matrix([[0.54375, -0.21875], [-0.21875, 0.09375]])),
                'incorrect value for AAtInv')


I am working on the file "Lab01/ML_lab1_review_student.ipynb".
The first task is:
(3c) PySpark's DenseVector
PySpark provides a DenseVector class within the module pyspark.mllib.linalg.  DenseVector is used to store arrays of values for use in PySpark.  DenseVector actually stores values in a NumPy array and delegates calculations to that object.  You can create a new DenseVector using DenseVector() and passing in an NumPy array or a Python list.
DenseVector implements several functions.  The only function needed for this course is DenseVector.dot(), which operates just like np.ndarray.dot().
Note that DenseVector stores all values as np.float64, so even if you pass in an NumPy array of integers, the resulting DenseVector will contain floating-point numbers. Also, DenseVector objects exist locally and are not inherently distributed.  DenseVector objects can be used in the distributed setting by either passing functions that contain them to resilient distributed dataset (RDD) transformations or by distributing them directly as RDDs.  You'll learn more about RDDs in the spark tutorial.
For this exercise, create a DenseVector consisting of the values [3.0, 4.0, 5.0] and compute the dot product of this vector with numpyVector.
Can you write Python code for it?

from pyspark.mllib.linalg import DenseVector

# TODO: Replace <FILL IN> with appropriate code
numpyVector = np.array([-3, -4, 5])
print '\nnumpyVector:\n{0}'.format(numpyVector)

# Create a DenseVector consisting of the values [3.0, 4.0, 5.0]
myDenseVector = DenseVector([3.0, 4.0, 5.0])
# Calculate the dot product between the two vectors.
denseDotProduct = myDenseVector.dot(numpyVector)

print 'myDenseVector:\n{0}'.format(myDenseVector)
print '\ndenseDotProduct:\n{0}'.format(denseDotProduct)

# TEST PySpark's DenseVector (3c)
Test.assertTrue(isinstance(myDenseVector, DenseVector), 'myDenseVector is not a DenseVector')
Test.assertTrue(np.allclose(myDenseVector, np.array([3., 4., 5.])),
                'incorrect value for myDenseVector')
Test.assertTrue(np.allclose(denseDotProduct, 0.0), 'incorrect value for denseDotProduct')