I am working on the file "Labs/Lab2/Lab2.ipynb".
The first task is:
With luck this now runs without error, and produces a sin wave with a period of 2*pi, just as it did before we added the multi-wavelength capability.
<div class = hw>
### Exercise 4
-------------------

Add another function to your python module - hellox3. This function should be broadly similar to hellox2, but should take a list of names and say something to each name in the list according to how many letters are in the name. You can get creative (within the bounds of politeness) about what exactly it says.
Can you write Python code for it?

from IPython.core.display import HTML
def css_styling():
    styles = open("../custom.css", "r").read()
    return HTML(styles)
css_styling()


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
1.10 Known issues
Quantities lose their units with some Numpy operations, e.g.:

np.dot
np.hstack
np.vstack
np.where
np.choose
np.vectorize

See Quantity Known Issues for more details.
1.11 Defining new units
You can also define custom units for something that isn't built-in to astropy.
Can you write Python code for it?

# fundamental unit
chuckle = u.def_unit('chuckle')

# compound unit
laugh = u.def_unit('laugh', 4 * chuckle)

(3 * laugh).to(chuckle)

bakers_fortnight = u.def_unit('bakers_fortnight', 13 * u.day)

(3 * bakers_fortnight).to(u.s)


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
1.3 Quantity objects
The most useful feature of units is the ability to attach them to scalars or arrays, creating Quantity objects. The easiest way to create a Quantity object is simply by multiplying the value with its unit.
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
1.6 Combining Quantities
Quantities can also be combined using Python numeric operators:
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
1.8 Decomposing units
Python/astropy will NOT combine units unless you ask it to. For example, the following multiplication will preserve both units, even though they are both lengths.
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
<div class=hw>
### Exercise 1a
------------

The *James Webb Space Telescope (JWST)* will be located at the second Sun-Earth Lagrange (L2) point, which is a special location in space where its orbital period will be equal to that of the Earth's, despite its greater distance from the Sun, so if it stays at this point, it will remain at a constant distance from Earth as both orbit the sun (see picture).

![](Lagrange_points.jpg)

L2 is located opposite the Sun at a distance from the Earth of approximately:

$$ r \approx R \left(\frac{M_{earth}}{3 M_{sun}}\right) ^{(1/3)} $$

where $R$ is the Sun-Earth distance.

Calculate the Earth-L2 distance in kilometers and miles:

* *Hint*:  the mile unit is defined as ``u.imperial.mile`` (see [imperial units](http://docs.astropy.org/en/v0.2.1/units/index.html#module-astropy.units.imperial))
Can you write Python code for it?

# answer here (km)

# answer here (mile)


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
<div class=hw>
### Exercise 2
---------------------

Define units equal to the following (which you should look up or justify an estimate for)
a) the circumference of the earth
b) the diameter of the Milky Way galaxy
c) the distance across the US
d) the distance between two people holding hands
e) the mass of an elephant

Then, convert the following physical quantities into these units

a) the distance to the moon in earth circumferences
b) 1 Mpc (Mega parsec) to Milky Way Diameters
c) the distance across the US to human armspans (how many people holding hands could fit across the width of the US?)
d) the mass of Jupiter in elephants

## 2. Typesetting with LaTeX

Jupyter markdown cells also understand a commonly-used typesetting language called LaTeX. LaTeX is used to compose most scientific publications. In fact, most scientific journals have custom "style" files that contain all of the rules about how to format documents for their publication (e.g. font size, font type, margins, etc.). We won't be using a standalone LaTeX editor in this class, but we will use LaTeX "math mode" syntax to insert equations into markdown cells as needed. LaTeX formats equations beautifully, and if you know this you'll never have to use Microsoft Word's equation editor again!

The interpreter that reads Jupyter markdown cells is so-called "MathJax aware", which means it understands a subset of common LaTeX commands called MathJax.

### 2.1 Equations

Technically speaking, you shouldn't have to surround MathJax syntax with \$ symbols, but in practice, you should, particularly if you have an equation embedded in a long block of text. The \$ symbols tell the markdown cell explicitly that you want it to typeset an equation or symbol.

For example, typing \$ F = m \times a \$ in a markdown cell results in a nicely typeset $ F = m \times a $.

### 2.2 Special Symbols

The \\times is an example of a command telling MathJax to insert a certain symbol. A list of common LaTeX symbols is...
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
<div class=sidebar>
Equally useful and operating based on the same basic principle, there are a number of functions that can be exercised on a variable using variablename.function. To call up a list of these, place your cursor after the x. and hit just tab.
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
<div class=sidebar>
Here are some particularly useful ones for array statistics - minimum and maximum values, standard deviation, and mean.
Can you write Python code for it?

x.min() #minimum array value

x.max() #maximum array value

x.std() #standard deviation

x.mean()


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
<div class=sidebar>
If you forget what the specific attribute is or want some hints as to what attributes are available, you can use another jupyter help method that we haven't mentioned yet - the shift + tab method. Basically, you can hit shift + tab, shift + tab + tab at any point while typing a python command in a jupyter code cell, and it will pull up increasing levels of help according to how many tabs you've hit. For example, in the cell below, put your cursor after the x. and hit shift+tab, then try shift + tab + tab, etc. shift + tab will give you just some basic information about the variable, and shift + tab + tab will give you additional information, like a list of possible attributes that you can call
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
<div class=sidebar>
many additional useful functions are built into numpy, and are called in the usual way, for example
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
A completely equivalent (but more verbose) way of doing the same thing is to use the Quantity object's initializer, demonstrated below.  In general, the simpler form (above) is preferred, as it is closer to how such a quantity would actually be written in text.  The initalizer form has more options, though, which you can learn about from the astropy reference documentation on Quantity.
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
Addition and subtraction require compatible unit types. The cells below provide some examples. Note that the units don't have to be the same they only have to be compatible (i.e. both lengths or both times)
Can you write Python code for it?

q1 = 3 * u.m
q1 + (5 * u.m)

q1 + (5. * u.kpc)

# this will fail because the units are not compatible
q1 + (10. * u.km / u.s)


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
Also convenient if you forget what cgs vs. si base units are is the bases attribute
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
Care needs to be taken with dimensionless units.  Passing dimensionless values to an inverse trigonometric function gives a result without units:
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
Here are some astronomy specific units that we will be using often
Can you write Python code for it?

u.pc.__doc__

u.pc.physical_type

u.arcsec.__doc__

u.arcsec.physical_type

u.au.__doc__

u.au.physical_type


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
However, if you explicitly tell it to, the units module can decompose a quantity into a set of base units using the
decompose() method. By default, units will be decomposed to S.I. units
Can you write Python code for it?

q = 3. * u.cm * u.pc / u.g / u.year**2
q


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
However, note that converting the constant alone results in just the scale factor between the two and no units
Can you write Python code for it?

(u.Msun).to(u.kg)

# to keep the units, use a Quantity (value and unit)
(1. * u.Msun).to(u.kg)


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
If you want units other than the SI/ CGS standard, you can then convert the output using .to, as before
An example
Kepler's law can be used to estimate the (circular) orbital speed of the Earth around the sun using:
$$v = \sqrt{\frac{G M_{\odot}}{r}}$$
Can you write Python code for it?

v = np.sqrt(G * 1 * u.M_sun / (1 * u.au))
v

v.decompose()

v.to(u.km / u.s)


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
Please see the complete list of available units.
1.2 Composite units
Composite units are created using Python numeric operators, e.g. "*" (multiplication), "/" (division), and "**" (power).
Can you write Python code for it?

u.km / u.s

u.imperial.mile / u.h

(u.eV * u.Mpc) / u.Gyr

u.cm**3

u.m / u.kg / u.s**2


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
There are a few subtelties to note here. First, both of the following methods for importing astropy.constants are perfectly legitimate:
1) from astropy import constants (or from astropy import constants as cst)
    Here, you would import all of the constant definitions in the module, but would have to refer to them with the constants. (or cst.) prefix. For example, G would now be constants.G.
2) from astropy.constants import *
    Here the * means "all", so this command will import all of the constants defintions from the module. The danger here is that you will overwrite any assigned variables with those names (e.g. "c") with the constants. Generally speaking, it's best to use the method shown and only import the constants that you need OR use 1) above.
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
Units have docstrings defining them. You can call up the docstring, data type and other ancillary information  with ? as we've done before and it will pop up a separate frame at the bottom of the window that you close when done with it.
If you explicitly want to print the docstring to output inside the notebook (which can be quite useful so that you can use it later), you can do that with _doc_ for the docstring. The units functions in astropy have an additional property called physical type that will tell you what KIND of measurement they are (length, area, time, etc.). You call up this property with physical_type, as below.
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
cm and pc are both units of length, so will be combined. The units provided will all also be converted to their SI equiavelents (length to m, mass to g, time to seconds)
Can you write Python code for it?


I am working on the file "Labs/Lab3/Lab3.ipynb".
The first task is:
u.dimensionless_unscaled creates a Quantity with a "dimensionless unit" and therefore gives a result with units:
Can you write Python code for it?

np.arcsin(1.0 * u.dimensionless_unscaled)

np.arcsin(1.0 * u.dimensionless_unscaled).to(u.degree)


I am working on the file "Labs/Lab4/Lab4.ipynb".
The first task is:
2.2 Concatenating strings
Can you write Python code for it?

# note the ; allows us to do two calculations on the same line
s = "spam" ; e = "eggs"
print(s + e)

print(s + " and " + e)

# this one won't work
print('I want' + 3 + ' eggs and no ' + s)

# but this will
print('I want ' + str(3) + ' eggs and no ' + s)


I am working on the file "Labs/Lab4/Lab4.ipynb".
The first task is:
2.4 Comparing Strings
Can you write Python code for it?

print("spam" == "good"); print("spam" == "spam")

"spam" < "zoo"

"s" < "spam"


I am working on the file "Labs/Lab4/Lab4.ipynb".
The first task is:
3.2 While and Iteration
Can you write Python code for it?

x = 1
y = 0

while y < 10:
    print("yo" if x > 0 else "dude")
    x *= -1
    y += 1


I am working on the file "Labs/Lab4/Lab4.ipynb".
The first task is:
4. Writing and Reading Files with Magic Commands
Can you write Python code for it?

%%file number_game.py
# The above "magic" command, denoted with the double %% saves the contents
# of the current cell to file.  We'll see more of these later

x = 0
max_tries = 10
count = 0

while True:
    x_new = int(input("Enter a new number: "))
    if x_new > x:
        print(" -> it's bigger than the last!")
    elif x_new < x:
        print(" -> it's smaller than the last!")
    else:
        print(" -> no change!  I'll exit now")
        break

    x = x_new

    count += 1
    if count > max_tries:
        print("too many tries...")
        break

%run number_game.py
# this magic command runs the given file. It's like typing python number_game.py in the command line


I am working on the file "Labs/Lab4/Lab4.ipynb".
The first task is:
<div class=hw>
## Exercise 2
------------------

Write a function that does the following:
    1) Takes a required input ***string array*** containing the names of the people in your group.
    2) Asks the person with the longer name for their order first
    3) Asks each person to enter (using user input) how many eggs they want, and then whether or not they want spam
    4) prints the order in the form "*name* wants *x* eggs and *[spam/no spam]*" with a new line between the two orders
    5) prints "name:" + "egg" a number of times equal to the number of eggs ordered + "and" + "SPAM!" or "NO SPAM!"

## 3. More about Conditionals

### 3.1 One-Line Conditionals
Can you write Python code for it?

x = 1

if x > 0:
    print("yo")
else:
    print("dude")

# the one line version
"yo" if x > 0 else "dude"

# conditionals can lie within function calls
print("yo" if x > 0 else "dude")

z = "no"

np.sin(np.pi if z=="yes" else 0)


I am working on the file "Labs/Lab4/Lab4.ipynb".
The first task is:
<div class=hw>
### Exercise 3
-------------------

Copy the function above into a code cell and add a print statement that prints the variables x and y so that you can see what they are at each step. Use the output to figure out what the statements x \*= -1 and y += 1 are doing. Type an explanation of what they do into a markdown cell

### 3.3 Break Statements

Break statements allow you to terminate/exit a loop, as in the following example:
Can you write Python code for it?

#can also do this with a break statement
while True:
    print("yo" if x > 0 else "dude")
    x *= -1
    y += 1
    if y >= 10:
        break


I am working on the file "Labs/Lab4/Lab4.ipynb".
The first task is:

### Exercise 4
-------------------

Create a program (and write it to a .py file) which repeatedly asks the user for a word. The program should append all the words together. When the user types a "!", "?", or a ".", the program should print the resulting sentence and exit.
For example, a session might look like this:

%run exercise2.py
Enter a word (. ! or ? to end): My
Enter a word (. ! or ? to end): name
Enter a word (. ! or ? to end): is
Enter a word (. ! or ? to end): Walter
Enter a word (. ! or ? to end): White
Enter a word (. ! or ? to end): !

My name is Walter White!

### Exercise 5 (a classic)
--------------------

Write a program (and write it to a python file) that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz” instead of the number. A special python operator called modulo and represented with the % sign might help you. A couple of examples of its use are below. See if you can figure out what it does by experimenting.
Can you write Python code for it?

4 % 2

4 % 3

6.28 % 3.14

6.28 % 3.1

25 % 5

25 % 7

from IPython.core.display import HTML
def css_styling():
    styles = open("../custom.css", "r").read()
    return HTML(styles)
css_styling()


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
1. Python Return Statements
1.1 Functions Without Returns
Thus far we've not paid much attention to function return statements. As many of you have noticed, you don't HAVE to have a return statement at the end of a function. For example:
Can you write Python code for it?

def dummy_func(x=0, y=0):
    """This function takes two optional arguments (x and y) with default values of zero and adds them together
    to create a new variable z, then prints it."""
    z=x+y
    z

dummy_func()


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
<div class=hw>
### Exercise 2
---------------------

Make a plot that meets the following criteria:
* plots $x^a$ for all integer values of a between 0 and 5
* labels the x and y axes appropriately
* includes an appropriate descriptive title
* zooms in on a region of the plot where you find the differences between the functions to be the most interesting
* sets linestyles and colors to be distinct for each function
* makes a legend without a bounding box at an appropriate location
Can you write Python code for it?


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:

### Exercise 3
------------

### Exercise 3a
Use LaTeX syntax to write Planck's law (provided in class) in the cell below

***Insert Planck's Law here***

#### Exercise 3b
Now, code a function with one required input (temperature) that will return a numpy array called Planck with two columns and 100000 rows. The first column should contain wavelength values between 0 and 100000 nanometers, and the second column should contain the corresponding $B_\lambda$ values. The output flux column should be in units of $W/m^2/nm$. The cell below will import all of the constants that you need to accomplish this.
Can you write Python code for it?

import astropy.units as u
from astropy.constants import G, h, c, k_B

#the cell below should include your planck_func function definition

# insert test statements here


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:

#### Exercise 3f

Write a paragraph describing what normalization does and why it is useful. What physical properties are corrupted/lost by displaying the normalized flux instead of the true flux, and which are preserved? Include specific descriptions between your plots.

***insert normalization paragraph***

#### Exercise 3g

Write a paragraph describing the differences in peak wavelength between these three stars. You should include a precise value for the peak wavelength of each star (*Hint: find the **index** of the maximum flux value and use that to get the corresponding wavelength*).
Can you write Python code for it?

#you should insert some code here to find the peak wavelength for each star


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
<div class=hw>
Make the same plot as you did above, but with normalized flux along the y axis this time.
Can you write Python code for it?

#define your variables to be plotted (outputs from the function you defined) here

#insert your plot commands here


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
And when you want to assign them to variables, you use a similar syntax, though it seems a bit funny.
Can you write Python code for it?


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
As you can see above, the default for a legend is to place it at the upper right of the plot, even when it obscures the underlying lines and to draw a solid line around it (bounding box). Generally speaking, bounding boxes are rather ugly, so you should nearly always (unless you really want to set the legend apart) use the optional Boolean (True or False) keyword "frameon" to turn this off. Legend also takes the optional keyword loc to set the location of the legend. Loc should be a string, and you can see the full list of options by accessing help for the legend function by hitting shift+tab+tab inside of the plt.legend parentheses below.
Can you write Python code for it?

plt.plot(x,x**2, label='$x^2$')
plt.plot(x,x**3, label='$x^3$')
plt.legend(loc="lower right", frameon=False)


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
As you have already encountered in your labs and homeworks, it is often useful to overplot functions on top of one another, which we do with multiple plot commands. In this case, what you need to make a quality graphic is a legend to label which is which. For example, let's plot the cubic function on top of the quadratic. In doing so below, note that we don't actually have to specify a separate variable, but that the arguments to a plot command can be combinations of variables
Can you write Python code for it?


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
Here are some especially useful pyplot functions, called with plt.functionname(input(s)):

xlim and ylim set the range of the x and y axes, respectively and they have two required inputs - a minimum and maximum for the range. By default, pylab will set axis ranges to encompass all of the values in the x and y arrays that you specified in the plot call, but there are many cases where you might want to "zoom in" on certain regions of the plot.
xlabel and ylabel set the labels for the x and y axes, respectively and have a required string input.
title sets the title of the plot and also requires a string input.

Line properties are controlled with optional keywords to the plot function, namely the commands color, linestyle and linewidth. The first two have required string arguments (lists of the options are available here), and the third (linewidth) requires a numerical argument in multiples of the default linewidth (1).
These can be specified either in the call to the function or separately before or after the plot call.
See the cell below for an example of all of these at play
Can you write Python code for it?

plt.xlim(-5,5)
plt.ylim(0,20)
plt.xlabel("the independent variable (no units)")
plt.ylabel("the dependent variable (no units)")
plt.title("The Quadratic Function")
plt.plot(x, y, color='red',linestyle='--', linewidth=2.5)


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
Like the return statement in the temperature conversion function, break is used here to handle unrecognized input, but here it functions only to end the for loop and not to exit the function. It will still reach the two print statements at the bottom and print any orders that were collected before the bad input. To verify this, exectute the cell below. For Rowan's order, enter 2 for eggs and no for spam. For Tasha's order, enter 3 eggs and yuck for spam. At the end, the function will still print Rowan's order, and skip Tasha's.
Can you write Python code for it?


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
Note that when you define a function with multiple return variables and you assign those returned variables into stored variables with = you must have an equal number of assigned variables as returned variables. For example, the following will return an error.
Can you write Python code for it?


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
Note these functions can be set before or after the plot command as long as they're within the same cell.
Can you write Python code for it?

plt.plot(x, y, color='red',linestyle='--', linewidth=2.5)
plt.xlim(-5,5)
plt.ylim(0,20)
plt.xlabel("the independent variable (no units)")
plt.ylabel("the dependent variable (no units)")
plt.title("The Quadratic Function")


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
Note though that break will break you out of the for loop entirely. If you enter an unrecognized answer for spam in Rowan's order, it will not then ask Tasha for hers. Verify this by reexecuting the cell above and entering a bad value for spam in Rowan's order.
Note too that entering return in place of break in the function above would have stopped the code entirely at that point, and would not have printed the good order before exiting. Try swapping it out in the function definition to verify this.
1.3 Return Statements for Assigned Output
Return statements are not only useful as code breaks, however. Their main purpose is to return calculated values, arrays, etc. to the user so that they can be referenced in future code cells. So far our code has mostly involved printing things, but sometimes we want to use and manupulate the output of a function and so it needs to be passed back to the user. For example:
Can you write Python code for it?

def dummy_func(x=0, y=0):
    """This function takes two optional arguments (x and y) with default values of zero and adds them together
    to create a new variable z, then prints it."""
    print("x is", x)
    print("y is", y)
    z=x+y
    print("z is", z)
    return z

dummy_func()


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
OK back to the function dummy_func and the idea of return statements. Note that we did not include a return statement in the definition cell. A return statement is not necessary to delimit the end of a function, though it can be visually useful in that way.
In fact, a function will consist of any code that is indented beneath it. If you remove the indentation, any code that you write is no longer considered part of the function. For example:
Can you write Python code for it?

def dummy_func(x=0, y=0):
    """This function takes two optional arguments (x and y) with default values of zero and adds them together
    to create a new variable z, then prints it."""
    print("x is", x)
    print("y is", y)
    z=x+y
    print("z is", z)

print('This is not part of the function and will not be printed when I call it. It will print when I execute this cell')

dummy_func()


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
There are a  few things to note here. First that the function did not print out z, even though in an ordinary notebook cell (without a function definition) typing the variable name alone usually prints the value stored in it. That's not the case with functions because functions will only return to output what you tell them to. If you want to print the value of something within a function, then you must use a print statement. For example:
Can you write Python code for it?

def dummy_func(x=0, y=0):
    """This function takes two optional arguments (x and y) with default values of zero and adds them together
    to create a new variable z, then prints it."""
    print("x is", x)
    print("y is", y)
    z=x+y
    print("z is", z)

dummy_func(2,3)


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
This brings up another subtelty that it's worth reemphasizing here - optional arguments can be specified in a function call in any order but if you don't specify which variable is which in the function call, it will assume that you are specifying them in the order that you defined them in the function. For example, as I've written the function, you should be able to specify none, one or two arguments in any order.
Make sure you understand what each of the following cells are doing before moving on
Can you write Python code for it?

dummy_func(1)

dummy_func(y=1)

dummy_func(y=3,x=2)

dummy_func(3,2)


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
This gives you access to all of pyplot's functions in the usual way of calling modules (plt.functionname). For example:
Can you write Python code for it?


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
This is an important higher-order thing to note about functions - variables only have meaning within them, not outside them. If I want the function to return whatever I've told it to return (z) and store it in a variable, I have to tell it so via the following syntax:
Can you write Python code for it?


I am working on the file "Labs/Lab5/Lab5.ipynb".
The first task is:
This time, I've told python that it should take the output of dummy_func and store it in the variable z.
Functions can return arbitrary numbers of things as well
Can you write Python code for it?

def dummy_func(x=0, y=0):
    """This function takes two optional arguments (x and y) with default values of zero and adds them together
    to create a new variable z, then prints it."""
    print("x is", x)
    print("y is", y)
    z=x+y
    print("z is", z)
    return x, y, z

dummy_func()


I am working on the file "Labs/Lab5/lab5_assignment.ipynb".
The first task is:
The data in column index 2 represents proportion of non-retail business acres in the town, while column index 12 is the median value of owner-occupied homes in the neighbourhood, priced in $1000's.
The target variable (i.e. what we want to predict) are the house prices, these are stored in bost.target. Let's plot house prices as a function of our inputs to see what we're dealing with.
Can you write Python code for it?

from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt

fig = plt.figure(figsize=(10,10))
for i in range(1, 10):
    ax = fig.add_subplot(3, 3, i, projection='3d')
    ax.view_init(elev=10., azim = (360.0 / 9.) * (i - 1))
    ax.scatter(xs[:,0], xs[:,1], bost.target)

plt.show()


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
.sort() changes the the list in place
Can you write Python code for it?

v.index(4)   ## lookup the index of the entry 4

v.index(3)

v.count(3)

v.insert(0, "it's full of stars")
v

v.remove(1)
v


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
1. Tuples
1.1 Defining Tuples
Tuples are denoted with round parentheses
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
1.1 Using Glob
glob is an extremely useful function in Python. First, move into your data directory using cd, then ls to see the file list:
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
1.2 Indexing Tuples
Indexing works the same way as for arrays:
Can you write Python code for it?

t[0]

t[-1]

t[-2:]  # get the last two elements, return as a tuple


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
1.2 Iterating to View Header Info
We now would like to take all of our FITS files and sort them into subfolders based on the type of image, e.g., calibration, science, standard star, etc. Helpfully, the file names include some clues about what sort of files we have. However, if you remember from our observing run and log sheets, not all of the file names are always correct. This can happen quite easily at the telescope, because camera software programs typically have various automated ways of naming files (often different from observatory to observatory) and sometimes require the user to remember to change settings while observing.
(To avoid this, some observatories name their files in a uniform way, like "Image_00001.fits" or with a time/date stamp, like "NACO_2017-10-05T00:04:18.fits", which provide unique identifiers.)
Therefore, to check our data and sort it accurately, we'll look at the header information first to get a sense of the type of files we're dealing with. There are a couple different ways we can look at FITS headers; in Lab 4, we used fits.open, which is very versatile. Here, we'll use two new functions to quickly view the header and data:
fits.getheader('filename')
and
fits.getdata('filename')
Can you write Python code for it?

# Complete this cell to save the header of the zeroth (that is, the first) FITS file in the all_fits list:

test_header =

# Now, view the header info here:


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
1.3 The Sorting Function
Because data taken in different filters and with different exposure times require matching calibration files, our goal is to create the following directory structure:

We want a generic function that:
(1) takes as input:
 the name of a single FITS file,
 the desired folder name,
 the desired type of FITS file,
 the header keyword to match the FITS file type
(2) reads in the file's header information,
(3) reads in the file's type based on the keyword
(4) checks if the desired folder name exists, and makes a new directory if it doesn't
(5) checks if the file's type matches the desired type of fitsfile
(6) moves the file into the new or existing folder, if the file matches the right file type.

We've provided a function that does all of these steps, but lacks a docstring. You'll be using this function multiple times, so discuss within your group exactly how the function works, then update the docstring and add comments within the function (using #).
Can you write Python code for it?

def filesorter(filename, foldername, fitskeyword_to_check, keyword):
    '''
    Edit this docstring to describe the function purpose and use.
    '''
    if os.path.exists(filename):
        pass
    else:
        print(filename + " does not exist or has already been moved.")
        return

    header = fits.getheader(filename)
    fits_type = header[keyword]

    if os.path.exists(foldername):
        pass
    else:
        print("Making new directory: " + foldername)
        os.mkdir(foldername)

    if fits_type == fitskeyword_to_check:
        destination = foldername + '/'
        print("Moving " + filename + " to: ./" + destination + filename)
        os.rename(filename, destination + filename)
    return


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
1.3 Tuple Modification
Tuples cannot be modified. The following cell will spit out an error.
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
2.2 Dark Frames
Bias subtracting the darks:
We want to median combine our darks, but contribution from bias is present in every image taken, so our first step after creating a master bias frame is always to subtract it from every other image. Array subtraction is more straightforward -- as long as two arrays have the same dimension, they can be subtracted from one another on a pixel-by-pixel basis.
Below, write a generalized function below that subtracts Master_Bias.fit from a single frame. Your function should:
1) take a FITS file name and path to the master bias as inputs,
2) load in the data for the file to be calibrated,
3) loads in the header information for that file,
4) loads in the data for Master_Bias.fit as a variable (remember, it's located in a different folder than where you are now! Hence the second input),
5) subtract the bias from the input FITS file, and
6) save (writeto) the new bias-subtracted FITS file with a modified name (e.g., cal-001_dark60.fit would become b_cal-001_dark60.fit, for bias-subtraction).
Once you've written and tested your function, you will apply it to all of the dark frames.
The first step is to move into the darks directory from your current location and glob the dark files.
Can you write Python code for it?

cd ../darks

darkfiles = glob.glob('*fit')

# Write your bias subtraction function here:

def bias_subtract(filename, path_to_bias):
    '''
    Add your docstring here.
    '''
    # Your code goes here.
    fits.writeto('b_' + filename, ) # finish this code too to save the FITS
    return

# Test out your function on an individual frame (remember, we defined "master_bias_path" just before Section 2.2:

bias_subtract('cal-001_dark60.fit', master_bias_path)

# Did it work? You can test whether the bias subtraction worked by viewing the before/after frames in DS9.

# Now write and execute a for loop that subtracts the bias from each of the dark frames.


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
2.4 Making a Master Flat Field
The final step in creating master calibrations is to make a master flat field. Before we median combine into a single image, we want to divide each individual flat by its median pixel value, such that all the pixel values in each frame have values of approximately ~1.0 -- that is, we want to normalize them. Only then do we median combine the stack of normalized flat fields to create a master flat.
We can do this in a single function by editing the mediancombine function from earlier, and simply adding a single new line of code.
In line 11 below, add this extra line of code that normalizes im before adding it to the 3D array of fits_stack:
Can you write Python code for it?

def norm_combine_flats(filelist):
    '''
    Edit this docstring accordingly!
    '''
    n = len(filelist)
    first_frame_data = fits.getdata(filelist[0])
    imsize_y, imsize_x = first_frame_data.shape
    fits_stack = np.zeros((imsize_y, imsize_x , n), dtype = np.float32)
    for ii in range(0, n):
        im = fits.getdata(filelist[ii])
        norm_im =  # finish new line here to normalize flats
        fits_stack[:,:,ii] = norm_im
    med_frame = np.median(fits_stack, axis=2)
    return med_frame


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
2.4 Searching, Sorting, and Counting Lists
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
2.5 Exploring List Methods
Jupyter is your new best friend: it's tab-completion allows you to
explore all methods available to an object.  (This only works in jupyter, not in the command line)
Type
v.
and then the tab key to see all the available methods:
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
4. Dictionaries
4.1 Defining Dictionaries
Dictionaries are one-to-one mappings of objects. They are often useful when you want to assign multiple named properties to individuals. Each entry in a dictionary has a set of "keys" that can be assigned unique values.
We'll show four ways to make a Dictionary
Can you write Python code for it?

# number 1... curly braces & colons
d = {"favorite cat": None,
     "favorite spam": "all"}
d

# number 2
d = dict(one = 1, two=2, cat='dog')
d

# number 3 ... just start filling in items/keys
d = {}  # empty dictionary
d['cat'] = 'dog'
d['one'] = 1
d['two'] = 2
d

# number 4... start with a list of tuples and then use the dict function to create a dictionary with them
mylist = [("cat","dog"), ("one",1), ("two",2)]
dict(mylist)

dict(mylist) == d


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
<div class=hw>
### Exercise 1
----------

Write a loop over the words in this list and print the words longer than three characters in length:
Can you write Python code for it?

L = ["Oh", "Say", "does", "that", "star",
     "spangled", "banner", "yet", "wave"]


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
<div class=hw>
### Exercise 2
--------------

Below is a list of information on 50 of the largest near-earth asteroids.

(a) Given this list of asteroid information, find and list all asteroids with semi-major axis (a) within 0.2AU of earth, and with eccentricities (e) less than 0.5.

(b) Note that the object below is a list (denoted with square brackets) of tuples (denoted with round brackets), and that the orbit class object is a dictionary. Create a dictionary where the name of each asteroid is the key, and the object stored under that key is a three element tuple (semi-major axis (AU), eccentricity, orbit class).

(c) using the list (and not the dictionary), print the list of asteroids according to:
    (i) alphabetical by asteroid name
    (ii) in order of increasing semi-major axis
    (iii) in order of increasing eccentricity
    (iv) alphabetically by class (two-stage sorting)

    hint: use the "sorted" function rather than object.sort, and check out the function "itemgetter" from the python module "operator"
    Bonus points if you can get it to print with the columns lined up nicely!
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
As discussed in your reading and in lecture, what we ultimately want is a median combination of all of the individual bias frames. So we will create a stack of bias images, and then we will take the median values at each pixel location in the stack to create the final combined frame.
Can you write Python code for it?

# Complete: Define a new variable, n, and determine how many bias files there are (length of biasfiles):
n =

# And use fits.getdata to read in the data for only the first bias frame (the zeroth element of the bias list):
first_bias_data =


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Dark subtract the flat fields:
Now, we will want to subtract the dark contribution from the flat fields. This can be accomplished by creating a new function below, dark_subtract, that looks very much like your bias_subtract function.

Most importantly, make sure that the dark you subtract matches the exposure time of the flat fields!

Check the flat exposure times in the cell below. What is/are the value(s)?
Answer:
Can you write Python code for it?

# Check the flat field exposure times here for the files in your directory:


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Dictionaries can contain dictionaries!
Can you write Python code for it?

d = {'favorites': {'cat': None, 'spam': 'all'},\
     'least favorite': {'cat': 'all', 'spam': None}}
d['least favorite']['cat']


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
How many bias files are there?
Answer:
What are the dimensions of the first bias frame?
Answer:
Creating a blank 3-D array and adding images to it
In order to take the median value of the stacked bias frames, we'll need to insert them into a larger array first. We can do this by creating a "blank" 3-D array filled with zero values with dimensions of (y dimension, x dimension, number of images):
Can you write Python code for it?

# Create blank stack of arrays to hold each of the frames
# **IMPORTANT** Note that Y is listed first! This is a pecularity in how python reads arrays:

biasarray = np.zeros((imsize_y, imsize_x , n), dtype = np.float32)

# Now check the shape of our new "blank" array:
biasarray.shape

# In this cell, check what the values in the biasarray look like now:


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
In practice, this is equivalent to the python arange command that you've already seen, but note that arange creates a numpy array with all of the elements between the start and stop point, and is therefore more efficient for large loops. Still, it's usefult to be aware of range as well. arange can also be used on non-integers, which is quite useful. Note that the second cell below will result in an error.
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
In the cell below, use the mediancombine function from earlier to combine all of the bias-subtracted dark frames into a single master dark. We will call it Master_Dark_60s.fit , since dark frames need to match exposure times.
Be careful using glob to select only the darks that have been bias-subtracted!
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
In the cell below, use this method to determine the dimensions of the first bias image and define them as new variables:
Can you write Python code for it?

# Complete to get the dimensions of first_bias_data, then check the values

imsize_y, imsize_x =
imsize_y, imsize_x


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Like strings, you can also "multiply" tuples to duplicate elements
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Lists can contain multiple data types, including tuples and other lists
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Note the above did something, but not exactly what you might think. It created a three element tuple, where the first (index 0) and third (index 2) elements are themselves tuples
Can you write Python code for it?

len(newt), type(newt[0]), type(newt[1]), type(newt[2])


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Of course, our FITS images are 2-D, and we will be working with arrays of various dimensions -- not all FITS images are the same size (e.g. 1024 x 1024). In numpy, we can define arrays of various sizes as follows. Execute the cell, and Jupyter will print the formatted version:
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
One last note on the master bias -- we will need to determine the path to the Master_Bias.fit file, because we will use functions that call it from different folders. We can do this using os.getcwd (get current working directory) and adding a string with the filename, as follows:
Can you write Python code for it?

master_bias_path = os.getcwd() + '/Master_Bias.fit'
master_bias_path


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Similarly, we could have done this without assigning a the new variable, but note that this erases memory if the original.
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Taking the median and saving the master bias:
Now the final steps to create a master bias frame are to:
(1) take the median of the 3-D array, along the appropriate axis, which will collapse the image to a regular two dimensional array,
and
(2) save this new 2-D array -- the master bias -- as a brand-new fits file with a new name, giving it the same header as the the first bias image for simplicity.
Can you write Python code for it?

# Take the median of the 3-D stack:
med_bias = np.median(biasarray, axis = 2) # axis=2 means stack along the *third* axis, since python is zero-indexed

# And get the header for the first image in the list:
biasheader = fits.getheader(biasfiles[0])

# Define a name for the new output FITS file:
master_bias = 'Master_Bias.fit'

# Save your new FITS file!
fits.writeto(master_bias, med_bias, biasheader, clobber=True)


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
The set function will make a set out of whatever is provided.
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
The uniqueness aspect is the key here. Note that the output of the cell below is the same as the one above.
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
This can have its uses, but more often you will want to create a tuple identical to the original but with different elements, for which we use concatenation instead, just like we did with strings.
But concatenation is tricky. What's wrong with the following statement?
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
This is probably the most important thing you'll learn today
2.6  Iterating Over Lists
Can you write Python code for it?

a = ['cat', 'window', 'defenestrate']
for x in a:
    print(x, len(x))

#enumerate is a useful command that returns ordered pairs of the form
#(index, array element) for all of the elements in a list
for i,x in enumerate(a):
    print(i, x, len(x))

# print all the elements in the list with spaces between
for x in a:
    print(x, end=' ')


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Tuples are most commonly used in functions that return multiple arguments.
2. Lists
2.1 Defining Lists
Python lists are denoted with square brackets. We've dealt with them indirectly a bit already in this class, but it's worth discussing them explicitly here.
Can you write Python code for it?

v = [1,2,3]
print(len(v))
print(type(v))


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Typically you would have to scale the 60s master dark to different exposure times, but to save you a bit of effort, we've scaled them for you ahead of time. Both 1s and 10s master dark frames can be found in the "ExtraFiles" folder in the top level directory. Copy these files into your 'darks' folder.
When you save your dark-subtracted FITS file, be sure to add another prefix to the file name, and it's important to only dark subtract the bias-subtracted images. For example, this would change 'b_cal-001_Vflat.fit' to 'db_cal-001_Vflat.fit'.
Can you write Python code for it?

# Copy other master darks to directory in the following cell:

# Write your dark subtraction function here:

def dark_subtract(filename, path_to_dark):
    '''
    Add your docstring here.
    '''

    # Your code goes here.

    return

# Now dark subtract the bias-subtracted flat fields:

# Did that work?

ls


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
Unlike tuples, however, lists are mutable.
Can you write Python code for it?

v[0] ="green egg"
v[1] += ",love it." # this takes what's already in v[1] and adds what comes after +=
v


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
We can also do this with a list comprehension:
Can you write Python code for it?

L = [num for num in range(100) if (num % 7 == 0) or (num % 11 == 0)]
print(L)

# Can also operate on each element:
L = [2 * num for num in range(100) if (num % 7 == 0) or (num % 11 == 0)]
print(L)


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
We can also use our new variable to view specific header keyword values, such as the image type, or object:
("IMAGETYP" and "OBJECT", since FITS files can only have 8 character keywords)
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
We can make an image stack of bias frames by adding the data from each of the FITS files into the "blank" stack, one by one:
Can you write Python code for it?

# Insert each bias frame into a three dimensional stack, one by one:

for ii in range(0, n):
    im = fits.getdata(biasfiles[ii])
    biasarray[:,:,ii] = im

# How do the biasarray values look now?
biasarray


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
What does glob do? What kind of Python object is a_few_files?
Answer:

Now use glob to create a new variable, all_fits, that contains the names of all of the FITS files in your directory. We will use this variable a number of times.
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
What inputs does the fits.writeto function require to save a new FITS files?
Answer:
The final step is to check the final master bias frame to see if it appears normal. In DS9, open up a single raw calibration bias frame as well as the the new Master_Bias.fit that you just created.
How do the two compare? Do the pixel values seem reasonable? Do the dimensions of the image make sense?
Answer:
Median Combination Function
For ease of use, let's write all of those proceeding steps into a single function that we can re-use later. Edit the docstring below and add any comments on how to use the function that future you will find helpful.
Can you write Python code for it?

def mediancombine(filelist):
    '''
    Edit this docstring accordingly!
    '''
    n = len(filelist)
    first_frame_data = fits.getdata(filelist[0])
    imsize_y, imsize_x = first_frame_data.shape
    fits_stack = np.zeros((imsize_y, imsize_x , n), dtype = np.float32)
    for ii in range(0, n):
        im = fits.getdata(filelist[ii])
        fits_stack[:,:,ii] = im
    med_frame = np.median(fits_stack, axis=2)
    return med_frame

# Now our step to create a median of all the bias frames is much simpler!
median_bias = mediancombine(biasfiles)

# Below, how would you save the new median_bias frame as a FITS file?
# Complete the function below and save the duplicate master bias as "Backup_MasterBias.fit")

fits.writeto()


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
You can index multi-element objects within a list as well, but in this case, you index variable[list element index][index of thing you want], as below. Note this is slightly different from the way you were taught to index a numpy array with arrayname[column,row], but the same syntax actually works with numpy arrays (arrayname[column][row])
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
You should now have twice the number of dark frames in your directory, half of which have the prefix 'b_'. These are the frames we want to median combine into the master dark!
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
and the following syntax results in a key called "0"
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
but you can create a new tuple by combining elements from other tuples
Can you write Python code for it?

newt = t[0:2], False, t[3:]
type(newt), newt


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
note: the backslash ('\') above allows you to break lines without interrupting the code. Not technically needed when defining a dictionary or list, but useful in many instances when you have a long operation that is unwiedy in a single line of code
 Dictionaries are used everywhere within Python...
Can you write Python code for it?

# globals() and locals() store all global and local variables (in this case, since we've imported numpy, quite a few)
globals().keys()


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
reverse is a keyword of the .sort() method
Can you write Python code for it?


I am working on the file "Labs/Lab6/Lab6.ipynb".
The first task is:
sets have unique elements. They can be
compared, differenced, unionized, etc.
Can you write Python code for it?

a = set("sp")
b = set("am")
print(a, b)

c = set(["a","m"])
c == b

"p" in a

a | b


I am working on the file "Labs/Lab6/Lab6_Assignment.ipynb".
The first task is:
Goals

Use the plots above to choose a column of data from the boston dataset suitable for regression analysis.
Use the SVD formula to write a function that computes the pseudo-inverse $X^+$ of a 2D numpy array $X$.
Solve the linear regression problem for your data (ie find $\theta$) using your pseudo-inverse implementation, and plot the resulting function against the data you chose.
Now write a function that takes $X$ and $\delta$ and returns the $\delta$-approximation to $X^+$ using the limit formula above.
Solve the linear regression problem again using your approximations for $\delta = 0.5, 0.1, 0.01$ and plot the results.

Validating your models.
When we train a model on some data, we'd like to have an idea of how well our model might generalize to new, unseen samples. Suppose our starting dataset has $m$ entries. The simplest approach to validation is to:

Split these $m$ samples into two subsets: training and validation. A common choice is to do a random 70%/30% split between training/validation.
Train the model on the training set.
Then compute the mean squared error between the known targets and our model's predictions on the validation set.

Now let's do a regression on the sklearn diabetes dataset, and use the simple validation procedure above to test generalizability of our model. First we plot the target variable, which represents how much a patient's diabetes has worsened in the past year, against each of the input variables. (See the diab['DESCR'] field for more information on the individual inputs.)
Can you write Python code for it?

diab = datasets.load_diabetes()

fig = plt.figure(figsize=(15, 15))

for i in range(0, 10):
    ax = fig.add_subplot(4, 3, i + 1)
    ax.set_xlabel(diab['feature_names'][i])
    xs, ys = diab.data[:, i], diab.target
    plt.scatter(xs, ys, marker='.')

plt.show()


I am working on the file "Labs/Lab6/Unix_Programming_Refresher.ipynb".
The first task is:
A1.2) A few more helpful commands
mkdir to make a new directory:
mkdir new_project_name
cp to copy a file:
cp existingfile newfilename
or
cp existingfile newlocation
mv to move or rename a file:
mv old_filename_oldlocation old_filename_newlocation
or
mv old_filename_oldlocation new_filename_oldlocation
rm to PERMANENTLY delete (remove) a file... (use with caution):
rm file_I_will_never_see_again
In the six cells below:
(1) Make a new directory, called temporary
(2) Go into that new directory
(3) Move the file test_file.txt from the original directory above (../test_file.txt) into your current location  using the .
(4) Create a copy of test_file.txt with a new, different filename of your choice.
(5) Delete the original test_file.txt
(6) Go back up into the original location where this notebook is located.
Can you write Python code for it?

# Make a new directory, "temporary"

# Move into temporary

# Move the test_file.txt into this current location

# Create a copy of the test_file.txt, name the copy however you like

# Delete the original test_file.txt

# Change directories to original location of notebook.


I am working on the file "Labs/Lab6/Unix_Programming_Refresher.ipynb".
The first task is:
Appendix 2: Optional Refresher on Conditional Statements and Iteration
A2.1) Conditional Statements
The use of tests or conditions to evaluate variables, values, etc., is a fundamental programming tool. Try executing each of the cells below:
Can you write Python code for it?

2 < 5

3 > 7

x = 11

x > 10

2 * x < x

3.14 <= 3.14 # <= means less than or equal to; >= means greater than or equal to

42 == 42

3e8 != 3e9 # != means "not equal to"

type(True)


I am working on the file "Labs/Lab6/Unix_Programming_Refresher.ipynb".
The first task is:
Now test out a few more things. In the blank cells below, try the following and discuss in your group what each does:
Can you write Python code for it?


I am working on the file "Labs/Lab6/Unix_Programming_Refresher.ipynb".
The first task is:
There are also other ways of iterating, which may be more convenient depending on what you're trying to do. A very common one is the while loop, which does exactly what it sounds like it should: it loops until some condition is met. For example:
Can you write Python code for it?

i = 0 # This starts the initial value off at zero

while i < 11:
    print(i)
    i = i + 3 # This adds three to the value of i, then goes back to the line #3 to check if the condition is met


I am working on the file "Labs/Lab6/Unix_Programming_Refresher.ipynb".
The first task is:
What does the asterisk symbol * do?
Answer: Is a placeholder (wildcard) for some text in a file/folder name.
Now, return to where you started, by moving up a directory:
(one directory up from where you are is denoted with .., while the current directory is denoted with .)
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Alternately we could choose to keep only rows where both tables had a valid measurement using an inner join:
Can you write Python code for it?

join(t, t2, keys=['name'], join_type='inner')


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Although indices are nice for reference, sometimes you might want the row labels to be more descriptive. What is the line below doing?
Can you write Python code for it?

# make the row names more interesting than numbers starting from zero
asteroids.index = ['Asteroid %d'%(i+1) for i in asteroids.index]

#and you can index multiple columns/rows in the usual way
asteroids.iloc[:10,:2]


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Astropy 1.1 and later provides a show_in_notebook() method that allows more interactive exploration of tables. It can be especially handy for large tables.
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Converting the table to numpy
Sometimes you may not want or be able to use a Table object and prefer to work with a plain numpy array (like if you read in data and then want to manipulate it.  This is easily done by passing the table to the np.array() constructor.
This makes a copy of the data.  If you have a huge table and don't want to waste memory, supply copy=False to the constructor, but be warned that changing the output numpy array will change the original table.
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Data frames are defined like dictionaries with a column header/label (similar to a key) and a list of entries.
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Exercises
Read the data
To start with, read in the two data files representing the master source list and observations source list.  The fields for the two tables are respectively documented in:

master_sources
obs_sources

You may use either pandas or astropy tables.
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
If you did the same in a terminal session you get a different view that isn't as pretty but does give a bit more information about the table:
>>> t
<Table rows=4 names=('name','flux')>
array([('source 1', 1.2), ('source 2', 2.2), ('source 3', 3.1),
       ('source 4', 4.3)],
      dtype=[('name', 'S8'), ('flux', '<f8')])

To get a plain view which is the same in notebook and terminal use print():
Can you write Python code for it?

print(t)

##similar, but nicer when there are lots and lots of rows/columns
t.pprint()


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Masked tables
Can you write Python code for it?

t2 = Table([['x', 'y', 'z'],
            [1.1, 2.2, 3.3]],
           names=['name', 'value'],
           masked=True)
t2

t2['value'].mask = [False, True, False]

print(t2)

t2['value'].fill_value = -99
print(t2.filled())


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Now recall our original table t:
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Now we can get a master table of flux measurements which are joined matching the values the name column.  This includes every row from each of the two tables, which is known as an outer join.
Can you write Python code for it?

t3 = join(t, t2, keys=['name'], join_type='outer')
print(t3)

mean(t3['flux2'])


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Pandas
Although astropy Tables has some nice functionality that Pandas doesn't and is also a simpler, easier to use package, Pandas is the more versatile and commonly used table manipluator for Python so I recommend you use it wherever possible.
Astropy 1.1 includes new to_pandas() and from_pandas() methods  that facilitate conversion to/from pandas DataFrame objects.  There are a few caveats in making these conversions:
 - Tables with multi-dimensional columns cannot be converted.
 - Masked values are converted to numpy.nan. Numerical columns, int or float, are thus converted to numpy.float while string columns with missing values are converted to object columns with numpy.nan values to indicate missing or masked data. Therefore, one cannot always round-trip between Table and DataFrame.
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Reading data
You can read data using the Table.read() method:
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
To get the table column names and data types using the colnames and dtype properties:
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
To pull up individual rows or entries, the fact that pandas dataframes always print the indices of rows off of their lefthand side helps. You index dataframes with .loc (if using column name) or .iloc (if using column index), as below
Can you write Python code for it?

asteroids.loc[4,"ra"]

asteroids.iloc[4,0] #same because column 0 is "ra"


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
Unlike astropy Tables, pandas can also read excel spreadsheets
Can you write Python code for it?

asteroids = pd.read_excel("data/asteroids5000.xlsx")
asteroids
#excel_data = Table.from_pandas(pd.read_excel("2mass.xls"))
#excel_data.show_in_notebook()


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
You can do lots more with this as well, including logical operations to parse the table
Can you write Python code for it?

asteroids.columns
ast_new = asteroids[asteroids.dist < 500]
ast_new


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
pandas dataframe columns can be called as python series using the syntax dataframe.columnlabel, as below, which is why it usually makes sense to define a column name/label that is short and has no spaces
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
pandas has built-in functions for reading all kinds of types of data. In the cell below, hit tab once after the r to see all of the read functions. In our case here, read_table will work fine
Can you write Python code for it?


I am working on the file "Labs/Lab7/.ipynb_checkpoints/Lab7-checkpoint.ipynb".
The first task is:
we can also convert the table that we already made with Astropy Tables to pandas dataframe format
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
1. Astropy Tables
The astropy Table class provides an extension of NumPy structured arrays for storing and manipulating heterogeneous tables of data. A few notable features of this package are:

Initialize a table from a wide variety of input data structures and types.
Modify a table by adding or removing columns, changing column names, or adding new rows of data.
Handle tables containing missing values.
Include table and column metadata as flexible data structures.
Specify a description, units and output formatting for columns.
Perform operations like database joins, concatenation, and grouping.
Manipulate multidimensional columns.
Methods for Reading and writing Table objects to files
Integration with Astropy Units and Quantities

or more information about the features and functionalities of Astropy tables, you can read the
astropy.table docs.
<div class=sidebar>

### Sidebar - Tables vs. Pandas DataFrames

The [Pandas](http://pandas.pydata.org/pandas-docs/stable/) package provides a powerful, high-performance table object via the [DataFrame](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html#pandas.DataFrame) class.  Pandas has a few downsides, including its lack of support for multidimensional table columns, but Pandas is the generally-used Python tables packange, and so we will use it here as well.  Pandas DataFrame functionality is very complementary to astropy Tables so astropy 1.1 and later provides interfaces for converting between astropy Tables and DataFrames. If you wish to learn more about Pandas, there are many resources available on-line.  A good starting point is the main tutorials site at http://pandas.pydata.org/pandas-docs/stable/tutorials.html.

### 1.1 Constructing Tables

There is great deal of flexibility in the way that a table can be initially constructed:

- Read an existing table from a file or web URL
- Add columns of data one by one
- Add rows of data one by one
- From an existing data structure in memory:

  - List of data columns
  - Dict of data...
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
1.4 Modifying Tables
Once the table exists with defined columns there are a number of ways to modify the table in place.  These are fully documented in the section Modifying a Table.
To give a couple of simple examples, you can add rows with the add_row() method or add new columns using dict-style assignment:
Can you write Python code for it?

t.add_row(('joe', 10.1))  # Add a new source at the end
t['logflux'] = log10(t['flux'])  # Compute the log10 of the flux
t


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
1.5 Converting Tables to Numpy
Sometimes you may not want or be able to use a Table object and prefer to work with a plain numpy array (like if you read in data and then want to manipulate it.  This is easily done by passing the table to the np.array() constructor.
This makes a copy of the data.  If you have a huge table and don't want to waste memory, supply copy=False to the constructor, but be warned that changing the output numpy array will change the original table.
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
1.6 Masking Tables
One of the most powerful concepts in table manipulation is using boolean selection masks to select only table entries that meet certain criteria.
Can you write Python code for it?

mask = t['flux'] > 3.0  # Define boolean (True/False) mask for all flux values > 3
mask

t[mask]  # Create a new table with only the "True" rows

t2 = Table([['x', 'y', 'z'],
            [1.1, 2.2, 3.3]],
           names=['name', 'value'],
           masked=True)
t2

t2['value'].mask = [False, True, False]

print(t2)

t2['value'].fill_value = -99
print(t2.filled())


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
1.8 Writing and Reading Tabular Data
You can write data using the Table.write() method:
Can you write Python code for it?

t3.write('test.fits', overwrite=True)

t3.write('test.vot', format='votable', overwrite=True)


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
2. Pandas
Although astropy Tables has some nice functionality that Pandas doesn't and is also a simpler, easier to use package, Pandas is the more versatile and commonly used table manipluator for Python so I recommend you use it wherever possible.
Astropy 1.1 includes new to_pandas() and from_pandas() methods  that facilitate conversion to/from pandas DataFrame objects.  There are a few caveats in making these conversions:
 - Tables with multi-dimensional columns cannot be converted.
 - Masked values are converted to numpy.nan. Numerical columns, int or float, are thus converted to numpy.float while string columns with missing values are converted to object columns with numpy.nan values to indicate missing or masked data. Therefore, one cannot always round-trip between Table and DataFrame.
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
3. Exercises
In these exercises, you will be dealing with two tables of information, described below. We'll be doing lots of manipulation of pandas dataframes in Labs 9, 11 and 13, so these exercises focus mostly on special functions of Astropy tables, but you should, wherever possible, try to figure out how to do the same thing with a Pandas dataframe.
master_sources
Each distinct X-ray source identified on the sky is represented in the catalog by a single "master source" entry and one or more "source observation" entries, one for each observation in which the source has been detected. The master source entry records the best estimates of the properties of a source, based on the data extracted from the set of observations in which the source has been detected.  The subset of fields in our exercise table file are:
Name | Description
------ | ------------
msid  | Master source ID
name  | Source name in the Chandra catalog
ra  | Source RA (deg)
dec | Source Dec (deg)
obs_sources
The individual source entries record all of the properties about a detection extracted from a single observation, as well as associated file-based data products, which are observation-specific.  The subset of fields in our exercise table file are:
Name | Description
------ | ------------
obsid | Observation ID
obi | Observation interval
targname | Target name
gti_obs | Observation date
flux_aper_b | Broad band (0.5 - 7 keV) flux (erg/cm2/sec)
src_cnts_aper_b | Broad band source counts
ra_b | Source RA (deg)
dec_b | Source Dec (deg)
livetime | Observation duration (sec)
posid | Position ID
theta | Off-axis angle (arcmin)
msid | Master source ID
<div class=hw>
### Exercise 1 - Read the data
----------------------------

To start with, read in the two data files representing the master source list and observations source list.  The fields for the two tables are respectively documented in:

- [master_sources](http://cxc.harvard.edu/csc/columns/master.html)
- [obs_sources](http://cxc.harvard.edu/csc/columns/persrc.html)

Read them in as...
Can you write Python code for it?

## code to read in source data here

## code to convert to pandas dataframes here


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
<div class=hw>
### Exercise 3 - Visualizing Data

Use the matplotlib [`hist()`]( http://matplotlib.org/api/pyplot_api.html?highlight=pyplot.hist#matplotlib.pyplot.hist) function to make a histogram of the source flux column.  Since the fluxes vary by orders of magnitude, use `numpy.log10` to put the fluxes in log space.
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
<div class=hw>
Display all the rows of the `master_sources` table using its `pprint()` method (astropy tables only).
Can you write Python code for it?

## code to pprint the rows of master_sources


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
<div class=hw>
Find and print the length of each table.
Can you write Python code for it?

## code to print the length of each table here


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
<div class=hw>
Find the column datatypes for each table (also a built-in method).
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
<div class=hw>
Is the length of the new `sources` the same as `obs_sources`?  What happened? Use specific examples in your explanations.
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
<div class=hw>
Let's now remove any sources that we think might not be associated with the source we pointed at ("background/foreground objects" - things that appear near the location of our source in the sky, but that aren't physically associated with it and are actually either much closer or much farther away).   To remove these potentially unassociated objects, make the same plot but using only observations where the source was within 4 arcminutes of the place where the telescope was pointed. *HINT*: use a boolean mask to select values of `theta` that are less than 4.0.
Can you write Python code for it?

# code to mask table and create new histogram


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
<div class=hw>
Print the 50th group and note which columns are the same for all group members and which are different.  Does this make sense?  In these few observations how many different target names were provided by observers?
Can you write Python code for it?

## code to print info for the 50th group


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
<div class=hw>
The `gti_obs` column name is a bit obscure (GTI is "good time interval", but it really just means "date"). Rename the `gti_obs` column to `obs_date`.
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
<div class=sidebar>

### Sidebar - Formatting Output

Notice that the `logflux` column really has too many output digits given the precision of the input values.  We can fix this by setting the format using normal Python formatting syntax:

The format operator in python acts on an object and reformatts it according to your specifications. The syntax is alwasy object.format = '%format_string', where format_string tells it how to format the output.  For now let's just deal with two of the more useful types:

***Float Formatting***
Floats are denoted with '%A.Bf', where A is the number of total characters you want, including the decimal point, and B is the number of characters that you want after the decimal. The f tells it that you would like the output as a float. If you don't specify A, python will keep as many characters as are currently to the left of the decimal point. If you specify more characters to the left of the decimal than are there, python will usually print the extra space as blank characters. If you want it to print leading zeroes instead, use the format '%0A.Bf'. This is not the case in tables though, where white space and leading zeroes will be ignored.

***Scientific Notation Formatting***
Sometimes in tables, we will be dealing with very large numbers. Exponential formatting is similar to float formatting in that you are formatting the float that comes before the "e" (meaning 10 to some power). Numbers in scientific notation print as X.YeNN where NN is the power of the exponent. The formatting string for floating point exponentials looks like "%A.Be" or "%A.BE", where e and E print lowwercase and capital es, respectively.

Should you need it in the future, [here](http://www.python-course.eu/python3_formatted_output.php) is a more detailed reference regarding string formatting.

Also useful is printing numbers in a given format, for which you use the syntax print('%format code'% object), as demonstrated below. Play around with the cells below to make sure you understand the subtelties...
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
Alternately we could choose to keep only rows where both tables had a valid measurement using an inner join:
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
And the opposite operation (conversion from pandas dataframe to astropy table) works as well
Can you write Python code for it?

t_pd = Table.from_pandas(pd_2mass)
t_pd.show_in_notebook()


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
Astropy 1.1 and later provides a show_in_notebook() method that allows more interactive exploration of tables. It can be especially handy for large tables.
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
Data frames are defined like dictionaries with a column header/label (similar to a key) and a list of entries.
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
Now say that we now got some additional flux values from a different reference for a different, but overlapping sample of sources:
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
Now we can get a master table of flux measurements which are joined matching the values the name column.  This includes every row from each of the two tables, which is known as an outer join.
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
To get the table column names and data types using the colnames and dtype properties:
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
To pull up individual rows or entries, the fact that pandas dataframes always print the indices of rows off of their lefthand side helps. You index dataframes with .loc (if using column name) or .iloc (if using column index), as below
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
Unlike astropy Tables, pandas can also read excel spreadsheets
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
You can read data using the Table.read() method:
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
insert explanation here
<div class=hw>
### Exercise 5 - Grouped properties of `sources`
---------------------------

When using tables, we may occasionally wish to group entries based on various properties, which is done using the [`group_by()`](http://astropy.readthedocs.org/en/stable/table/operations.html#id2) functionality.

This method makes a new table in which all the sources with the same entry for some property (that property is specified in the function call) are next to one another.

Make a new table `g_sources` which is the `sources` table grouped by the `msid` key using the `group_by()` method.
Can you write Python code for it?

## code to group sources by the msid key and write into new table g_sources


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
pandas dataframe columns can be called as python series using the syntax dataframe.columnlabel, as below, which is why it usually makes sense to define a column name/label that is short and has no spaces
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
pandas has built-in functions for reading all kinds of types of data. In the cell below, hit tab once after the r to see all of the read functions. In our case here, read_table will work fine
Can you write Python code for it?


I am working on the file "Labs/Lab7/Lab7.ipynb".
The first task is:
we can also convert the table that we already made with Astropy Tables to pandas dataframe format
Can you write Python code for it?


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
Here is the output:

Note that the contourf command doesn't draw contour lines, it just fills the space in between. If you want to label the contours, you can do that by calling contour after calling contourf. For example,
Can you write Python code for it?

plt.gca().set_aspect('equal')
plt.contourf(xx, yy, z2, arange(-0.7, 3.21, 0.5))
cs=plt.contour(xx, yy, z2, arange(-0.7,3.21,0.5), colors='white')
plt.clabel(cs)


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
Let's break down these commands so we can understand what they're doing. First, gca() stands for get current axes. This function lets us grab the axes we're currently plotting to -- something that will become important when we get to multiple axes in a few minutes. Then the command set_aspect('equal'), as applied to these axes, tells pyplot that we want the aspect ratio of the plot to be such that the spacing of points on the axes is equal, even if that means not filling the whole plot window.
The result of this procedure should look something like this:

For this to be more useful in a quantitative sense, it's helpful to have some control over the values at which contours are placed. Fortunately, this is easy to do with an optional additional argument to the contour function. After x, y, and z, one can pass in a fourth argument describing the contours. This can be either a single number, which just specifies how many contours to use, or an array giving the exact values to use for the contours. For example, for our sinc function we know that the maximum is 2.0, and the minimum is -0.43 (you can show this analytically, or just check by doing amax(z1) and amin(z1)), so we can choose contours to go from -0.4 to 1.0 in a spacing of 0.1.
Can you write Python code for it?

plt.gca().set_aspect('equal')
plt.contour(xx, yy, z1, arange(-0.4, 2.0, 0.1))


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
Note that although the underlying data are the same, changing vmin and vmax has the effect of highlighting different aspects of the data. Choice of colorbars and colorscales therefore has aesthetic and, in some cases, ethical effects (for example, you could obscure something that you didn't want a viewer to see with a calculated colorbar choice).
More information on imshow can be found at http://matplotlib.org/users/image_tutorial.html and http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.imshow.
Exercise 1
Write a function that creates a 2D array where the value of each pixel is the distance from some specified central pixel to that pixel. The function should assume an array size of 100x100, but should allow for specification of larger arrays. It should also assume that the centermost pixel of the array is the "central pixel" from which to calculate distances, unless otherwise specified. Once you have it working, create contour plots with labeled contour levels AND raster plots with colorbars for each of the following:
a) a 100x100 array with the central pixel as the reference pixel
b) a 200x200 array with the lower left corner as the reference pixel
c) a 100x300 array with pixel (75,200) as the reference pixel
Multiple Figures, and Multi-Panel Plots
Terminology and concepts
Thus far we have been dealing with a single figure, containing a single set of axes. However, it is often desirable to work with multiple plots at once, either in separate figures, or inside a single figure. Matplotlib makes it possible for us to do this. To understand how this works, we need to start with some basic terminology and concepts. The highest level object that a user of pyplot usually deals with is a figure. One can think of a figure as representing a single plotting window, or, if we're writing output to files, a single file.
At the next level down, most things that we can plot inside a figure will go into axes. Axes are exactly what they sound like: a pair of x and y axes, which are characterized by having some...
Can you write Python code for it?

f, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(10,3), sharey=True)
f.suptitle('Three plots')
im1 = ax1.set_aspect('equal')
ax1.contourf(xx, yy, z2, arange(-0.7, 3.21, 0.5))
cs=ax1.contour(xx, yy, z2, arange(-0.7,3.21,0.5), colors='white')
ax1.clabel(cs)
ax1.set_xlabel("distance from center")
ax1.set_ylabel("distance from center")
im2 = ax2.imshow(z2, aspect='equal', origin='lower', extent=(-2*pi, 2*pi, -2*pi, 2*pi))
f.colorbar(im2, ax=ax2)
im3 = ax3.imshow(z2, aspect='equal', origin='lower', extent=(-2*pi,2*pi,-2*pi,2*pi), vmin=-1, vmax=3)
f.colorbar(im3, ax=ax3)
plt.tight_layout()


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
Note that the colors option to the contour command specifies that all the contours should be drawn in white. The default colors will just blend in with the color fillings already used, so we won't be able to see the lines. Also, by default contours have the nice feature that, for contours that are all the same color, the negative ones will be drawn dashed. Here's the result:

Raster, or "heatmap", plots
A second useful way of representing quantitative information is with raster plots, also sometimes called heatmap or colormap plots. In such a plot, we assign every z value to a color, so that the color at a given (x,y) position is determined by the value of z there. It's something like a filled contour plot, but with colors assigned continuously rather than in discrete blocks.
The pyplot tool for making raster plots is called imshow(), short for image-show. Basic usage is as follows:
Can you write Python code for it?

plt.imshow(z2, aspect='equal', origin='lower', extent=(-2*pi, 2*pi, -2*pi, 2*pi))


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
Note that, to add labels, we assign a variable to the output of the contour function, and then pass that variable to the clabel function. That way we can have multiple sets of contours at once, and label them separately, since each will be referred to by its own set of variables.
Here is the result:
As usual, there are numerous options to control every aspect of the contours, including line styles and thicknesses, colors, label placement, font, etc. For details on contour, see http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.contour. For details on clabel, see  http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.clabel.
We can also make filled contours, which may be a little easier to view. Filled contours are made by the contourf() command, which acts almost exactly like contour. To make filled contours for our function z2, we can do
Can you write Python code for it?

plt.gca().set_aspect('equal')
plt.contourf(xx, yy, z2, arange(-0.7, 3.21, 0.5))


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
Notice that the syntax of imshow() is a little different than contour, and its operation is a bit different.
For imshow, you don't give arrays of x and y values. This is for two reasons. First, imshow is often used as a way of displaying images, in which case there aren't really x and y coordinates, just pixels values. Second, for contour, you are allowed to have x and y points that aren't evenly spaced, whereas for imshow() you aren't. Thus you don't need to specify every x and y point, just the minimum and maximum.
The mechanism for specifying the minimum and maximum is the extent keyword, which you set equal to a tuple of 4 numbers (xmin, xmax, ymin, ymax). If you don't specify the extent keyword, xmin and ymin are taken to be 0, and xmax and ymax to the number of elements in the data in the x and y directions.
The aspect ratio can be set in the imshow command itself, as opposed to through modifying the axes. This is done via the keyword aspect. Setting it to equal forces equal spacing on the two axes.
Finally, the origin keyword specifies where (0,0) is placed. This is another legacy of imshow also being used for image display. When displaying images, the normal convention is that pixel (0,0) is in the top left corner, (0,1) is right below it, (0,2) is right below that, etc. This is the default for imshow(). To display things as we normally plot data, where (0,0) is the origin, (0,1) is above it, etc., we specify origin='lower', which says to place the origin in the lower left rather than upper left corner.
Here's the result:

This is useful, but even more useful is if we can add some information on what the colors mean. The tool for doing this is a color bar, which can be added via the colorbar() command:
Can you write Python code for it?

plt.imshow(z2, aspect='equal', origin='lower', extent=(-2*pi, 2*pi, -2*pi, 2*pi))
plt.colorbar()


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
The first two lines compute the r and phi coordinates in a polar coordinate system. The second two lines define two functions that we'll play with. The function z1 is just the sinc function in 2D, where we take sinc(r) instead of sinc(x). We've put a factor of 2 inside the sin to change the period from 2 pi to pi. The second function z2 just takes z1 and multiplies it by a factor that varies between 1.5 and 0.5 depending on the angle phi, with a period of pi/2 radians in the phi direction.
Contour plots
How can we represent data like this? One simple way is through a contour plot. The archetypical contour plot is a topographic map: it shows curves of constant height, or more generally equal z value, as a function of (x,y) position. Matplotlib produces contour plots using the contour() command. Basic usage is simple: contour(xx, yy, z). The first two arguments are the x and y coordinates, and these can be either 1D arrays whose size matches the corresponding size of the z array, or 2D arrays whose shape matches that of the z array. Let's see what happens when we do this:
Can you write Python code for it?


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
The first two lines should look familiar: they just create two arrays, x and y, that go from -2 pi to 2 pi with a spacing of 0.01. The third line invokes a function you probably haven't seen before, called meshgrid, which is part of the numpy library. The meshgrid function does something very useful. In this case we've passed it two one-dimensional arrays, representing x and y coordinates. We can think of these two arrays as defining a 2D grid of points, with formed by combining all the possible x values with all the possible y values. The meshgrid function gives us back a pair of two-dimensional arrays, xx and yy, that represent the x and y coordinates of these mesh points. Specifically, xx[0,:] = x[0], xx[1,:] = x[1], xx[2,:] = x[2], ..., and similarly yy[:,0] = y[0], yy[:,1] = y[1], yy[:,2] = y[2], and so forth. The keyword indexing='ij' specifies that the x coordinate goes with the first dimension of the output arrays, and the y coordinate with the second. Finally, note that, though we've used it in 2D, the meshgrid command will work for an arbitrary number of dimensions.
Next, let's make some 2D functions from this data. The utility of meshgrid becomes clear as soon as we want to do this:
Can you write Python code for it?

r = sqrt(xx**2 + yy**2)
r.shape

phi = arccos(x/r)

z1 = sin(2*r) / r

z2 = z1 * (1 + 0.5*cos(4*phi))


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
The output looks like this:

It is also possible to control the color scale using the vmin and vmax keywords in the imshow command. These allow one to specify the minimum and maximum values for the color mapping by hand; by default the minimum used is the minimum of the input data, and the maximum is the maximum of the input data. Here's an example:
Can you write Python code for it?

plt.imshow(z2, aspect='equal', origin='lower', extent=(-2*pi,2*pi,-2*pi,2*pi), vmin=-1, vmax=3)
plt.colorbar()


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
The output looks like this:

This is a contour plot of the data. One minor annoyance is that the spacing on the x and y axes isn't equal, so things look stretched. We can fix that by changing the aspect ratio of the axes. To do that, we use the commands
Can you write Python code for it?

plt.gca().set_aspect('equal')
plt.contour(xx, yy, z1)


I am working on the file "Labs/Lab8/.ipynb_checkpoints/Lab 8-checkpoint.ipynb".
The first task is:
We can also label the contours using the clabel() command. Let's remake this plot using some labelled contours, placing the contours starting at -0.4 and going in intervals of 0.4.
Can you write Python code for it?

plt.gca().set_aspect('equal')
cs = plt.contour(xx, yy, z1, arange(-0.4, 2.0, 0.4))
plt.clabel(cs)


I am working on the file "Labs/Lab8/Lab 8.ipynb".
The first task is:
<div class=hw>
### Exercise 2
--------------------

Play around with the example above by changing various things unitl you understand everything its doing. Then, use what you learned to create a 2x2 set of raster plots generated by your radial array function from exercise 1. Each should have a colorbar, axis labels and a title, and the entire plot should have a title at the top.
Can you write Python code for it?

from IPython.core.display import HTML
def css_styling():
    styles = open("../custom.css", "r").read()
    return HTML(styles)
css_styling()


I am working on the file "Labs/Lab8/Lab 8.ipynb".
The first task is:
Exercise 4
Take the image that you used for exercises 1-3 (or another of your choice), read it in, pad it by 25 pixels in each direction with the padded region set to some constant value that stands out relative to the real-valued array, shift it by +10.3 pixels in x and -15.8 pixels in y, replace the padded values with NaNs and display below.
Can you write Python code for it?


I am working on the file "Labs/Lab8/Lab 8.ipynb".
The first task is:
Here is the output:

Note that the contourf command doesn't draw contour lines, it just fills the space in between. If you want to label the contours, you can do that by calling contour after calling contourf. For example,
Can you write Python code for it?


I am working on the file "Labs/Lab8/Lab 8.ipynb".
The first task is:
Notice that the syntax of imshow() is a little different than contour, and its operation is a bit different.
For imshow, you don't give arrays of x and y values. This is for two reasons. First, imshow is often used as a way of displaying images, in which case there aren't really x and y coordinates, just pixels values. Second, for contour, you are allowed to have x and y points that aren't evenly spaced, whereas for imshow() you aren't. Thus you don't need to specify every x and y point, just the minimum and maximum.
The mechanism for specifying the minimum and maximum is the extent keyword, which you set equal to a tuple of 4 numbers (xmin, xmax, ymin, ymax). If you don't specify the extent keyword, xmin and ymin are taken to be 0, and xmax and ymax to the number of elements in the data in the x and y directions.
The aspect ratio can be set in the imshow command itself, as opposed to through modifying the axes. This is done via the keyword aspect. Setting it to equal forces equal spacing on the two axes.
Finally, the origin keyword specifies where (0,0) is placed. This is another legacy of imshow also being used for image display. When displaying images, the normal convention is that pixel (0,0) is in the top left corner, (0,1) is right below it, (0,2) is right below that, etc. This is the default for imshow(). To display things as we normally plot data, where (0,0) is the origin, (0,1) is above it, etc., we specify origin='lower', which says to place the origin in the lower left rather than upper left corner.
Here's the result:

This is useful, but even more useful is if we can add some information on what the colors mean. The tool for doing this is a color bar, which can be added via the colorbar() command:
Can you write Python code for it?