I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Exercises

A positive integer $n$ is called deficient/perfect/abundant according to whether the sum of its proper divisors is less than/equal to/greater than $n$ itself.  Among the numbers up to 10000, how many are deficient, perfect, and abundant?

If $f(n)$ is a function with natural number input and real output, define $F(n)$ to be the function given by the formula $F(n) = \sum_{i=0}^n f(i)$.  Create a function sumfun(f) which takes as input a function f and outputs the function F as described above.

Consider the function $f(n)$ which counts the number of positive divisors of $n$ which are not divisible by 4.  Verify that this is a multiplicative function, and implement it using mult_function.

Write a function foursquare(n) which counts the number of ways that a positive integer n can be expressed as a*a + b*b + c*c + d*d for integers a, b, c, d.  Hint:  loop the variables through integers between $-\sqrt{n}$ and $\sqrt{n}$.  Compare the values of foursquare(n) to the multiplicative function in the previous problem.

A positive integer is "square-free" if it has no square factors besides 1.  The Mobius function $\mu(n)$ is defined by $\mu(n) = 0$ if $n$ is not square-free, and otherwise $\mu(n) = 1$ or $\mu(n) = -1$ according to whether $n$ has an even or odd number of prime factors.  Verify that the Mobius function is multiplicative and implement it.  Try to reproduce the graph of the Mertens function $M(n)$ as described at Wikipedia's article on the Mertens conjecture.  (See the previous Python notebook for an introduction to matplotlib for creating graphs.)
Can you write Python code for it?

#  Use this space to work on the exercises.


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Exercises

Create the list [1,100,2,99,3,98,4,97,...,50,51] with as few list commands as you can.

If you try the commands x = 7, y = 11, then x,y = y,x, what do you expect happens with id(x) and id(y) along the way?

How might you adapt the decompose function to work with all integers (positive and negative)?  Note that zero does not have a prime decomposition, but negative numbers have an associated sign.

Write a function multiply(A,B), in which the parameters A and B are prime decomposition dictionaries and the output is the prime decomposition of their product.

Write a function divides(A,B), in which the parameters A and B are prime decomposition dictionaries and the output is a boolean:  True if A divides B and false otherwise.

The radical of a positive integer N is the positive integer whose prime factors are the same as N, but in which every prime occurs with exponent 1.  For example, $rad(500) = 2 \cdot 5 = 10$.  Write a function radical(N) which computes the radical of N.  You can use the decompose(N) and recompose(N) functions along the way.
Can you write Python code for it?


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
It is sometimes useful to check the ID numbers of objects, to look "under the hood" a bit.  For example, consider the following.
Can you write Python code for it?

x = 3
y = 3
print(x == y)  # This should be true!

id(x)

id(y)


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Let's find the perfect numbers up to 10000.  It might take a few seconds.
Can you write Python code for it?

for j in range(1,10000):
    if is_perfect(j):
        print("{} is perfect!".format(j))


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Let's see how this works for the divisor-counting function.  This is the function $\sigma_0(n)$ whose value is the number of positive divisors of $n$.  For prime powers, it is easy to count divisors, $$\sigma_0(p^e) = e + 1.$$
Can you write Python code for it?


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Now that we have a function to compute the prime decomposition of a positive integer, we write a function to recover a positive integer from such a prime decomposition.  The function is deceptively simple, since Python makes it easy to iterate through the keys of a dictionary.  Make sure that you understand every line.
Can you write Python code for it?

def recompose(D):
    '''
    If D is a dictionary with prime keys and natural values,
    this function outputs the product of terms of the form
    key^value.  In this way, it recovers a single number from a
    prime decomposition.
    '''
    N = 1
    for p in D.keys():  # iterate p through all the keys of D.
        N = N * (p ** D[p])  # Note that D[p] refers to the value (exponent) for the key p.
    return N

D = decompose(1000)
print(D)

recompose(D)  # This should recover 1000.

recompose({2:1, 3:1, 5:1, 7:1})  # What will this give?

# Use this space to experiment with decompose and recompose.


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Now we exploit this function-as-object approach to create a Python function called mult_function.  Given a function f_pp(p,e), the function mult_function outputs the multiplicative function which coincides with f_pp on prime powers.  In other words, if f = mult_function(f_pp), then f(p**e) will equal f_pp(p,e).
Can you write Python code for it?

def mult_function(f_pp):
    '''
    When a function f_pp(p,e) of two arguments is input,
    this outputs a multiplicative function obtained from f_pp
    via prime decomposition.
    '''
    def f(n):
        D = decompose(n)
        result = 1
        for p in D:
            result = result * f_pp(p, D[p])
        return result

    return f


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Python won't be confused by this.  When we assigned x = 5, Python opened up a new memory slot for the number 5, and assigned x to refer to the number in this new slot.  Note that y still "points" at the old slot.  Python tries to be smart about memory, remembering where numbers are stored, and putting numbers into slots "under the hood" as it sees fit.
Can you write Python code for it?

id(3)  # Does Python remember where it put 3?

id(5)  # Does Python remember where it put 5?

id(4)  #  4 was probably not in memory before.  But now it is!

y = 5

id(y)  #  Did Python change the number in a slot?  Or did it point `y` at another slot?

id(L[2]) #  Python doesn't like to waste space.


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Since 'status' is already among the keys of nemo, the command nemo['status'] = 'found' does not create a new key this time.  It just changes the associated value from 'lost' to 'found'.
Can you write Python code for it?

nemo.keys()  # What are the keys of nemo now?

nemo.values()  # What are the values of nemo now?


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Since functions are Python objects, it is possible to define a function which takes a function as input and outputs a function too!  You can pass a function as an input parameter to another function, just as if it were any other variable.  You can output a function with the return keyword, just as if it were another variable.  And you can define a new function within the scope of a function!
Here's a basic example as a warmup.
Can you write Python code for it?

def addone(x):  # Let's make a simple function.
    return x + 1  # It's not a very interesting function, is it.

addone(10) # Predict the result.

def do_twice(f):
    '''
    If a function f is input, then the output is the function
    "f composed with f."
    '''
    def ff(x):  # Defines a new function ff!
        return f(f(x)) # This is what ff does.

    return ff

addtwo = do_twice(addone)  # addtwo is a function!

addtwo(10)  # What is the result?


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Since the divisor-counting function is multiplicative, we can implement it by applying mult_function to sigma0_pp.
Can you write Python code for it?

sigma0 = mult_function(sigma0_pp)

sigma0(100)  # How many divisors does 100 have?


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Some more useful list methods can be found at the official Python tutorial.
Prime decomposition dictionaries
If $N$ is a positive integer, then $N$ can be uniquely decomposed into a product of primes.  Here "uniquely" means that $N$ has a unique expression of the form
$$N = 2^{e_2} 3^{e_3} 5^{e_5} \cdots$$
in which the exponents $e_2, e_3, e_5$, etc., are natural numbers (and only finitely many are nonzero).
A Python dictionary is well-suited to store the resulting prime decomposition.  For example, we might store the prime decomposition $2^3 3^2 7$ with the dictionary {2:3, 3:2, 7:1}.  The primes which occur in the decomposition become the keys of the dictionary, and the natural number exponents becomes the values of the dictionary.
The functions below decompose a positive integer N into primes, storing the result in a dictionary.  The strategy is to repeatedly strip off (divide by) the smallest prime factor of a number, adjusting the dictionary along the way, until the number is reduced to 1.  The first function below finds the smallest prime factor of a number.
Can you write Python code for it?

from math import sqrt  # We'll want to use the square root.

def smallest_factor(n):
    '''
    Gives the smallest prime factor of n.
    '''
    if n < 2:
        return None # No prime factors!

    test_factor = 2 # The smallest possible prime factor.
    max_factor = sqrt(n) # we don't have to search past sqrt(n).

    while test_factor <= max_factor:
        if n%test_factor == 0:
            return test_factor
        test_factor = test_factor + 1  # This could be sped up.

    return n  # If we didn't find a factor up to sqrt(n), n itself is prime!

smallest_factor(105)

smallest_factor(1999**2)  # 1999 might be called the Prince of primes.

smallest_factor(11**3 * 13**9) # The result should be 11.

def decompose(N):
    '''
    Gives the unique prime decomposition of a positive integer N,
    as a dictionary with primes as keys and exponents as values.
    '''
    current_number = N  # We'll divide out factors from current_number until we get 1.
    decomp = {} # An empty dictionary to start.
    while current_number > 1:
        p = smallest_factor(current_number) # The smallest prime factor of the current number.
        if p in decomp.keys():  # Is p already in the list of keys?
            decomp[p] = decomp[p] + 1 # Increase the exponent (value with key p) by 1.
        else:  # "else" here means "if p is not in decomp.keys()".
            decomp[p] = 1  # Creates a new entry in the dictionary, with key p and value 1.
        current_number = current_number // p # Factor out p.  Integer division!
    return decomp

decompose(100) # What is the prime decomposition of 100?

decompose(56401910421778813463) # This should be quick.

decompose(1)  # Good to test the base case!

#  Use this space to experiment a bit with the decompose function.


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
Sometimes we'll want to change a dictionary.  Perhaps we learn that nemo has gotten lost.
Can you write Python code for it?

nemo['status'] = 'lost'

id(nemo)

id('status')

print(nemo)


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
The output of the keys() and values() methods are list-like.  As such, they are convenient for iteration and membership-testing.
Can you write Python code for it?

'color' in nemo.keys()

'taste' in nemo.keys()

'orange' in nemo.keys()  # Is 'orange' a key in the dictionary?

for k in nemo.keys():  # Iterates through the keys.
    print('Nemo\'s {} is {}.'.format(k,nemo[k]))  # \' is used to get a single-quote in a string.


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
There are a lot of interesting multiplicative functions.  We could implement each one by a two-step process as above:  implementing the function for prime powers, then defining a version for positive integers by using the decompose function.  But there's a shortcut for the second step, which brings in a very cool aspect of Python.
In Python, functions are Python objects.
Can you write Python code for it?

type(divisor_sum_pp) # Every object has a type.

id(divisor_sum_pp) # Yes, every object gets an ID number.


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
We mentioned earlier that keys() and values() are methods attached to the object nemo, and methods are functions which are attached to Python objects.
Python objects often (and often by default!) have methods attached to them.  Every dictionary and every list in Python comes with attached methods.  Methods can be used to extract properties of objects or change them.  Here are examples of some list methods.
Can you write Python code for it?

L = [2,3,5,7]
print(L) # Let's remember what the list L is.

L[0] # What is this?

id(L[0]) # What is the ID number of the 0th item in the list?

L.reverse() # The reverse() method changes L!
print(L)

L[0]  # We have definitely changed L.

L[3]  # The last item in the list L.

id(L[3]) # The ID number of the last item in the list L.


I am working on the file "P3wNT Notebook 4.ipynb".
The first task is:
What happened?  You probably noticed that both variables x and y have the same id number.  That means that Python is being efficient, and not filling up two different slots of memory with the same number (3).  Instead, it puts the number in one memory slot, and uses x and y as alternative names for this slot.
But what happens if we change a value of one variable?
Can you write Python code for it?


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
For some numbers -- the Carmichael numbers -- there are more bad witnesses than good witnesses.  For example, take the Carmichael number 41041, which is not prime ($41041 = 7 \cdot 11 \cdot 13 \cdot 41$).
Can you write Python code for it?

for witness in range(1,20):
    flt = pow(witness, 41040, 41041)
    if flt == 1:
        print("{} is a bad witness.".format(witness))
    else:
        print("{} raised to the 41040th power equals {}, mod 41041".format(witness, flt))


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
How fast is our new is_prime function?  Let's give it a try.
Can you write Python code for it?


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
It is straightforward to modify Pingala's algorithm to compute exponents in modular arithmetic.  Just reduce along the way.
Can you write Python code for it?

def powmod_Pingala(base,exponent,modulus):
    result = 1
    bitstring = bin(exponent)[2:] # Chop off the '0b' part of the binary expansion of exponent
    for bit in bitstring: # Iterates through the "letters" of the string.  Here the letters are '0' or '1'.
        if bit == '0':
            result = (result*result) % modulus
        if bit == '1':
            result = (result*result * base) % modulus
    return result

powmod_Pingala(5,90,91)


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
It might seem tedious to carry out this "reduction mod m" at every step along the way.  But the advantage is that you never have to work with numbers much bigger than the modulus (m) if you can reduce modulo m at each step.
For example, consider the following computation.
Can you write Python code for it?

3**999 # A very large integer.

(3**999) % 1000  # What are the last 3 digits of 3 raised to the 999 power?


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
Note that we use "natural representatives" when doing modular arithmetic in Python.  So the only numbers whose square is 1 mod 41 are 1 and 40.  (Note that 40 is the natural representative of -1, mod 41).  If we consider the "square roots of 1" with a composite modulus, we find more (as long as the modulus has at least two odd prime factors).
Can you write Python code for it?

for x in range(91):
    if x*x % 91 == 1:
        print("{} squared is congruent to 1, mod 91.".format(x))  # What numbers do you think will be printed?


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
Python outputs binary expansions as strings, beginning with '0b'.  To summarize, we can compute an exponent like $b^e$ by the following process:
Pingala's Exponentiation Algorithm

Set the number to 1.

Read the bits of $e$, from left to right.
    a. When the bit is zero, square the number.
    b. When the bit is one, square the number, then multiply by $b$.

Output the resulting number.
Can you write Python code for it?

def pow_Pingala(base,exponent):
    result = 1
    bitstring = bin(exponent)[2:] # Chop off the '0b' part of the binary expansion of exponent
    for bit in bitstring: # Iterates through the "letters" of the string.  Here the letters are '0' or '1'.
        if bit == '0':
            result = result*result
        if bit == '1':
            result = result*result * base
    return result

pow_Pingala(5,90)


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
The codes "BIT 1" and "BIT 0" tell us what happened at each step, and allow the process to be reversed.  In a line with BIT 0, the exponent gets doubled as one goes up one line (e.g., from 11 to 22).  In a line with BIT 1, the exponent gets doubled then increased by 1 as one goes up one line (e.g., from 2 to 5).
We can use these BIT codes in order to compute an exponent.  Below, we follow the BIT codes to compute $5^{90}$.
Can you write Python code for it?

n = 1  # This is where we start.
n = n*n * 5 # BIT 1 is interpreted as square-then-multiply-by-5, since the exponent is doubled then increased by 1.
n = n*n  # BIT 0 is interpreted as squaring, since the exponent is doubled.
n = n*n * 5 # BIT 1
n = n*n * 5 # BIT 1 again.
n = n*n # BIT 0
n = n*n * 5 # BIT 1
n = n*n # BIT 0

print(n)  # What we just computed.
print(5**90) # I hope these match!!


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
The fully built-in modular exponentiation pow(b,e,m) command is probably the fastest.  But our implementation of Pingala's algorithm isn't bad -- it probably beats the simple (b**e) % m command (in the powermod_1 function), and it's certainly faster than our naive loop in powermod_2.
One can quantify the efficiency of these algorithms by analyzing how the time depends on the size of the input parameters.  For the sake of exposition, let us keep the base and modulus constant, and consider how the time varies with the size of the exponent.
As a function of the exponent $e$, our powmod_Pingala algorithm required some number of multiplications, bounded by twice the number of bits of $e$.  The number of bits of $e$ is approximately $\log_2(e)$.  The size of the numbers multiplied is bounded by the size of the (constant) modulus.  In this way, the time taken by the powmod_Pingala algorithm should be $O(\log(e))$, meaning bounded by a constant times the logarithm of the exponent.
Contrast this with the slow powermod_2 algorithm, which performs $e$ multiplications, and has thus has runtime $O(e)$.
The Miller-Rabin test
Pingala's algorithm is effective for computing exponents, in ordinary arithmetic or in modular arithmetic.  In this way, we can look for violations of Fermat's Little Theorem as before, to find witnesses to non-primality.  But if we look more closely at the algorithm... we can sometimes find violations of the ROO property of primes.  This strengthens the primality test.
To see this, we create out a "verbose" version of Pingala's algorithm for modular exponentiation.
Can you write Python code for it?

def powmod_verbose(base, exponent, modulus):
    result = 1
    print("Computing {} raised to {}, modulo {}.".format(base, exponent, modulus))
    print("The current number is {}".format(result))
    bitstring = bin(exponent)[2:] # Chop off the '0b' part of the binary expansion of exponent
    for bit in bitstring: # Iterates through the "letters" of the string.  Here the letters are '0' or '1'.
        sq_result = result*result % modulus  # We need to compute this in any case.
        if bit == '0':
            print("BIT 0:  {} squared is congruent to {}, mod {}".format(result, sq_result, modulus))
            result = sq_result
        if bit == '1':
            newresult = (sq_result * base) % modulus
            print("BIT 1:  {} squared times {} is congruent to {}, mod {}".format(result, base, newresult, modulus))
            result = newresult
    return result

powmod_verbose(2,560,561)  # 561 is a Carmichael number.


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
The pow(b,e,m) command should give a significant speedup, as compared to the pow(b,e) command.  Remember that ns stands for nanoseconds (billionths of a second) and µs stands for microseconds (millionths of a second).
Exponentiation runs so quickly because not only is Python reducing modulo m along the way, it is performing a surprisingly small number of multiplications.  In our loop approach, we computed $3^{999}$ by multiplying repeatedly.  There were 999 multiplications!  But consider this carefully -- did we need to perform so many multiplications?  Can you compute $3^{999}$ with far fewer multiplications?  What if you can place results in memory along the way?
In the next section, we study a very efficient algorithm for computing such exponents.  The goal in designing a good algorithm is to create something which runs quickly, minimizes the need for memory, and runs reliably for all the necessary input values.  Often there are trade-offs between speed and memory usage, but our exponentiation algorithm will be excellent in both respects.  The ideas go back to Pingala, an Indian mathematician of the 2nd or 3rd century BCE, who developed his ideas to enumerate possible poetic meters (arrangements of long and short syllables into verses of a given length).
You may wonder why it is necessary to learn the algorithm at all, if Python has an optimized algorithm built into its pow command.  First, it is interesting!  But also, we will need to understand the algorithm in finer detail to implement the Miller-Rabin test:  a way of quickly testing whether very large numbers are prime.
Exercises

Adapt the solve_LDE function from PwNT Notebook 2, in order to write a function modular_inverse(a,m) which computes the multiplicative inverse of a modulo m (if they are coprime).

Use the timeit and randint functions to investigate how the speed of the command pow(a,e,m) depends on how many digits the numbers a, e, and m have.  Note that randint(10**(d-1), 10**d - 1) will produce a random integer with d digits.  If you...
Can you write Python code for it?


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
The results will probably on the order of a millisecond, perhaps even a tenth of a millisecond ($10^{-4}$ seconds) for non-primes!  That's much faster than looking for factors, for numbers of this size.  In this way, we can test primality of numbers of hundreds of digits!
For an application, let's find some Mersenne primes.  Recall that a Mersenne prime is a prime of the form $2^p - 1$.  Note that when $2^p - 1$ is prime, it must be the case that $p$ is a prime too.  We will quickly find the Mersenne primes with $p$ up to 1000 below!
Can you write Python code for it?

for p in range(1,1000):
    if is_prime(p):  # We only need to check these p.
        M = 2**p - 1 # A candidate for a Mersenne prime.
        if is_prime(M):
            print("2^{} - 1 = {} is a Mersenne prime.".format(p,M))


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
The syntax of the timeit function is a bit challenging.  The first parameter is the Python code which we are timing (as a string), in this case powermod_*(3,999,1000).  The second parameter probably looks strange.  It exists because the timeit function sets up a little isolation chamber to run the code -- within this isolation chamber, it only knows standard Python commands and not the new functions you've created.  So you have to import your functions (powermod_1 or powermod_2) into its isolation box.  Where are these imported from?  They are contained in __main__ which is the location of all of your other code.  Finally, the third parameter number is the number of times the timeit function will repeat the code (by default, this might be a large number like 1000000).
The output of the timeit function is a float, which represents the number of seconds taken for all of the repetitions.  Contrast this with the timeit magic, which found the average.  So you need to divide by the number parameter (10000 in the above examples) to find the average time taken.
Using the timeit function, we can compare the performance of the two powermod functions on multiple inputs, wrapping the whole comparison process in a bigger function.  We choose our inputs randomly in order to estimate the expected performance of our functions.  To choose random inputs, Python has a package aptly called random.
Can you write Python code for it?

from random import randint # randint chooses random integers.

print("My number is {}".format(randint(1,10))) # Run this line many times over to see what happens!


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
To compute the last three digits of $3^{999}$, Python works with some very large integers along the way.  But what if we could reduce modulo 1000 at every step?  Then, as Python multiplies terms, it will never have to multiply numbers bigger than 1000.  Here is a brute-force implementation.
Can you write Python code for it?

P = 1  # The "running product" starts at 1.
for i in range(999):  # We repeat the following line 999 times, as i traverses the list [0,1,...,998].
    P = (P * 3)%1000  # We reduce modulo 1000 along the way!
print(P)


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
We can learn something from the previous two examples.  Namely, 91 and 12319 are not prime numbers.  We say that 7 witnesses the non-primality of 12319.  Moreover, we learned this fact without actually finding a factor of 12319!  Indeed, the factors of 12319 are 97 and 127, which have no relationship to the "witness" 7.
In this way, Fermat's Little Theorem -- a statement about prime numbers -- can be turned into a way of discovering that numbers are not prime.  After all, if $p$ is not prime, then what are the chances that $a^{p-1} \equiv 1$ mod $p$ by coincidence?
Can you write Python code for it?


I am working on the file "P3wNT Notebook 5.ipynb".
The first task is:
We have described two properties of prime numbers, and therefore two possible indicators that a number is not prime.

If $p$ is a number which violates Fermat's Little Theorem, then $p$ is not prime.

If $p$ is a number which violates the ROO property, then $p$ is not prime.

The Miller Rabin test will combine these indicators.  But first we have to introduce an ancient algorithm for exponentiation.
Pingala's exponentiation algorithm
If we wish to compute $5^{90}$ mod $91$, without the pow command, we don't have to carry out 90 multiplications.  Instead, we carry out Pingala's algorithm.  To understand this algorithm, we begin with the desired exponent (e.g. $e=90$), and carry out a series of steps:  replace $e$ by $e/2$ if $e$ is even, and replace $e$ by $(e-1) / 2$ if $e$ is odd.  Repeat this until the exponent is decreased to zero.  The following function carries out this process on any input $e$.
Can you write Python code for it?

def Pingala(e):
    current_number = e
    while current_number > 0:
        if current_number%2 == 0:
            current_number = current_number // 2
            print("Exponent {} BIT 0".format(current_number))
        if current_number%2 == 1:
            current_number = (current_number - 1) // 2
            print("Exponent {} BIT 1".format(current_number))

Pingala(90)


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
Each "letter" of a string again belongs to the string type.  A string of length one is called a character.
Can you write Python code for it?

print(type(W))
print(type(W[0]))  # W[0] is a character.


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
In the seconds or minutes your computer was running, it checked the primality of almost 90 thousand numbers, each with 100 digits.  Not bad!
The Diffie-Hellman protocol
When we study protocols for secure communication, we must keep track of the communicating parties (often called Alice and Bob), and who has knowledge of what information.  We assume at all times that the "wire" between Alice and Bob is tapped -- anything they say to each other is actively monitored, and is therefore public knowledge.  We also assume that what happens on Alice's private computer is private to Alice, and what happens on Bob's private computer is private to Bob.  Of course, these last two assumptions are big assumptions -- they point towards the danger of computer viruses which infect computers and can violate such privacy!
The goal of the Diffie-Hellman protocol is -- at the end of the process -- for Alice and Bob to share a secret without ever having communicated the secret with each other.  The process involves a series of modular arithmetic calculations performed on each of Alice and Bob's computers.
The process begins when Alice or Bob creates and publicizes a large prime number p and a primitive root g modulo p.  It is best, for efficiency and security, to choose a safe prime p.  Alice and Bob can create their own safe prime, or choose one from a public list online, e.g., from the RFC 3526 memo.  Nowadays, it's common to take p with 2048 bits, i.e., a prime which is between $2^{2046}$ and $2^{2047}$ (a number with 617 decimal digits!).
For the purposes of this introduction, we use a smaller safe prime, with about 256 bits.  We use the SystemRandom functionality of the random package to create a good random prime.  It is not so much of an issue here, but in general one must be very careful in cryptography that one's "random" numbers are really "random"!  The SystemRandom function uses chaotic properties of your computer's innards in order to initialize a random number generator, and is considered cryptographically secure.
Can you write Python code for it?

from random import SystemRandom # Import the necessary package.

r = SystemRandom().getrandbits(256)
print("The random integer is {}".format(r))
print("with binary expansion {}".format(bin(r)))  #  r is an integer constructed from 256 random bits.
print("with bit-length {}.".format(len(bin(r)) - 2))  # In case you want to check.  Remember '0b' is at the beginning.

def getrandSGprime(bitlength):
    '''
    Creates a random Sophie Germain prime p with about
    bitlength bits.
    '''
    while True:
        p = SystemRandom().getrandbits(bitlength) # Choose a really random number.
        if is_SGprime(p):
            return p


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
Let's see the effect of the Caesar cipher on our ASCII table.
Can you write Python code for it?

for a in range(32,127):
    c = chr(a)
    print("ASCII {} is {}, which shifts to {}".format(a, c, Caesar_shift(c,5))) # Shift by 5.


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
Let's test this out by finding the Sophie Germain primes up to 100, and their associated safe primes.
Can you write Python code for it?

for j in range(1,100):
    if is_SGprime(j):
        print(j, 2*j+1)


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
Next we find a primitive root, modulo the safe prime p.
Can you write Python code for it?

def findprimroot_safe(p):
    '''
    Finds a primitive root,
    modulo a safe prime p.
    '''
    b = 2  # Start trying with 2.
    while True:  # We just keep on looking.
        if is_primroot_safe(b,p):
            return b
        b = b + 1 # Try the next base.  Shouldn't take too long to find one!

g = findprimroot_safe(p)
print(g)


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
Now Alice, on her private computer, computes $B^a$ mod $p$.  She can do that because everyone knows $B$ and $p$, and she knows $a$ too.
Similarly, Bob, on his private computer, computes $A^b$ mod $p$.  He can do that because everyone knows $A$ and $p$, and he knows $b$ too.
Alice and Bob do not share the results of their computations!
Can you write Python code for it?

print(pow(B,a,p))  # This is what Alice computes.

print(pow(A,b,p))  # This is what Bob computes.


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
Now we can use the Caesar cipher to encrypt strings.
Can you write Python code for it?

def Caesar_cipher(plaintext, shift):
    ciphertext = ''
    for c in plaintext:  # Iterate through the characters of a string.
        ciphertext = ciphertext + Caesar_shift(c,shift)
    return ciphertext

print(Caesar_cipher('Hello!  Can you read this?', 5))  # Shift forward 5 units in ASCII.


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
Since we only work with the ASCII range between 32 and 126, it will be useful to "cycle" other numbers into this range.  For example, we will interpret 127 as 32, 128 as 33, etc., when we convert out-of-range numbers into characters.
The following function forces a number into a given range, using the mod operator.  It's a common trick, to make lists loop around cyclically.
Can you write Python code for it?

def inrange(n,range_min, range_max):
    '''
    The input number n can be any integer.
    The output number will be between range_min and range_max (inclusive)
    If the input number is already within range, it will not change.
    '''
    range_len = range_max - range_min + 1
    a = n % range_len
    if a < range_min:
        a = a + range_len
    return a

inrange(13,1,10)

inrange(17,5,50)


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
The Vigenère cipher
The Caesar cipher is pretty easy to break, by a brute force attack (shift by all possible values) or a frequency analysis (compare the frequency of characters in a message to the frequency of characters in typical English messages, to make a guess).
The Vigenère cipher is a variant of the Caesar cipher which uses an ecryption key to vary the shift-parameter throughout the encryption process.  For example, to encrypt the message "This is very secret" using the key "Key", you line up the characters of the message above repeated copies of the key.
T | h | i | s |   | i | s |   | v | e | r | y |   | s | e | c | r | e | t
--|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|--
K | e | y | K | e | y | K | e | y | K | e | y | K | e | y | K | e | y | K
Then, you turn everything into ASCII (or your preferred numerical system), and use the bottom row to shift the top row.
ASCII message | 84 | 104 | 105 | 115 | 32 | 105 | 115 | 32 | 118 | 101 | 114 | 121 | 32 | 115 | 101 | 99 | 114 | 101 | 116
---|-----|-----
Shift | 75 | 101 | 121 | 75 | 101 | 121 | 75 | 101 | 121 | 75 | 101 | 121 | 75 | 101 | 121 | 75 | 101 | 121 | 75
ASCII shifted | 159 | 205 | 226 | 190 | 133 | 226 | 190 | 133 | 239 | 176 | 215 | 242 | 107 | 216 | 222 | 174 | 215 | 222 | 191
ASCII shifted in range |   64 | 110 | 36 | 95 | 38 | 36 | 95 | 38 | 49 | 81 | 120 | 52 | 107 | 121 | 32 | 79 | 120 | 32 | 96
Finally, the shifted ASCII codes are converted back into characters for transmission.  In this case, the codes 64,110,36,95, etc., are converted to the ciphertext "@n$_&$_&1Qx4ky Ox \`"
The Vigenère cipher is much harder to crack than the Caesar cipher, if you don't have the key.  Indeed, the varying shifts make frequency analysis more difficult.  The Vigenère cipher is weak by today's standards (see Wikipedia for a description of 19th century attacks), but illustrates the basic actors in a symmetric key cryptosystem: the plaintext, ciphertext, and a single key.  Today, symmetric key cryptosystems like...
Can you write Python code for it?

def Vigenere_cipher(plaintext, key):
    ciphertext = '' # Start with an empty string
    for j in range(len(plaintext)):
        c = plaintext[j] # the jth letter of the plaintext
        key_index = j % len(key) # Cycle through letters of the key.
        shift = ord(key[key_index]) # How much we shift c by.
        ciphertext = ciphertext + Caesar_shift(c,shift) # Add new letter to ciphertext
    return ciphertext

print(Vigenere_cipher('This is very secret', 'Key')) # 'Key' is probably a bad key!!


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
The Vigenère cipher is called a symmetric cryptosystem, because the same key that is used to encrypt the plaintext can be used to decrypt the ciphertext.  All we do is subtract the shift at each stage.
Can you write Python code for it?

def Vigenere_decipher(ciphertext, key):
    plaintext = '' # Start with an empty string
    for j in range(len(ciphertext)):
        c = ciphertext[j] # the jth letter of the ciphertext
        key_index = j % len(key) # Cycle through letters of the key.
        shift = - ord(key[key_index]) # Note the negative sign to decipher!
        plaintext = plaintext + Caesar_shift(c,shift) # Add new letter to plaintext
    return plaintext

print(Vigenere_decipher('@n$_&$_&1Qx4ky Ox `', 'Key'))

# Try a few cipher/deciphers yourself to get used to the Vigenere system.


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
The following code will produce a table of the ASCII characters with codes between 32 and 126.  This is a good range which includes all the most common English characters and symbols on a U.S. keyboard.  Note that ASCII code 32 corresponds to an empty space (an important character for long messages!)
Can you write Python code for it?

for a in range(32,127):
    c = chr(a)
    print("ASCII {} is {}".format(a, c))


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
The function above searches and searches among random numbers until it finds a Sophie Germain prime.  The (possibly endless!) search is performed with a while True: loop that may look strange.  The idea is to stay in the loop until such a prime is found.  Then the return p command returns the found prime as output and halts the loop.  One must be careful with while True loops, since they are structured to run forever -- if there's not a loop-breaking command like return or break inside the loop, your computer will be spinning for a long time.
Can you write Python code for it?

q = getrandSGprime(256) # A random ~256 bit Sophie Germain prime
p = 2*q + 1 # And its associated safe prime

print("p is ",p)  # Just to see what we're working with.
print("q is ",q)


I am working on the file "P3wNT Notebook 6.ipynb".
The first task is:
Woah!  What happened?  In terms of exponents, it's elementary.  For
$$B^a = (g^{b})^a = g^{ba} = g^{ab} = (g^a)^b = A^b.$$
So these two computations yield the same result (mod $p$, the whole way through).
In the end, we find that Alice and Bob share a secret.  We call this secret number $S$.
$$S = B^a = A^b.$$
Can you write Python code for it?

S = pow(B,a,p)  # Or we could have used pow(A,b,p)
print(S)


I am working on the file "P3wNT Notebook 7.ipynb".
The first task is:
Encryption and decryption
Now we explain how the public key $(N,e)$ and the private key $(p,q)$ are used to encrypt and decrypt a (numerical) message $m$.  If you wish to encrypt/decrypt a text message, one case use a numerical scheme like ASCII, of course.  The message should be significantly shorter than the modulus, $m < N$ (ideally, shorter than the private key primes), but big enough so that $m^e$ is much bigger than $N$ (not usually a problem if $e = 65537$).
The encryption procedure is simple -- it requires just one line of Python code, using the message and public key.  The ciphertext $c$ is given by the formula
$$c = m^e \text{ mod } N,$$
where here we mean the "natural representative" of $m^e$ modulo $N$.
Can you write Python code for it?

def RSA_encrypt(message, N, e):
    '''
    Encrypts message, using the public keys N,e.
    '''
    return pow(message, e, N)

c = RSA_encrypt(17,N,e) # c is the ciphertext.
print("The ciphertext is {}".format(c)) # A very long number!


I am working on the file "P3wNT Notebook 7.ipynb".
The first task is:
Exercises

What is the largest integer whose totient is 100?  Study this by a brute force search, i.e., looping through the integers up to 10000

For which integers $n$ is it true that $\phi(n) = n/2$?  Study this by a brute force search in order to make a conjecture.  Then prove it if you can.

Compute the totient of the numbers from 1 to 10000, and analyze the results.  The totient $\phi(n)$ is always less than $n$ when $n > 1$, but how does the ratio $\phi(n) / n$ behave? Create a graph.  What is the average value of the ratio?

If $a^{323} \equiv 802931$, mod $5342481$, and $GCD(a, 5342481) = 1$, and $0 < a < 5342481$, then what is $a$?

Challenge:  Create a function superpow(x,y,z,m) which computes $x^{y^z}$ modulo $m$ efficiently, when $GCD(x,m) = 1$ and $m$ is small enough to factor.

The RSA protocol
Like Diffie-Hellman, the RSA protocol involves a series of computations in modular arithmetic, taking care to keep some numbers private while making others public.  RSA was published two years after Diffie-Hellman, in 1978 by Rivest, Shamir, and Adelman (hence its name).  The great advance of the RSA protocol was its asymmetry.  While Diffie-Hellman is used for symmetric key cryptography (using the same key to encrypt and decrypt), the RSA protocol has two keys:  a public key that can be used by anyone for encryption and a private key that can be used by its owner for decryption.
In this way, if Alice publishes her public key online, anyone can send her an encrypted message.  But as long as she keeps her private key private, only Alice can decrypt the messages sent to her.  Such an asymmetry allows RSA to be used for authentication -- if the owner of a private key has an ability nobody else has, then this ability can be used to prove the owner's identity.  In practice, this is one of the most common applications of RSA, guaranteeing that we are communicating with the intended person.
In the RSA protocol, the private key is a pair of large (e.g. 512 bit) prime numbers, called...
Can you write Python code for it?

from random import SystemRandom, randint

def Miller_Rabin(p, base):
    '''
    Tests whether p is prime, using the given base.
    The result False implies that p is definitely not prime.
    The result True implies that p **might** be prime.
    It is not a perfect test!
    '''
    result = 1
    exponent = p-1
    modulus = p
    bitstring = bin(exponent)[2:] # Chop off the '0b' part of the binary expansion of exponent
    for bit in bitstring: # Iterates through the "letters" of the string.  Here the letters are '0' or '1'.
        sq_result = result*result % modulus  # We need to compute this in any case.
        if sq_result == 1:
            if (result != 1) and (result != exponent):  # Note that exponent is congruent to -1, mod p.
                return False  # a ROO violation occurred, so p is not prime
        if bit == '0':
            result = sq_result
        if bit == '1':
            result = (sq_result * base) % modulus
    if result != 1:
        return False  # a FLT violation occurred, so p is not prime.

    return True  # If we made it this far, no violation occurred and p might be prime.

def is_prime(p, witnesses=50):  # witnesses is a parameter with a default value.
    '''
    Tests whether a positive integer p is prime.
    For p < 2^64, the test is deterministic, using known good witnesses.
    Good witnesses come from a table at Wikipedia's article on the Miller-Rabin test,
    based on research by Pomerance, Selfridge and Wagstaff, Jaeschke, Jiang and Deng.
    For larger p, a number (by default, 50) of witnesses are chosen at random.
    '''
    if (p%2 == 0): # Might as well take care of even numbers at the outset!
        if p == 2:
            return True
        else:
            return False

    if p > 2**64:  # We use the probabilistic test for large p.
        trial = 0
        while trial < witnesses:
            trial = trial + 1
            witness = randint(2,p-2) # A good range for possible witnesses
            if Miller_Rabin(p,witness) == False:
    ...


I am working on the file "P3wNT Notebook 7.ipynb".
The first task is:
In every line ending with True, the first and third numbers should match.  This will happen in some False lines too, but not reliably since Euler's theorem does not apply there.
We found the exponent 27 -- reversing the cubing operation -- by an ad hoc sort of procedure.  The relationship between 27 and 3, which made things work, is that
$$3 \cdot 27 \equiv 1 \text{ mod } 40.$$
In other words, $3 \cdot 27 = 1 + 40k$ for some (positive, in fact) integer $k$.
Recalling that $40 = \phi(100)$, this relationship and Euler's theorem imply that
$$a^{3 \cdot 27} = a^{1 + 40k} \equiv a^1 = a \text{ mod } 100.$$
By this argument, we have the following consequence of Euler's theorem.
Theorem:  If $GCD(a,m) = 1$, and $ef \equiv 1$ mod $\phi(m)$, then
$$a^{ef} \equiv a \text{ mod } \phi(m).$$
In this way, "raising to the $f$ power" is like "taking the $e$-th root", modulo $m$.
If we are given $f$, then $e$ is a multiplicative inverse of $f$ modulo $\phi(m)$.  In particular, such a multiplicative inverse exists if and only if $GCD(e,\phi(m)) = 1$.  The following function computes a multiplicative inverse, by adapting the solve_LDE function from Notebook 2.  After all, solving $ex \equiv 1$ mod $m$ is equivalent to solving the linear Diophantine equation $ex + my = 1$ (and only caring about the $x$-value).
Can you write Python code for it?

def mult_inverse(a,m):
    '''
    Finds the multiplicative inverse of a, mod m.
    If GCD(a,m) = 1, this is returned via its natural representative.
    Otherwise, None is returned.
    '''
    u = a # We use u instead of dividend.
    v = m # We use v instead of divisor.
    u_hops, u_skips = 1,0 # u is built from one hop (a) and no skips.
    v_hops, v_skips = 0,1 # v is built from no hops and one skip (b).
    while v != 0:   # We could just write while v:
        q = u // v  # q stands for quotient.
        r = u % v  # r stands for remainder.  So u = q(v) + r.

        r_hops = u_hops - q * v_hops  # Tally hops
        r_skips = u_skips - q * v_skips  # Tally skips

        u,v = v,r  # The new dividend,divisor is the old divisor,remainder.
        u_hops, v_hops = v_hops, r_hops # The new u_hops, v_hops is the old v_hops, r_hops
        u_skips, v_skips = v_skips, r_skips # The new u_skips, v_skips is the old v_skips, r_skips

    g = u # The variable g now describes the GCD of a and b.
    if g == 1:
        return u_hops % m
    else:  # When GCD(a,m) is not 1...
        return None

mult_inverse(3,40)  # 3 times what is congruent to 1, mod 40?

mult_inverse(5,40)  # None should be returned.


I am working on the file "P3wNT Notebook 7.ipynb".
The first task is:
That's the entire process of encryption and decryption.  Encryption requires just the public key $(N,e)$ and decryption requires the private key $(p,q)$ too.  Everything else is modular arithmetic, using Euler's theorem and the Euclidean algorithm to find modular multiplicative inverses.
From a practical standpoint, there are many challenges, and we just mention a few here.
Key generation:  The person who constructs the private key $(p,q)$ needs to be careful.  The primes $p$ and $q$ need to be pretty large (512 bits, or 1024 bits perhaps), which is not so difficult.  They also need to be constructed randomly.  For imagine that Alice comes up with her private key $(p,q)$ and Anne comes up with her private key $(q,r)$, with the same prime $q$ in common.  Their public keys will include the numbers $N = pq$ and $M = qr$.  If someone like Arjen comes along and starts taking GCDs of all the public keys in a database, that person will stumble upon the fact that $GCD(N,M) = q$, from which the private keys $(p,q)$ and $(q,r)$ can be derived.  And this sort of disaster has happened!  Poorly generated keys were stored in a database, and discovered by Arjen Lenstra et al..
Security by difficulty of factoring:  The security of RSA is based on the difficulty of obtaining the private key $(p,q)$ from the public key $(N,e)$.  Since $N = pq$, this is precisely the difficulty of factoring a large number $N$ into two primes (given the knowledge that it is the product of two primes).  Currently it seems very difficult to factor large numbers.  The RSA factoring challenges give monetary rewards for factoring such large $N$.  The record (2017) is factoring a 768-bit (232 digit) number, RSA-768.  For this reason, we may consider a 1024-bit number secure for now (i.e. $p$ and $q$ are 512-bit primes), or use a 2048-bit number if we are paranoid.  If quantum computers develop sufficiently, they could make factoring large numbers easy, and RSA will have to be replaced by a quantum-secure protocol.
Web of Trust:  Trust needs to begin...
Can you write Python code for it?

from hashlib import sha512

print(sha512("I like sweet potato hash.".encode('utf-8')).digest()) # A 64-character string of hash.


I am working on the file "P3wNT Notebook 7.ipynb".
The first task is:
The totient is a multiplicative function, meaning that if $GCD(a,b) = 1$, then $\phi(ab) = \phi(a) \phi(b)$.  Therefore, the totient of number can be found quickly from the totient of the prime powers within its decomposition.  We can re-implement the totient using our functions for prime decomposition and multiplicative functions from Notebook 4.
Can you write Python code for it?

from math import sqrt  # We'll want to use the square root.

def smallest_factor(n):
    '''
    Gives the smallest prime factor of n.
    '''
    if n < 2:
        return None # No prime factors!

    test_factor = 2 # The smallest possible prime factor.
    max_factor = sqrt(n) # we don't have to search past sqrt(n).

    while test_factor <= max_factor:
        if n%test_factor == 0:
            return test_factor
        test_factor = test_factor + 1  # This could be sped up.

    return n  # If we didn't find a factor up to sqrt(n), n itself is prime!

def decompose(N):
    '''
    Gives the unique prime decomposition of a positive integer N,
    as a dictionary with primes as keys and exponents as values.
    '''
    current_number = N  # We'll divide out factors from current_number until we get 1.
    decomp = {} # An empty dictionary to start.
    while current_number > 1:
        p = smallest_factor(current_number) # The smallest prime factor of the current number.
        if p in decomp.keys():  # Is p already in the list of keys?
            decomp[p] = decomp[p] + 1 # Increase the exponent (value with key p) by 1.
        else:  # "else" here means "if p is not in decomp.keys()".
            decomp[p] = 1  # Creates a new entry in the dictionary, with key p and value 1.
        current_number = current_number // p # Factor out p.
    return decomp

def mult_function(f_pp):
    '''
    When a function f_pp(p,e) of two arguments is input,
    this outputs a multiplicative function obtained from f_pp
    via prime decomposition.
    '''
    def f(n):
        D = decompose(n)
        result = 1
        for p in D:
            result = result * f_pp(p, D[p])
        return result

    return f


I am working on the file "P3wNT Notebook 7.ipynb".
The first task is:
This should be much faster than the previous brute-force computation of the totient.
Modular roots
A consequence of Euler's theorem is that -- depending on the modulus -- some exponentiation can be "reversed" by another exponentiation, a form of taking a "root" in modular arithmetic.  For example, if we work modulo $100$, and $GCD(a,100) = 1$, then Euler's theorem states that
$$a^{40} \equiv 1 \text{ mod } 100.$$
It follows that $a^{80} \equiv 1$ and $a^{81} \equiv a$, modulo $100$.  Expanding this, we find
$$a \equiv a^{81} = a^{3 \cdot 27} = (a^3)^{27} \text{ mod } 100.$$
What all this computation shows is that "raising to the 27th power" is like "taking the cube root", modulo 100 (and with appropriate bases).
Can you write Python code for it?

for b in range(20):
    b_cubed = pow(b,3,100)
    bb = pow(b_cubed,27,100)
    print(b, b_cubed, bb, GCD(b,100) == 1)


I am working on the file "P3wNT Notebook 7.ipynb".
The first task is:
To decrypt the ciphertext, we need to "undo" the operation of raising to the $e$-th power modulo $N$.  We must, effectively, take the $e$-th root of the ciphertext, modulo $N$.  This is what we studied earlier in this notebook.  Namely, if $c \equiv m^e \text{ mod } N$ is the ciphertext, and $ef \equiv 1$ modulo $\phi(N)$, then
$$c^f \equiv m^{ef} \equiv m \text{ mod } N.$$
So we must raise the ciphertext to the $f$ power, where $f$ is the multiplicative inverse of $e$ modulo $\phi(N)$.  Given a giant number $N$, it is difficult to compute the totient $\phi(N)$.  But, with the private key $p$ and $q$ (primes), the fact that $N = pq$ implies
$$\phi(N) = (p-1) \cdot (q-1) = pq - p - q + 1 = N - p - q + 1.$$
Armed with the private key (and the public key, which everyone has), we can decrypt a message in just a few lines of Python code.
Can you write Python code for it?

def RSA_decrypt(ciphertext, p,q,N,e):
    '''
    Decrypts message, using the private key (p,q)
    and the public key (N,e).  We allow the public key N as
    an input parameter, to avoid recomputing it.
    '''
    tot = N - (p+q) + 1
    f = mult_inverse(e,tot)  # This uses the Euclidean algorithm... very quick!
    return pow(ciphertext,f,N)

RSA_decrypt(c,p,q,N,e)  # We decrypt the ciphertext... what is the result?


I am working on the file "P3wNT Notebook 7.ipynb".
The first task is:
When $p^e$ is a prime power, the numbers among $1, \ldots, p^e$ are coprime to $p^e$ precisely when they are not multiples of $p$.  Therefore, the totient of a prime power is pretty easy to compute:
$$\phi(p^e) = p^e - p^{e-1} = p^{e-1} (p-1).$$
We implement this, and use the multiplicative function code to complete the implementation of the totient.
Can you write Python code for it?

def totient_pp(p,e):
    return (p**(e-1)) * (p-1)


I am working on the file "P4.1_FlopyIntro-Toth_Flowsystem.ipynb".
The first task is:
Create the MODFLOW Model Object
Create a flopy MODFLOW object: flopy.modflow.Modflow.
Can you write Python code for it?

# Assign name and create modflow model object
modelname = 'P4-1'
exe_name = os.path.join(datapath, 'mf2005')
print 'Model executable: ', exe_name
MF = flopy.modflow.Modflow(modelname, exe_name=exe_name, model_ws=modelpath)


I am working on the file "P4.1_FlopyIntro-Toth_Flowsystem.ipynb".
The first task is:
Define the Model Extent, Grid Resolution, and Characteristics
It is normally good practice to group things that you might want to change into a single code block.  This makes it easier to make changes and rerun the code.
Can you write Python code for it?

# model domain and grid definition
# for clarity, user entered variables are all caps; python syntax are lower case or mixed case
# we will use a layer orientation profile for easy plotting (see Box 4.2 on page 126)
LX = 200.
LY = 100.
ZTOP = 1.  # the "thickness" of the profile will be 1 m (= ZTOP - ZBOT)
ZBOT = 0.
NLAY = 1
NROW = 5
NCOL = 10
DELR = LX / NCOL  # recall that MODFLOW convention is DELR is along a row, thus has items = NCOL; see page XXX in AW&H (2015)
DELC = LY / NROW  # recall that MODFLOW convention is DELC is along a column, thus has items = NROW; see page XXX in AW&H (2015)
DELV = (ZTOP - ZBOT) / NLAY
BOTM = np.linspace(ZTOP, ZBOT, NLAY + 1)
HK = 10.
VKA = 1.
print "DELR =", DELR, "  DELC =", DELC, '  DELV =', DELV
print "BOTM =", BOTM


I am working on the file "P4.1_FlopyIntro-Toth_Flowsystem.ipynb".
The first task is:
Discretization Package
Create a flopy discretization package object: flopy.modflow.ModflowDis.
Can you write Python code for it?

# Create the discretization object
TOP = np.ones((NROW, NCOL),dtype=np.float)

DIS_PACKAGE = flopy.modflow.ModflowDis(MF, NLAY, NROW, NCOL, delr=DELR, delc=DELC,
                               top=TOP, botm=BOTM[1:], laycbd=0)
# print DIS_PACKAGE uncomment this to see information about the flopy object


I am working on the file "P4.1_FlopyIntro-Toth_Flowsystem.ipynb".
The first task is:
Post Processing the Results
To read heads from the MODFLOW binary output file, we can use the flopy.utils.binaryfile module. Specifically, we can use the HeadFile object from that module to extract head data arrays.
Can you write Python code for it?

#imports for plotting and reading the MODFLOW binary output file
import matplotlib.pyplot as plt
import flopy.utils.binaryfile as bf

#Create the headfile object and grab the results for last time.
headfile = os.path.join(modelpath, modelname + '.hds')
headfileobj = bf.HeadFile(headfile)

#Get a list of times that are contained in the model
times = headfileobj.get_times()
print 'Headfile (' + modelname + '.hds' + ') contains the following list of times: ', times

#Get a numpy array of heads for totim = 1.0
#The get_data method will extract head data from the binary file.
HEAD = headfileobj.get_data(totim=1.0)

#Print statistics on the head
print 'Head statistics'
print '  min: ', HEAD.min()
print '  max: ', HEAD.max()
print '  std: ', HEAD.std()

#Create a contour plot of heads
FIG = plt.figure(figsize=(15,15))

#setup contour levels and plot extent
LEVELS = np.arange(100, 109, 0.5)
EXTENT = (DELR/2., LX - DELR/2., DELC/2., LY - DELC/2.)
print 'Contour Levels: ', LEVELS
print 'Extent of domain: ', EXTENT

#Make a contour plot on the first axis
AX1 = FIG.add_subplot(1, 2, 1, aspect='equal')
AX1.contour(np.flipud(HEAD[0, :, :]), levels=LEVELS, extent=EXTENT)

#Make a color flood on the second axis
AX2 = FIG.add_subplot(1, 2, 2, aspect='equal')
cax = AX2.imshow(HEAD[0, :, :], extent=EXTENT, interpolation='nearest')
cbar = FIG.colorbar(cax, orientation='vertical', shrink=0.25)


I am working on the file "P4.1_FlopyIntro-Toth_Flowsystem.ipynb".
The first task is:
Preconditioned Conjugate Gradient Solver
Create a flopy pcg package object: flopy.modflow.ModflowPcg.
Can you write Python code for it?

PCG_PACKAGE = flopy.modflow.ModflowPcg(MF) # we'll use the defaults for the PCG solver
# print PCG_PACKAGE  # uncomment this at far left to see the information about the flopy PCG object


I am working on the file "P4.1_FlopyIntro-Toth_Flowsystem.ipynb".
The first task is:
Writing the MODFLOW Input Files
Before we create the model input datasets, we can do some directory cleanup to make sure that we don't accidently use old files.
Can you write Python code for it?

#Before writing input, destroy all files in folder
#This will prevent us from reading old results
modelfiles = os.listdir(modelpath)
for filename in modelfiles:
    f = os.path.join(modelpath, filename)
    if modelname in f:
        try:
            os.remove(f)
            print 'Deleted: ', filename
        except:
            print 'Unable to delete: ', filename

#Now write the model input files
MF.write_input()


I am working on the file "P4.1_FlopyIntro-Toth_Flowsystem.ipynb".
The first task is:
Yup.  It's that simple, the model datasets are written using a single command (mf.write_input).
Check in the model working directory and verify that the input files have been created.  Or if you might just add another cell, right after this one, that prints a list of all the files in our model directory.  The path we are working in is returned from this next block.
Can you write Python code for it?

# return current working directory
modelpath


I am working on the file "P4.2_Flopy_dam_cross_section.ipynb".
The first task is:
Basic Package
Create a flopy basic package object: flopy.modflow.ModflowBas.
Can you write Python code for it?

# Variables for the BAS package
IBOUND = np.ones((NLAY, NROW, NCOL), dtype=np.int32)  # all nodes are active (IBOUND = 1)

# make the top of the profile specified head by setting the IBOUND = -1
IBOUND[:, 0, 0:81] = -1  #don't forget arrays are zero-based!
IBOUND[:, 0, 121:200] = -1  #don't forget arrays are zero-based!
print IBOUND

STRT = 40 * np.ones((NLAY, NROW, NCOL), dtype=np.float32)  # set starting head to 40 through out model domain
STRT[:, 0, 0:81] = 76.5       # pool elevation upstream of the dam
STRT[:, 0, 121:200] = 36.5     # pool elevation downstream of the dam
print STRT

BAS_PACKAGE = flopy.modflow.ModflowBas(MF, ibound=IBOUND, strt=STRT)
# print BAS_PACKAGE  # uncomment this at far left to see the information about the flopy BAS object


I am working on the file "P4.2_Flopy_dam_cross_section.ipynb".
The first task is:
Define the Model Extent, Grid Resolution, and Characteristics
It is normally good practice to group things that you might want to change into a single code block.  This makes it easier to make changes and rerun the code.
Can you write Python code for it?

# model domain and grid definition
# for clarity, user entered variables are all caps; python syntax are lower case or mixed case
# we will use a layer orientation profile for easy plotting (see Box 4.2 on page 126)
LX = 200.
LY = 26.5
ZTOP = 1.  # the "thickness" of the profile will be 1 m (= ZTOP - ZBOT)
ZBOT = 0.
NLAY = 1
NROW = 53
NCOL = 200
DELR = LX / NCOL  # recall that MODFLOW convention is DELR is along a row, thus has items = NCOL; see page XXX in AW&H (2015)
DELC = LY / NROW  # recall that MODFLOW convention is DELC is along a column, thus has items = NROW; see page XXX in AW&H (2015)
DELV = (ZTOP - ZBOT) / NLAY
BOTM = np.linspace(ZTOP, ZBOT, NLAY + 1)
HK = 10.
VKA = 1.
print "DELR =", DELR, "  DELC =", DELC, '  DELV =', DELV
print "BOTM =", BOTM


I am working on the file "P4.2_Flopy_dam_cross_section.ipynb".
The first task is:
Discretization Package
Create a flopy discretization package object: flopy.modflow.ModflowDis.
Can you write Python code for it?

# Create the discretization object
TOP = np.ones((NROW, NCOL),dtype=np.float)

DIS_PACKAGE = flopy.modflow.ModflowDis(MF, NLAY, NROW, NCOL, delr=DELR, delc=DELC,
                               top=TOP, botm=BOTM[1:], laycbd=0)
# print DIS_PACKAGE  #uncomment this on far left to see information about the flopy object


I am working on the file "P4.2_Flopy_dam_cross_section.ipynb".
The first task is:
Layer Property Flow Package
Create a flopy layer property flow package object: flopy.modflow.ModflowLpf.
Can you write Python code for it?

LPF_PACKAGE = flopy.modflow.ModflowLpf(MF, hk=HK, vka=VKA)  # we defined the K and anisotropy at top of file
# print LPF_PACKAGE  # uncomment this at far left to see the information about the flopy LPF object


I am working on the file "P4.2_Flopy_dam_cross_section.ipynb".
The first task is:
Output Control
Create a flopy output control object: flopy.modflow.ModflowOc.
Can you write Python code for it?

OC_PACKAGE = flopy.modflow.ModflowOc(MF)   # we'll use the defaults for the model output
# print OC_PACKAGE  # uncomment this at far left to see the information about the flopy OC object


I am working on the file "P4.2_Flopy_dam_cross_section.ipynb".
The first task is:
Preconditioned Conjugate Gradient Solver
Create a flopy pcg package object: flopy.modflow.ModflowPcg.
Can you write Python code for it?


I am working on the file "P4.2_Flopy_dam_cross_section.ipynb".
The first task is:
Setup a New Directory and Change Paths
For this tutorial, we will work in the 21_FlopyIntro directory, which is located up one folder and over to the Data folder.  We can use some fancy Python tools to help us manage the directory creation.  Note that if you encounter path problems with this workbook, you can stop and then restart the kernel and the paths will be reset.
Can you write Python code for it?

# Set the name of the path to the model working directory
dirname = "P4-2_DamProfile"
datapath = os.getcwd()
modelpath = os.path.join(datapath, dirname)
print 'Name of model path: ', modelpath

# Now let's check if this directory exists.  If not, then we will create it.
if os.path.exists(modelpath):
    print 'Model working directory already exists.'
else:
    print 'Creating model working directory.'
    os.mkdir(modelpath)


I am working on the file "P4.2_Flopy_dam_cross_section.ipynb".
The first task is:
Writing the MODFLOW Input Files
Before we create the model input datasets, we can do some directory cleanup to make sure that we don't accidently use old files.
Can you write Python code for it?


I am working on the file "P4.2_Flopy_dam_cross_section.ipynb".
The first task is:
Yup.  It's that simple, the model datasets are written using a single command (mf.write_input).
Check in the model working directory and verify that the input files have been created.  Or if you might just add another cell, right after this one, that prints a list of all the files in our model directory.  The path we are working in is returned from this next block.
Can you write Python code for it?


I am working on the file "P4.3_Flopy_Hubbertville_areal_model.ipynb".
The first task is:
Basic Package
Create a flopy basic package object: flopy.modflow.ModflowBas.
Can you write Python code for it?

# Variables for the BAS package
IBOUND = np.ones((NLAY, NROW, NCOL), dtype=np.int32)  # all nodes are active (IBOUND = 1)

# make the top of the profile specified head by setting the IBOUND = -1
IBOUND[:, 0, :] = -1  #don't forget arrays are zero-based!
IBOUND[:, -1, :] = -1  #-1 is Python for last in array
print IBOUND

STRT = 1015 * np.ones((NLAY, NROW, NCOL), dtype=np.float32)  # set starting head to 1010 m through out model domain
STRT[:, 0, :] = 1000.       # river stage for setting constant head
STRT[:, -1, :] = 1000.     # wetland stage for setting constant head
print STRT

BAS_PACKAGE = flopy.modflow.ModflowBas(MF, ibound=IBOUND, strt=STRT)
# print BAS_PACKAGE  # uncomment this at far left to see the information about the flopy BAS object


I am working on the file "P4.3_Flopy_Hubbertville_areal_model.ipynb".
The first task is:
Create the MODFLOW Model Object
Create a flopy MODFLOW object: flopy.modflow.Modflow.
Can you write Python code for it?

# Assign name and create modflow model object
modelname = 'P4-3'
#exe_name = os.path.join(datapath, 'mf2005.exe') # for Windows OS
exe_name = os.path.join(datapath, 'mf2005')     # for Mac OS
print 'Model executable: ', exe_name
MF = flopy.modflow.Modflow(modelname, exe_name=exe_name, model_ws=modelpath)


I am working on the file "P4.3_Flopy_Hubbertville_areal_model.ipynb".
The first task is:
Output Control
Create a flopy output control object: flopy.modflow.ModflowOc.
Can you write Python code for it?


I am working on the file "P4.3_Flopy_Hubbertville_areal_model.ipynb".
The first task is:
P4.3 Part d.
Run the model in part (c) again but this time assume the width of the river is 5 m. What is the effect of reducing the width of the river?
Can you write Python code for it?

WIDTH_RIVER = 5.
#area is the nodal area, DELR x DELC which was entered above
print "DELR = ", DELR
print "River width =", WIDTH_RIVER
print "River area in node =", DELR * WIDTH_RIVER
#conductance is leakance x area
print 'River Leakance = ', RIV_LEAKANCE
RIV_COND = RIV_LEAKANCE * DELR * WIDTH_RIVER
print 'River Conductance =', RIV_COND

#We enter RIV Package data by "layer-row-column-data" = lrcd
stress_period_data = [
                      [0, 0, 0, RIV_STAGE, RIV_COND, SED_BOT_RIVER],   #layer, row, column, stage conductance, river bottom
                      [0, 0, 1, RIV_STAGE, RIV_COND, SED_BOT_RIVER],   #remember Python indexing is zero based
                      [0, 0, 2, RIV_STAGE, RIV_COND, SED_BOT_RIVER],
                      [0, 0, 3, RIV_STAGE, RIV_COND, SED_BOT_RIVER],
                      [0, 0, 4, RIV_STAGE, RIV_COND, SED_BOT_RIVER],
                      [0, 0, 5, RIV_STAGE, RIV_COND, SED_BOT_RIVER],
                      [0, 0, 6, RIV_STAGE, RIV_COND, SED_BOT_RIVER],
                      [0, 0, 7, RIV_STAGE, RIV_COND, SED_BOT_RIVER],
                      [0, 0, 8, RIV_STAGE, RIV_COND, SED_BOT_RIVER]]

print stress_period_data

riv = flopy.modflow.ModflowRiv(MF, stress_period_data=stress_period_data)
#delete old files to prevent us from reading old results
modelfiles = os.listdir(modelpath)
for filename in modelfiles:
    f = os.path.join(modelpath, filename)
    if modelname in f:
        try:
            os.remove(f)
            print 'Deleted: ', filename
        except:
            print 'Unable to delete: ', filename

#Now write the model input files and rerun MODFLOW
MF.write_input()
print "New MODFLOW input files = ", modelfiles
print "You can check the newly created files in", modelpath
#rerun MODFLOW-2005
silent = False  #Print model output to screen?
pause = False   #Require user to hit enter? Doesn't mean much in Ipython notebook
report = True   #Store the output from the model in buff
success, buff = MF.run_model(silent=silent, pause=pause,...


I am working on the file "P4.3_Flopy_Hubbertville_areal_model.ipynb".
The first task is:
Post Processing the Results
To read heads from the MODFLOW binary output file, we can use the flopy.utils.binaryfile module. Specifically, we can use the HeadFile object from that module to extract head data arrays.
Can you write Python code for it?

#imports for plotting and reading the MODFLOW binary output file
import matplotlib.pyplot as plt
import flopy.utils.binaryfile as bf

#Create the headfile object and grab the results for last time.
headfile = os.path.join(modelpath, modelname + '.hds')
headfileobj = bf.HeadFile(headfile)

#Get a list of times that are contained in the model
times = headfileobj.get_times()
print 'Headfile (' + modelname + '.hds' + ') contains the following list of times: ', times

#Get a numpy array of heads for totim = 1.0
#The get_data method will extract head data from the binary file.
HEAD = headfileobj.get_data(totim=1.0)

#Print statistics on the head
print 'Head statistics'
print '  min: ', HEAD.min()
print '  max: ', HEAD.max()
print '  std: ', HEAD.std()

#Create a contour plot of heads
FIG = plt.figure(figsize=(15,13))

#setup contour levels and plot extent
LEVELS = np.arange(1000., 1011., 0.5)
EXTENT = (DELR/2., LX - DELR/2., DELC/2., LY - DELC/2.)
print 'Contour Levels: ', LEVELS
print 'Extent of domain: ', EXTENT

#Make a contour plot on the first axis
AX1 = FIG.add_subplot(1, 2, 1, aspect='equal')
AX1.set_xlabel("x")
AX1.set_ylabel("y")
YTICKS = np.arange(0, 11000, 500)
AX1.set_yticks(YTICKS)
AX1.set_title("Hubbertville contour map")
AX1.text(2000, 10500, r"River", fontsize=10, color="blue")
AX1.text(1800, 340, r"Green Swamp", fontsize=10, color="green")
AX1.contour(np.flipud(HEAD[0, :, :]), levels=LEVELS, extent=EXTENT)

#Make a color flood on the second axis
AX2 = FIG.add_subplot(1, 2, 2, aspect='equal')
AX2.set_xlabel("x")
AX2.set_ylabel("y")
AX2.set_yticks(YTICKS)
AX2.set_title("Hubbertville color flood")
AX2.text(2000, 10500, r"River", fontsize=10, color="black")
AX2.text(1800, 340, r"Green Swamp", fontsize=10, color="black")
cax = AX2.imshow(HEAD[0, :, :], extent=EXTENT, interpolation='nearest')
cbar = FIG.colorbar(cax, orientation='vertical', shrink=0.45)


I am working on the file "P4.3_Flopy_Hubbertville_areal_model.ipynb".
The first task is:
Preconditioned Conjugate Gradient Solver
Create a flopy pcg package object: flopy.modflow.ModflowPcg.
Can you write Python code for it?

PCG_PACKAGE = flopy.modflow.ModflowPcg(MF, mxiter=500, iter1=100, hclose=1e-04, rclose=1e-03, relax=0.98, damp=0.5)
# print PCG_PACKAGE  # uncomment this at far left to see the information about the flopy PCG object


I am working on the file "P4.3_Flopy_Hubbertville_areal_model.ipynb".
The first task is:
Running the Model
Flopy has several methods attached to the model object that can be used to run the model.  They are run_model, run_model2, and run_model3.  Here we use run_model3, which will write output to the notebook.
Can you write Python code for it?

silent = False  #Print model output to screen?
pause = False   #Require user to hit enter? Doesn't mean much in Ipython notebook
report = True   #Store the output from the model in buff
success, buff = MF.run_model(silent=silent, pause=pause, report=report)


I am working on the file "P4.3_Flopy_Hubbertville_areal_model.ipynb".
The first task is:
Why when the gw divide is simulated as a no-flow might the total flux to Green Swamp be different (and lower)?
P4.3 Part c.
Run the model in part (a) again but this time use a HDB to represent the river. The stage of the river is 1000 m and the width is 500 m. The vertical hydraulic conductivity of the riverbed sediments is 5 m/day and the thickness of the sediments is 1 m. The elevation of the bottom of the sediments is 995 m. Compare results with those in part (a).
Can you write Python code for it?

#We have to recreate the IBOUND and STRT heads of Part a.
#This is just copied directly from Part a. above to start clean
# Variables for the BAS package
IBOUND = np.ones((NLAY, NROW, NCOL), dtype=np.int32)  # all nodes are active (IBOUND = 1)

# make the top of the profile specified head by setting the IBOUND = -1
IBOUND[:, 0, :] = -1  #don't forget arrays are zero-based!
IBOUND[:, -1, :] = -1  #-1 is Python for last in array
print IBOUND

#BUT in Part c. the river to the north is a head-dependent BCs, not specified head
#so we have to change it from -1 (specified head) to 1 (active cells)
IBOUND[:, 0, :] = 1  #don't forget arrays are zero-based!
print IBOUND

#In the same way, we need to reset the Starting Head array, but only need
# values set for the specified head boundary used for Green Swamp in the south
STRT = 1015 * np.ones((NLAY, NROW, NCOL), dtype=np.float32)  # set starting head to 1010 m through out model domain
STRT[:, -1, :] = 1000.     # wetland stage for setting constant head
print STRT

#we have to update the MODFLOW's BAS Package with the new IBOUND and STRT heads
BAS_PACKAGE = flopy.modflow.ModflowBas(MF, ibound=IBOUND, strt=STRT)

#now we need to add a HDB - the RIV Package is a good choice
#recall that a RIV node has a river stage, a conductance, and a bottom elevation
RIV_STAGE = 1000.
Kv_RIVER = 5.
b_RIVER = 1.
WIDTH_RIVER = 500.
SED_BOT_RIVER = 995.
# conductance = leakance x cross-sectional area
# leakance = Kv/b
RIV_LEAKANCE = Kv_RIVER / b_RIVER
print "River sediment leakance =", RIV_LEAKANCE

#area is the nodal area, DELR x DELC which was entered above
print "DELR = ", DELR
print "River width =", WIDTH_RIVER
print "River area in node =", DELR * WIDTH_RIVER

#conductance is leakance x area
RIV_COND = RIV_LEAKANCE * DELR * WIDTH_RIVER
print 'River Conductance =', RIV_COND

#We enter RIV Package data by "layer-row-column-data" = lrcd
stress_period_data = [
                      [0, 0, 0, RIV_STAGE, RIV_COND, SED_BOT_RIVER],   #layer, row, column, stage conductance, river bottom
  ...


I am working on the file "P4.3_Flopy_Hubbertville_areal_model.ipynb".
The first task is:
Yup.  It's that simple, the model datasets are written using a single command (mf.write_input).
Check in the model working directory and verify that the input files have been created.  Or if you might just add another cell, right after this one, that prints a list of all the files in our model directory.  The path we are working in is returned from this next block.
Can you write Python code for it?

# return current working directory
print "You can check the newly created files in", modelpath


I am working on the file "P4.4_Flopy_Hubbertville_areal_model_with_pumping.ipynb".
The first task is:
Basic Package
Create a flopy basic package object: flopy.modflow.ModflowBas.
Can you write Python code for it?

# Variables for the BAS package
IBOUND = np.ones((NLAY, NROW, NCOL), dtype=np.int32)  # all nodes are active (IBOUND = 1)

# make the top of the profile specified head by setting the IBOUND = -1
IBOUND[:, 0, :] = -1  #don't forget arrays are zero-based!
IBOUND[:, -1, :] = -1  #-1 is Python for last in array
print IBOUND

STRT = 1010 * np.ones((NLAY, NROW, NCOL), dtype=np.float32)  # set starting head to 1010 m through out model domain
STRT[:, 0, :] = 1000.       # river stage for setting constant head
STRT[:, -1, :] = 1000.     # wetland stage for setting constant head
print STRT

BAS_PACKAGE = flopy.modflow.ModflowBas(MF, ibound=IBOUND, strt=STRT)
# print BAS_PACKAGE  # uncomment this at far left to see the information about the flopy BAS object


I am working on the file "P4.4_Flopy_Hubbertville_areal_model_with_pumping.ipynb".
The first task is:
Discretization Package
Create a flopy discretization package object: flopy.modflow.ModflowDis.
Can you write Python code for it?

# Create the discretization object
TOP = ZTOP * np.ones((NROW, NCOL),dtype=np.float)

DIS_PACKAGE = flopy.modflow.ModflowDis(MF, NLAY, NROW, NCOL, delr=DELR, delc=DELC,
                               top=TOP, botm=BOTM[1:], laycbd=0)
# print DIS_PACKAGE  #uncomment this on far left to see information about the flopy object


I am working on the file "P4.4_Flopy_Hubbertville_areal_model_with_pumping.ipynb".
The first task is:
Newton-Raphson Solver
Create a flopy pcg package object: flopy.modflow.ModflowNwt.
Can you write Python code for it?

NWT_PACKAGE = flopy.modflow.ModflowNwt(MF, headtol=0.001, fluxtol=5000, maxiterout=900)
# print NWT_PACKAGE  # uncomment this at far left to see the information about the flopy PCG object


I am working on the file "P4.4_Flopy_Hubbertville_areal_model_with_pumping.ipynb".
The first task is:
Output Control
Create a flopy output control object: flopy.modflow.ModflowOc.
Can you write Python code for it?


I am working on the file "P4.4_Flopy_Hubbertville_areal_model_with_pumping.ipynb".
The first task is:
P4.4 Part b
Compare groundwater discharge to the swamp under the prepumping scenario in Problem P4.3 with the results under the pumping scenarios. In light of the modeling results, consider what might be meant by “significantly reduced” as used by the state agency (see discussion in Problem P4.3). Make a list of physical, geochemical and ecological conditions that potentially could be affected by a change in groundwater flow to the Green Swamp.
Can you write Python code for it?

#let's compare the three formulations:
#1) gw divide simulated; 2) gw divide as no flow BC; and 3) gw divide as specified head BC
print "P4.3 Flux to Swamp (no pumping/simulated/specified head) =", 22570
print "Correct New Flux to Swamp (pumping/simulated divide) =", FLUX_TO_SWAMP
print "Correct percent flux reduction =", 100 * (FLUX_TO_SWAMP-22570)/22570
print ""
print "Incorrect New Flux to Swamp (pumping/spec head divide) = ", FLUX_TO_SWAMP_SPEC_HEAD
print "Incorrect percent flux reduction =", 100 * (FLUX_TO_SWAMP_SPEC_HEAD-22570)/22570
print ""
print "P4.3 Flux to Swamp (no pumping/no flow divide) =", 20302
print "Incorrect New Flux to Swamp (pumping/no flow divide) = ", FLUX_TO_SWAMP_NO_FLOW
print "Incorrect percent flux reduction =", 100 * (FLUX_TO_SWAMP_NO_FLOW-20302)/20302


I am working on the file "P4.4_Flopy_Hubbertville_areal_model_with_pumping.ipynb".
The first task is:
Setup a New Directory and Change Paths
For this tutorial, we will work in a new subdirectory underneath the directory where the notebook is located. We can use some fancy Python tools to help us manage the directory creation. Note that if you encounter path problems with this workbook, you can stop and then restart the kernel and the paths will be reset.
Can you write Python code for it?

# Set the name of the path to the model working directory
dirname = "P4-4_Hubbertville"
datapath = os.getcwd()
modelpath = os.path.join(datapath, dirname)
print 'Name of model path: ', modelpath

# Now let's check if this directory exists.  If not, then we will create it.
if os.path.exists(modelpath):
    print 'Model working directory already exists.'
else:
    print 'Creating model working directory.'
    os.mkdir(modelpath)


I am working on the file "P4.4_Flopy_Hubbertville_areal_model_with_pumping.ipynb".
The first task is:
Testing your Skills

Is the total volumetric flux out near that reported in the MODFLOW list file?

Experiment with horizontal grid resolution, well location, recharge, pumping rate, and aquifer characteristics.  Rerun the model and post process to evaluate the effects.

P4.3 Part a.  GW divide as No-flow and Specified Head BCs
Using the steady-state heads derived in P4.3, run the model using first a no-flow boundary and then a specified head boundary at the location of the groundwater divide. Compare results with those in part (a). Compute the discharge to Green Swamp under each representation. What is the effect of assigning an internal boundary on the results?
GW divide as No-flow BC
Can you write Python code for it?

# recall from P4.3 that the GW divide was located at Rows 10 and 11;
# where each head value in the column was 1005.50
#like P4.3, lets set Rows 10 and 11 to no flow
#(set that row to 0 in the MODFLOW IBOUND array)
IBOUND[:, 10, :] = 0
IBOUND[:, 11, :] = 0
print IBOUND

#we have to update the MODFLOW's BAS Package with the new IBOUND array
BAS_PACKAGE = flopy.modflow.ModflowBas(MF, ibound=IBOUND, strt=STRT)

#Before writing input, destroy all files in folder to prevent reusing old files
#Here's the working directory
print modelpath
#Here's what's currently in the working directory
modelfiles = os.listdir(modelpath)
print modelfiles

#delete these files to prevent us from reading old results
modelfiles = os.listdir(modelpath)
for filename in modelfiles:
    f = os.path.join(modelpath, filename)
    if modelname in f:
        try:
            os.remove(f)
            print 'Deleted: ', filename
        except:
            print 'Unable to delete: ', filename

#Now write the model input files
MF.write_input()
print "New MODFLOW input files = ", modelfiles
print "You can check the newly created files in", modelpath

#rerun MODFLOW-2005
silent = False  #Print model output to screen?
pause = False   #Require user to hit enter? Doesn't mean much in Ipython notebook
report = True   #Store the output from the model in buff
success, buff = MF.run_model(silent=silent, pause=pause, report=report)

#As before, let's look at the results and compare to P4-3 Part a.
#imports for plotting and reading the MODFLOW binary output file
import matplotlib.pyplot as plt
import flopy.utils.binaryfile as bf

#Create the headfile object and grab the results for last time.
headfile = os.path.join(modelpath, modelname + '.hds')
headfileobj = bf.HeadFile(headfile)

#Get a list of times that are contained in the model
times = headfileobj.get_times()
print 'Headfile (' + modelname + '.hds' + ') contains the following list of times: ', times

#Get a numpy array of heads for totim = 1.0
#The get_data method will extract head data from...


I am working on the file "P4.4_Flopy_Hubbertville_areal_model_with_pumping.ipynb".
The first task is:
Upstream Weighting Package (like Layer Property Flow Package)
Create a flopy layer property flow package object: flopy.modflow.ModflowUpw.
Can you write Python code for it?

LPF_PACKAGE = flopy.modflow.ModflowUpw(MF, laytyp=1, hk=HK, vka=VKA)  # we defined the K and anisotropy at top of file
# print LPF_PACKAGE  # uncomment this at far left to see the information about the flopy LPF object


I am working on the file "P4.4_Flopy_Hubbertville_areal_model_with_pumping.ipynb".
The first task is:
Well Package
Create a flopy output control object: flopy.modflow.ModflowWel.
Can you write Python code for it?

WEL_PACKAGE = flopy.modflow.ModflowWel(MF, stress_period_data=[0,4,4,WELLQ])  # remember python 0 index, layer 0 = layer 1 in MF
#print WEL_PACKAGE  # uncomment this at far left to see the information about the flopy WEL object


I am working on the file "P4.4_Flopy_Hubbertville_areal_model_with_pumping.ipynb".
The first task is:
Writing the MODFLOW Input Files
Before we create the model input datasets, we can do some directory cleanup to make sure that we don't accidently use old files.
Can you write Python code for it?

#Before writing input, destroy all files in folder to prevent reusing old files
#Here's the working directory
print modelpath
#Here's what's currently in the working directory
modelfiles = os.listdir(modelpath)
print modelfiles
#delete these files to prevent us from reading old results
modelfiles = os.listdir(modelpath)
for filename in modelfiles:
    f = os.path.join(modelpath, filename)
    if modelname in f:
        try:
            os.remove(f)
            print 'Deleted: ', filename
        except:
            print 'Unable to delete: ', filename

#Now write the model input files
MF.write_input()


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Create the MODFLOW Model Object
Create a flopy MODFLOW object: flopy.modflow.Modflow.
Can you write Python code for it?

# Assign name and create modflow model object
modelname = 'P4-5'
#exe_name = os.path.join(datapath, 'mfnwt.exe') # for Windows OS
exe_name = os.path.join(datapath, 'mfnwt')     # for Mac OS
print 'Model executable: ', exe_name
MF = flopy.modflow.Modflow(modelname, exe_name=exe_name, model_ws=modelpath)


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Define the Model Extent, Grid Resolution, and Characteristics
It is normally good practice to group things that you might want to change into a single code block.  This makes it easier to make changes and rerun the code.
Can you write Python code for it?

# model domain and grid definition
# for clarity, user entered variables are all caps; python syntax are lower case or mixed case
# In a contrast to P4.1 and P4.2, this is an areal 2D model
LX = 4500.
LY = 11000.   # note that there is an added 500m on the top and bottom to represent the boundary conditions,that leaves an aqufier lenght of 10000 m
ZTOP = 1030.  # the system is unconfined so set the top above land surface so that the water table never > layer top
ZBOT = 980.
NLAY = 1
NROW = 22
NCOL = 9
DELR = LX / NCOL  # recall that MODFLOW convention is DELR is along a row, thus has items = NCOL; see page XXX in AW&H (2015)
DELC = LY / NROW  # recall that MODFLOW convention is DELC is along a column, thus has items = NROW; see page XXX in AW&H (2015)
DELV = (ZTOP - ZBOT) / NLAY
BOTM = np.linspace(ZTOP, ZBOT, NLAY + 1)
HK = 50.
VKA = 1.
RCH = 0.001
WELLQ = 0.   #recall MODFLOW convention, negative means pumped out of the model domain (=aquifer)
print "DELR =", DELR, "  DELC =", DELC, '  DELV =', DELV
print "BOTM =", BOTM
print "Recharge =", RCH
print "Pumping well rate =", WELLQ


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Discretization Package
Create a flopy discretization package object: flopy.modflow.ModflowDis.
Can you write Python code for it?


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
How do the two versions of starting head compare?  What does this conceptualization need to keep this problem from being a relative result?
Part b.
Take one of the models you designed in part (a) and replace one constant flux node on either the river or swamp boundary with a specified head node equal to 1000 m. Run the model to steady state. Compare the results with those in
part (a) and with P4.3a. Explain the differences.
Can you write Python code for it?

# let's use the first model of Part a so that the starting head of 1000 m is not below the bottom of the aquifer
TOP = 1030.
BOT = 980.

#we'll set the cell in the first row and first column to a specified head (changing it's IBOUND value to -1)
IBOUND[:, 0, 0] = -1
print IBOUND

#change the starting head to equal 1000 m
STRT = 1000 * np.ones((NLAY, NROW, NCOL), dtype=np.float32)
print STRT

#we have to update MODFLOW's BAS Package with the new IBOUND and STRT heads
BAS_PACKAGE = flopy.modflow.ModflowBas(MF, ibound=IBOUND, strt=STRT)

print 'aquifer top =', TOP
print 'aquifer bottom =', BOT
#we have to update MODFLOW's DIS Package for the new tops and bottoms
DIS_PACKAGE = flopy.modflow.ModflowDis(MF, NLAY, NROW, NCOL, delr=DELR, delc=DELC,
                               top=TOP, botm=BOT, laycbd=0)

#delete old files to prevent us from reading old results
modelfiles = os.listdir(modelpath)
for filename in modelfiles:
    f = os.path.join(modelpath, filename)
    if modelname in f:
        try:
            os.remove(f)
            print 'Deleted: ', filename
        except:
            print 'Unable to delete: ', filename

#Now write the model input files
MF.write_input()
print "New MODFLOW input files = ", modelfiles
print "You can check the newly created files in", modelpath

#rerun MODFLOW-2005
silent = False  #Print model output to screen?
pause = False   #Require user to hit enter? Doesn't mean much in Ipython notebook
report = True   #Store the output from the model in buff
success, buff = MF.run_model(silent=silent, pause=pause, report=report)

#As before, let's look at the results and compare to P4-3 Part a.
#imports for plotting and reading the MODFLOW binary output file
import matplotlib.pyplot as plt
import flopy.utils.binaryfile as bf

#Create the headfile object and grab the results for last time.
headfile = os.path.join(modelpath, modelname + '.hds')
headfileobj = bf.HeadFile(headfile)

#Get a list of times that are contained in the model
times = headfileobj.get_times()
print 'Headfile (' +...


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Look at the bottom of the MODFLOW output file (ending with a *.list) and note the water balance reported.
Can you write Python code for it?

#look at the head in column = 4 from headobj, and then plot it
#print HEAD along a column; COL is a variable that allows us to change this easily
COL = 4
print HEAD[0,:,COL]

# we see this is what we want, but is flipped because MODFLOW's array does not = Python, so we reverse the order (flip them) and call it
Y = np.flipud(HEAD[0,:,COL])
print Y

#for our cross section create X-coordinates to match with heads
XCOORD = np.arange(0, 11000, 500) + 250
print XCOORD

fig = plt.figure(figsize=(10, 5))
ax = fig.add_subplot(1, 1, 1)
TITLE =  'cross section of head along Column = ({0})'.format(COL)
ax.set_title(TITLE)
ax.set_xlabel('y')
ax.set_ylabel('head')
ax.set_xlim(0, 11000.)
ax.set_ylim(980.,1020.)
ax.text(10480, 998, r"River", fontsize=10, color="blue",rotation='vertical')
ax.text(300, 998, r"Green Swamp", fontsize=10, color="green",rotation='vertical')
ax.text(5300,1009., r"Groundwater Divide", fontsize=10, color="black",rotation='vertical')
ax.plot(XCOORD, Y)


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Newton-Raphson Solver
Create a flopy pcg package object: flopy.modflow.ModflowNwt.
Can you write Python code for it?


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Output Control
Create a flopy output control object: flopy.modflow.ModflowOc.
Can you write Python code for it?

OC_PACKAGE = flopy.modflow.ModflowOc(MF)   # we'll use the defaults for the model output
# print OC_PACKAGE  # uncomment this at far left to see the information about the flopy OC object


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Post Processing the Results
To read heads from the MODFLOW binary output file, we can use the flopy.utils.binaryfile module. Specifically, we can use the HeadFile object from that module to extract head data arrays.
Can you write Python code for it?

#imports for plotting and reading the MODFLOW binary output file
import matplotlib.pyplot as plt
import flopy.utils.binaryfile as bf

#Create the headfile object and grab the results for last time.
headfile = os.path.join(modelpath, modelname + '.hds')
headfileobj = bf.HeadFile(headfile)

#Get a list of times that are contained in the model
times = headfileobj.get_times()
print 'Headfile (' + modelname + '.hds' + ') contains the following list of times: ', times

#Get a numpy array of heads for totim = 1.0
#The get_data method will extract head data from the binary file.
HEAD = headfileobj.get_data(totim=1.0)

#Print statistics on the head
print 'Head statistics'
print '  min: ', HEAD.min()
print '  max: ', HEAD.max()
print '  std: ', HEAD.std()

#Create a contour plot of heads
FIG = plt.figure(figsize=(15,13))

#setup contour levels and plot extent
LEVELS = np.arange(989., 1005., 0.5)
EXTENT = (DELR/2., LX - DELR/2., DELC/2., LY - DELC/2.)
print 'Contour Levels: ', LEVELS
print 'Extent of domain: ', EXTENT

#Make a contour plot on the first axis
AX1 = FIG.add_subplot(1, 2, 1, aspect='equal')
AX1.set_xlabel("x")
AX1.set_ylabel("y")
YTICKS = np.arange(0, 11000, 500)
AX1.set_yticks(YTICKS)
AX1.set_title("Hubbertville contour map")
AX1.text(2000, 10500, r"River", fontsize=10, color="blue")
AX1.text(1800, 340, r"Green Swamp", fontsize=10, color="green")
AX1.contour(np.flipud(HEAD[0, :, :]), levels=LEVELS, extent=EXTENT)

#Make a color flood on the second axis
AX2 = FIG.add_subplot(1, 2, 2, aspect='equal')
AX2.set_xlabel("x")
AX2.set_ylabel("y")
AX2.set_yticks(YTICKS)
AX2.set_title("Hubbertville color flood")
AX2.text(2000, 10500, r"River", fontsize=10, color="black")
AX2.text(1800, 340, r"Green Swamp", fontsize=10, color="black")
cax = AX2.imshow(HEAD[0, :, :], extent=EXTENT, interpolation='nearest')
cbar = FIG.colorbar(cax, orientation='vertical', shrink=0.45)


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Running the Model
Flopy has several methods attached to the model object that can be used to run the model.  They are run_model, run_model2, and run_model3.  Here we use run_model3, which will write output to the notebook.
Can you write Python code for it?

silent = False  #Print model output to screen?
pause = False   #Require user to hit enter? Doesn't mean much in Ipython notebook
report = True   #Store the output from the model in buff
success, buff = MF.run_model(silent=silent, pause=pause, report=report)


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Setup a New Directory and Change Paths
For this tutorial, we will work in a new subdirectory underneath the directory where the notebook is located. We can use some fancy Python tools to help us manage the directory creation. Note that if you encounter path problems with this workbook, you can stop and then restart the kernel and the paths will be reset.
Can you write Python code for it?

# Set the name of the path to the model working directory
dirname = "P4-5_Hubbertville"
datapath = os.getcwd()
modelpath = os.path.join(datapath, dirname)
print 'Name of model path: ', modelpath

# Now let's check if this directory exists.  If not, then we will create it.
if os.path.exists(modelpath):
    print 'Model working directory already exists.'
else:
    print 'Creating model working directory.'
    os.mkdir(modelpath)


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Upstream Weighting Package (like Layer Property Flow Package)
Create a flopy layer property flow package object: flopy.modflow.ModflowUpw.
Can you write Python code for it?


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Well Package
Create a flopy output control object: flopy.modflow.ModflowWel.
Can you write Python code for it?

STRESS_PERIOD_DATA = [
                      [0, 0, 0, -2500],   #layer, row, column, Specified flux Q from P4.3 northern BC
                      [0, 0, 1, -2500],   #remember Python indexing is zero based
                      [0, 0, 2, -2500],
                      [0, 0, 3, -2500],
                      [0, 0, 4, -2500],
                      [0, 0, 5, -2500],
                      [0, 0, 6, -2500],
                      [0, 0, 7, -2500],
                      [0, 0, 8, -2500],
                      [0, 21, 0, -2500],   #layer, row, column, Specified flux Q from P4.3 southern BC
                      [0, 21, 1, -2500],   #remember Python indexing is zero based
                      [0, 21, 2, -2500],
                      [0, 21, 3, -2500],
                      [0, 21, 4, -2500],
                      [0, 21, 5, -2500],
                      [0, 21, 6, -2500],
                      [0, 21, 7, -2500],
                      [0, 21, 8, -2500]]

WEL_PACKAGE = flopy.modflow.ModflowWel(MF, stress_period_data=STRESS_PERIOD_DATA)  # remember python 0 index, layer 0 = layer 1 in MF
#print WEL_PACKAGE  # uncomment this at far left to see the information about the flopy WEL object


I am working on the file "P4.5_Flopy_Hubbertville_areal_model_BCs.ipynb".
The first task is:
Writing the MODFLOW Input Files
Before we create the model input datasets, we can do some directory cleanup to make sure that we don't accidently use old files.
Can you write Python code for it?

#Before writing input, destroy all files in folder to prevent reusing old files
#Here's the working directory
print modelpath
#Here's what's currently in the working directory
modelfiles = os.listdir(modelpath)
print modelfiles

#delete these files to prevent us from reading old results
modelfiles = os.listdir(modelpath)
for filename in modelfiles:
    f = os.path.join(modelpath, filename)
    if modelname in f:
        try:
            os.remove(f)
            print 'Deleted: ', filename
        except:
            print 'Unable to delete: ', filename

#Now write the model input files
MF.write_input()


I am working on the file "P4.6_Flopy_Hubbertville_no_flow_BC_test.ipynb".
The first task is:
Basic Package
Create a flopy basic package object: flopy.modflow.ModflowBas.
Can you write Python code for it?

# Variables for the BAS package
IBOUND = np.ones((NLAY, NROW, NCOL), dtype=np.int32)  # all nodes are active (IBOUND = 1)
# Let's check and make sure that our 2 additional columns showed up in the right place and the
# model domain dimension is correct.
print IBOUND

# like P4.3a, make the top of the profile specified head by setting the IBOUND = -1
IBOUND[:, 0, :] = -1  #don't forget arrays are zero-based!
IBOUND[:, -1, :] = -1  #-1 is Python for last in array
# let's see recreate the implicit side no-flow BCs explicitly to match P4.3a
IBOUND[:, :, 0] = 0  #don't forget arrays are zero-based
IBOUND[:, :, -1] = 0  #-1 is Python for last in array
print IBOUND

STRT = 1015 * np.ones((NLAY, NROW, NCOL), dtype=np.float32)  # set starting head to 1010 m through out model domain
STRT[:, 0, :] = 1000.       # river stage for setting constant head
STRT[:, -1, :] = 1000.     # wetland stage for setting constant head
print STRT

BAS_PACKAGE = flopy.modflow.ModflowBas(MF, ibound=IBOUND, strt=STRT)
# print BAS_PACKAGE  # uncomment this at far left to see the information about the flopy BAS object