I am working on the file "Code/Resample Sensor Data to 10 Hz Sampling Rate.ipynb".
The first task is:
3 rows in the resampled database are nan's.  Just drop them.
Can you write Python code for it?

rdf = rdf.dropna()

resampled_rate = 1 / np.diff(rdf['motionTimestamp_sinceReboot'])

plt.hist(resampled_rate, bins=20)


I am working on the file "Code/Resample Sensor Data to 10 Hz Sampling Rate.ipynb".
The first task is:
For SensorLog data, sampling rate can be obtained directly from 'motionTimestamp_sinceReboot'
Can you write Python code for it?

sampling_rate = 1 / np.diff(df['motionTimestamp_sinceReboot'])

plt.hist(sampling_rate)


I am working on the file "Code/Resample Sensor Data to 10 Hz Sampling Rate.ipynb".
The first task is:
The sampling rate was 15 Hz most of the time.  Sometimes it was 30 Hz.  Very occasionally it was close to 10 Hz.  This is somewhat annoying, so let's resample to 10 Hz.
First, we have to make a DateTime column from the loggingTime columns and set it as the index
Can you write Python code for it?

df['DateTime'] = pd.DatetimeIndex(df['loggingTime'])
df = df.set_index('DateTime')


I am working on the file "Code/Rotate Sensor Data to Vehicle Reference Frame.ipynb".
The first task is:
Nice, GravityZ = -1 and GravityX = GravityY = 0
Can you write Python code for it?

# indices 7000 to 10000 include a series of sharp turns in the Censio parking lot
rdf[7000:10000][['gyroRotationX', 'gyroRotationY', 'gyroRotationZ']].plot()


I am working on the file "Code/Rotate Sensor Data to Vehicle Reference Frame.ipynb".
The first task is:
Now let's investigate the rotated XYZ signals
Can you write Python code for it?


I am working on the file "Code/Rotate Sensor Data to Vehicle Reference Frame.ipynb".
The first task is:
The XYZ axes from SensorLog are in the frame of the iPhone.  On the way to Censio, my phone was placed flat on the driver seat.  On the return trip Nick had my phone in his pocket at a slight angle. This is reflected in the step function like behavior of the gravityXYZ time series.
Can you write Python code for it?

# plot gravity signal
df[['motionGravityX', 'motionGravityY', 'motionGravityZ']].plot()


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

Assigning aliases to columns

SELECT

      column_a AS alias_a                 ➞ creates an alias for column_a

FROM

     table_name

WHERE

     alias_a = x                                   ➞ optional; use the alias in the WHERE clause

ORDER BY

     alias_a                                            ➞ optional; use the alias in the ORDER BY clause

[LIMIT clause]

Aliases allow you to rename columns and tables in your query.  They will come in handy as we learn to do more with the data.
In plain English, the query below can be read as "Show me the book column from the book_table, but rename the column to book_title.
Can you write Python code for it?

as_q = widgets.Textarea(value=
'''SELECT
    book AS book_title
FROM
    book_table''',
width = '50em', height = '14em')
display(as_q)

as_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(as_b)
run_q(as_q, as_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

Conditional Type | Microsoft SQL Server | MySQL | Oracle | SQLite
:--------------- |:------------------ |:--- |:---- |:----
IF | IF logical_test PRINT value_if_true  | IF(logical_test, value_if_true, value_if_false) (same as Excel) | IF logical_test THEN value_if_true ELSIF...END IF | NOT SUPPORTED
CASE WHEN | ✓ | ✓ | ✓ | ✓

SELECT

       CASE WHEN some_column = x THEN value_if_true

                   WHEN some_column = y THEN other_value_if_true

                    ELSE value_if_false

                    END

FROM

       some_table

Because SQLite doesn't support IF statements, we're going to focus on CASE WHEN. CASE WHEN lets you accomplish the same thing by setting logical tests and conditional values, but it has the added bonus of freeing you from ever needing to nest multiple IF statements.
Let's start very simple. The following query uses a logical test to create a column where the value is "true" if the author is Austen, and "false" if the author is not Austen:
Can you write Python code for it?

case1_q = widgets.Textarea(value=
'''SELECT
    last_name,
    CASE WHEN last_name = 'Austen' THEN 'True'
        ELSE 'False'
        END
FROM
    auth_table''',
width = '50em', height = '13em')
display(case1_q)

case1_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(case1_b)
run_q(case1_q, case1_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

SELECT

      column_a,
      SUM(column_b)                    ➞ sums up the values in column_b

FROM

      table_name

[WHERE clause]

GROUP BY                                    ➞ creates one group for each unique value in column_a

     column_a

[ORDER BY clause]

[LIMIT clause]

GROUP BY creates a group for each unique value in the column you specify. You'll always use it in conjunction with functions - it creates segments for your results. In plain English, the query below says: "Show me the average revenue per book from the sales_table"
Can you write Python code for it?

group_q = widgets.Textarea(value=
'''SELECT
    book,
    AVG(revenue)
FROM
    sales_table
GROUP BY
    book''',
width = '50em', height = '20em')
display(group_q)

group_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(group_b)
run_q(group_q, group_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

SELECT

      column_a,

      FUNCTION(column_b)

FROM

      table_name

[WHERE clause]

GROUP BY

      column_a HAVING FUNCTION(column_b) > x                   ➞ returns groups whose value is greater than x

[ORDER BY clause]

[LIMIT clause]

Use HAVING with GROUP BY in order to filter out groups that don't meet your criteria.  Below, the plain English translation of this query says, "show my the total revenue for each book, but only show me books that have total revenue over $100"
Can you write Python code for it?

having_q = widgets.Textarea(value=
'''SELECT
    book,
    SUM(revenue)
FROM
    sales_table
GROUP BY
    book HAVING SUM(revenue) > 100''',
width = '50em', height = '11em')
display(having_q)

having_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(having_b)
run_q(having_q, having_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

SELECT

      table_x.column_a,                                 ➞ read this as "column_a from table_x"

      table_y.column_b,                                 ➞ "column_b from table_y"

FROM

     table_x

     JOIN table_y

      ON table_x.key_column_x = table_y.key_column_y
                                 ➞ table_x's key_column_x has corresponding values with table_y's key_column_y

[WHERE clause]

[ORDER BY clause]

[LIMIT clause]

The ability to join tables is the most fundamental and useful part about relational databases. Different tables have columns with corresponding values, and you can use these columns as "keys" to join the two tables.
The format table_x.key_column can be read as "key_column from table_x"; it tells SQL the tables where a column is located. We didn't need this before because we were only using one table at a time, so SQL knew exactly which table we were talking about. When we deal with more than one table, we need to be more specific.  So for example, book_table.book means "the book column from the book_table", and auth_table.last_name means "the last_name column from the auth_table."
Think back to our original discussion of splitting up our author and book data onto...
Can you write Python code for it?

join_q = widgets.Textarea(value=
'''SELECT
    *
FROM
    book_table
    JOIN auth_table
        ON book_table.author = auth_table.last_name''',
width = '50em', height = '11em')
display(join_q)

join_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(join_b)
run_q(join_q, join_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

Microsoft SQL Server | MySQL | Oracle | SQLite
:------------------: | :---: | :----: | :----:
SELECT TOP N column_name  | LIMIT N | WHERE ROWNUM <= N | LIMIT N

SELECT

     column_a

FROM

     table_name

[WHERE clause]

[ORDER BY clause]

LIMIT N                             ➞ limits the result-set to N rows

LIMIT lets you set a maximum limit to the number of rows that your query returns. You've seen the query below before, but now we've added a LIMIT clause:
Can you write Python code for it?

limit_q = widgets.Textarea(value=
'''SELECT
    *
FROM
    sales_table
LIMIT 5''',
width = '50em', height = '14em')
display(limit_q)

limit_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(limit_b)
run_q(limit_q, limit_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

SELECT

      table_x.column_a,

     table_y.column_b

     table_z.column_c

FROM

      table_x

      JOIN table_y ON table_x.key_column = table_y.key_column

      JOIN table_z ON table_x.other_key_column = table_z.other_key_column

[WHERE clause]

[ORDER BY clause]

[LIMIT clause]

As long as they are directly related or related by the transitive property, you can join multiple tables. Consider the sales_table and the auth_table in a Venn Diagram - there's no relation at all:

However, when the book_table enters the picture, suddenly the sales_table and auth_table have a connection:

Now we have an opportunity to join all three!
Challenge:
Write a query to show the first and last name of the author, the book title, the COGs, and the revenue from each transaction.
 Extra credit once you've completed the challenge: Only return rows where the book was written by an English author. Sort your results so that the transaction with the most revenue appears first.
Can you write Python code for it?

multi_join_chall = widgets.Textarea(value=
'',
width = '50em', height = '25em')
display(multi_join_chall)

multi_join_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(multi_join_chall_b)
run_q(multi_join_chall, multi_join_chall_b)

multi_join_chall_c ='''SELECT
    first_name,
    last_name,
    book_table.book,
    cogs,
    revenue
FROM
    book_table
    JOIN auth_table
        ON book_table.author = auth_table.last_name
    JOIN
        sales_table
        ON book_table.book = sales_table.book'''
cheat(multi_join_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

SELECT

       column_a

FROM

       table_x

WHERE

      column_a IN (SELECT column_b FROM table_y)

Read first: For this section, we'll need the extra rows in the auth_table and book_table that we added during the JOIN exercises. If you've closed the program or re-run it since you last added those rows, then click the button below to re-add them.
Can you write Python code for it?

insert_b = widgets.Button(description="Read the paragraph above before clicking", width='25em', height='3em', color='white',background_color='#1f5fd6', border_color='#1f5fd6')
display(insert_b)
def insert_button(b):
    insert_q1 = '''INSERT INTO auth_table VALUES ('Tolstoy', 'Leo', 'Russia', 1828)'''
    insert_q2 = '''INSERT INTO auth_table VALUES ('Twain', 'Mark', 'USA', 1835)'''
    insert_q3 = '''INSERT INTO book_table VALUES ('Jude the Obscure', '11.25', 'Hardy')'''
    insert_q4 = '''INSERT INTO book_table VALUES ('The Age of Innocence', '14.20', 'Wharton')'''
    query_list = [insert_q1, insert_q2, insert_q3, insert_q4]
    for query in query_list:
        run(query)
    print('New rows have been added to auth_table and book_table!')
insert_b.on_click(insert_button)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

SELECT

      column_a + column_b,                 ➞ adds the values in column_a and columns_b

      column_a - column_b,                 ➞ subtracts

      column_a * column_b,                 ➞ multiplies

      column_a / column_b,                 ➞ divides

      (column_a + column_b) * column_c,                 ➞ use parentheses to make more complex calculations

FROM

      table_name

[WHERE clause]

[ORDER BY clause]

[LIMIT clause]

This is pretty straightforward. Let's start by calculating gross profit per transaction: revenue minus cogs.  Recall again that we can use S.book or B.book - we'll get the same results.
Can you write Python code for it?

op_q = widgets.Textarea(value=
'''SELECT
    B.book,
    S.revenue,
    B.cogs,
    S.revenue - B.cogs
FROM
    book_table B
    JOIN sales_table S ON B.book = S.Book''',
width = '50em', height = '18em')
display(op_q)

op_q_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(op_q_b)
run_q(op_q, op_q_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

Microsoft SQL Server | MySQL | Oracle | SQLite
:------------------: | :---: | :----: | :----:
GROUP BY column_a WITH ROLLUP  | GROUP BY column_a WITH ROLLUP | GROUP BY ROLLUP (column_a) | not supported
Unfortunately, SQLite doesn't have a simple way to do ROLLUP like the other RDBMSs, so we can't practice it here. However, the concept is very straightforward: it's exactly like using UNION to add a summary row, except way simpler.  Below is what the query would look like if we were using Microsoft or MySQL.  Take a look at the query and the output to understand what's going on, even if you can't practice it:

SELECT

      book,

      SUM(revenue) AS total_revenue,

      COUNT(*) AS count_of_sales

FROM

      sales_table

GROUP BY

      book WITH ROLLUP

Can you write Python code for it?

HTML(run('''SELECT
    book,
    SUM(revenue) AS total_revenue,
    COUNT(*) AS count_of_sales
FROM
    sales_table
GROUP BY
    book

UNION ALL

SELECT
   'NULL',
   SUM(revenue),
   COUNT(*)
FROM
    sales_table''').to_html(index=False))


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

SELECT

     *                    ➞ an asterisk means "all columns"

FROM

     table_name

To see the actual data in a table, we'll use SELECT and FROM clauses.  In the SELECT clause, you tell SQL which columns you want to see.  In the FROM clause, you tell SQL the table where those columns are located.  An asterisk returns all columns from a particular table.
In plain English, the query below says: "Show me all columns and their data from the book_table"
Can you write Python code for it?

select = widgets.Textarea(value=
'''SELECT
    *
FROM
    book_table''',
width = '50em', height = '7em')
display(select)

select_button = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(select_button)
run_q(select, select_button)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Previous        |       Table of Contents       |        Next

SELECT

      column_a

FROM

      table_name

WHERE

      column_a = x                 ➞ result-set will only include rows where value of column_a is x

WHERE lets you filter results so you only see rows that specifically match your criteria.  Below there are few more options for the WHERE clause:
Options for WHERE | Description
:------- | :-------------
col = 'some_text' | Put text in quotations. Capitalization is important!
col != x | Return rows where col's values DO NOT equal x
col < x | Return rows where col's value is less than x
col <= x |  Return rows where col's value is less OR EQUAL TO than x
col IN (x, y) | Values can equal EITHER x OR y
col NOT IN (x, y) | Value are NEITHER x NOR y
col BETWEEN x AND y | Values are between x and y
col = x AND another_col = y | Returns rows when col's values are x AND another_col's values are y
col = x OR another_col = y | Returns rows when col's values are x OR another_col's values are y

WHERE & text values
Below, we use WHERE to tell SQL to only show us rows in the book_table when Hemingway is the author. In plain English, we're saying "Show me information about books that are written by Hemingway in the book_table"
Can you write Python code for it?

where_q = widgets.Textarea(value=
'''SELECT
    *
FROM
    book_table
WHERE
    author = 'Hemingway' ''',
width = '50em', height = '10em')
display(where_q)

where_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(where_b)
run_q(where_q, where_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

WHERE with AND/OR
So far, we've only filtered by a specific column (like the revenue column , country, or author columns).  Sometimes you'll want to filter by multiple columns. This is where AND and OR come in handy.
Can you write Python code for it?

and_q = widgets.Textarea(value=
'''SELECT
    *
FROM
    book_table
WHERE
    author = 'Hemingway'
    AND cogs > 11''',
width = '50em', height = '12em')
display(and_q)

and_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(and_b)
run_q(and_q, and_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Use multiple joins sparingly: Multiple joins can put a lot of strain on servers because SQL has to do a lot of work matching up all that data. The more tangential the relationship, the worse it gets. Avoid more than 2 degrees of separatation, and avoid joining 2 or more large tables. It's ok if one of your tables is big, but the others should be small.

Previous        |       Table of Contents       |        Next

There are more ways to join two tables than the method we just covered. However, not all RDBMS support these different join methods. We'll learn about each of these methods, even if we can only practice 2 of them in SQLite.
 Join Type |Microsoft SQL Server | MySQL | Oracle | SQLite
:----: | :------------------: | :---: | :----: | :----:
JOIN or INNER JOIN | ✓ | ✓ | ✓ | ✓
LEFT JOIN or LEFT OUTER JOIN | ✓ | ✓ | ✓ | ✓
RIGHT JOIN or RIGHT OUTER JOIN | ✓ | ✓ | ✓ | not supported
OUTER JOIN or FULL OUTER JOIN | ✓ | not supported | ✓ | not supported
We're going to leave behind the book database for the next lesson, since a different data set will help illustrate the point a little better.
Until now, the tables that we've joined have had columns that correspond perfectly - that is to say, every value that appears in one table also appears in the other. There aren't any authors that appear in the auth_table that don't also appear at least once in the book_table, and vice versa.
Sometimes, you'll have two tables with corresponding columns, but they don't match perfectly.
Consider the two tables below. The first lists tech companies and their CEOs, the second lists  publicly traded companies and their share price.

Amazon, Alphabet, and Microsoft all appear on both tables. But Uber, SpaceX...
Can you write Python code for it?

inner_join_q = widgets.Textarea(value=
'''SELECT
    *
FROM
    tech_cos
    JOIN public_cos
        ON tech_cos.company = public_cos.company''',
width = '50em', height = '11em')
display(inner_join_q)

inner_join_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(inner_join_b)
run_q(inner_join_q, inner_join_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Use LIKE sparingly: It's a great tool, but it really puts the strain on your database's servers. Use it only when a table is pretty small or when you've limited your result-set by using additional filters in the WHERE clause.

Previous        |       Table of Contents       |        Next

SELECT

      column_a

FROM

      table_name

[WHERE clause, optional]

ORDER BY                                              ➞ sorts the result-set by column_a

      column_a DESC                             ➞ DESC is optional, it sorts results in descending order

Without an ORDER BY clause, the default result-set will be sorted by however it appears in the database (which is crap-shoot depending on the type of table). Use ORDER BY to sort your result-set by a particular column, and add DESC to sort in descending order (Z→A, 100→1).
Can you write Python code for it?

order_q = widgets.Textarea(value=
'''SELECT
    *
FROM
    book_table
ORDER BY
    book''',
width = '50em', height = '11em')
display(order_q)

order_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(order_b)
run_q(order_q, order_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:

Use asterisks sparingly. Usually, you'll select specific columns from a table rather than all columns. Using an asterisk to select all columns is okay when the table is small or when you tightly constrain your selection of rows. Otherwise, select specific columns and use WHERE and LIMIT (taught below) to go easy on your servers.

SELECT specific columns:

SELECT

      column_a,                 ➞ separate multiple columns with commas

      column_b                   ➞ optional, but conventional, to also use a return

FROM

      table_name

Instead of using an asterisk for "all columns", you can specify a particular column or columns.   In plain English: "Show me the data in the book and author columns from the book_table"
Can you write Python code for it?

select_col = widgets.Textarea(value=
'''SELECT
    book,
    author
FROM
    book_table''',
width = '50em', height = '8em')
display(select_col)

select_col_button = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(select_col_button)
run_q(select_col, select_col_button)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Answer to the extra credit:
Can you write Python code for it?

multi_join_chall_c2 ='''SELECT
    first_name,
    last_name,
    book_table.book,
    cogs,
    revenue
FROM
    book_table
    JOIN auth_table
        ON book_table.author = auth_table.last_name
    JOIN
        sales_table
        ON book_table.book = sales_table.book
WHERE
    country = 'England'
ORDER BY
    revenue DESC'''
cheat(multi_join_chall_c2)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Assigning aliases to tables

SELECT

      X.column_a,

      Y.column_b

FROM

      table_x X                                                ➞ assigns table_x the alias X

      JOIN table Y                                         ➞ assigns table_y the alias Y

           ON X.key_column = Y.key_column            ➞ table aliases can be used as substitutes in the table_x.column_a format

When dealing with one or more tables in a query, we commonly assign capitalized one-letter aliases to tables. Writing X.key_column is much shorter than table_x.key_column, and coders like shortcuts. They also typically won't use AS when assigning aliases to tables (although it makes no difference either way).
When you're dealing with only one table, it's unnecessary to use table aliases because SQL knows exactly what columns you are referring to. However, when you are dealing with 2 or more tables, particularly tables that have columns with the same names (like book, which is a column in both sales_table and book_table), then aliases are extremely handy.
Can you write Python code for it?

as_table_q = widgets.Textarea(value=
'''SELECT
    S.book,
    S.revenue,
    B.cogs
FROM
    book_table B
    JOIN sales_table S
        ON S.book = B.book''',
width = '50em', height = '12em')
display(as_table_q)

as_table_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(as_table_b)
run_q(as_table_q, as_table_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
COUNT(*) vs COUNT(column_name)
Can you write Python code for it?

insert_null_b = widgets.Button(description='Click here JUST ONCE before starting', width='20em', height='3em', color='white',background_color='#1f5fd6', border_color='#1f5fd6')
display(insert_null_b)
def insert_button(b):
    null_query1 = '''INSERT INTO auth_table VALUES ('Homer', NULL, 'Greece', NULL)'''
    run(null_query1)
    print('A new row has been added to the auth_table!')
insert_null_b.on_click(insert_button)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge Continued...

Delete * and replace it with COUNT(first_name), rerun
Add a line (but don't erase anything) to the SELECT clause: COUNT(*)
Can you write Python code for it?

star_chall_c ='''SELECT
    COUNT(first_name),
    COUNT(*)
FROM
    auth_table
'''
cheat(star_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge Continued:
Start by copying the query that you wrote in the previous challenge and pasting it in the box below.
1. Change the line book_table.book = sales_table.book to book = book and rerun. What's going wrong? Fix it and rerun.
2. Change the query so that you only see the book title listed once. If you get stuck, remember that table_x.column_a means "column_a from table_x".
Can you write Python code for it?

join_chall2 = widgets.Textarea(value=
'',
width = '50em', height = '17em')
display(join_chall2)

join_chall2_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(join_chall2_b)
run_q(join_chall2, join_chall2_b)

join_chall_c2 ='''SELECT
    book_table.book,           [NOTE: or you could use "sales_table.book" instead]
    cogs,
    author,
    id,
    revenue
FROM
    book_table
    JOIN sales_table
        ON book_table.book = sales_table.book
LIMIT 20'''
cheat(join_chall_c2)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:

Copy and paste the query you just wrote for the previous challenge.
Add the following columns in the SELECT clause (in addition to author and gross margin columns), and give every column an alias:
total revenue
total cogs
a count of the number of individual transactions
BONUS: a count of the distinct book titles sold
BONUS: a comma-separated list of the book titles with no repeats

Only include results where the author isn't Faulker and the book isn't Hamlet
Sort your results so that the author with the highest average gross margin is listed first
Can you write Python code for it?

group_chall3 = widgets.Textarea(value='', width = '50em', height = '26em')
display(group_chall3)

group_chall3_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(group_chall3_b)
run_q(group_chall3, group_chall3_b)

group_chall3_c ='''SELECT
    B.author AS author_name,
    SUM(S.revenue) AS total_revenue,
    SUM(B.cogs) AS total_cogs,
    (SUM(S.revenue) - SUM(B.cogs))/SUM(S.revenue) AS gross_margin,
    COUNT(*) AS transaction_count,
    COUNT(DISTINCT(S.book)) AS distinct_book_titles,
    GROUP_CONCAT(DISTINCT(S.book)) AS book_list
FROM
    book_table B
    JOIN sales_table S ON B.book = S.book
WHERE
    author_name != 'Faulkner'
    AND S.book != 'Hamlet'
GROUP BY
    author_name
ORDER BY
    gross_margin DESC'''
cheat(group_chall3_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:

In the SELECT clause:
Use CASE WHEN to create a column that creates buckets for author's birth_year: "Before 1700", "1700-1800", "After 1800"
Use CASE WHEN to create a column that returns the count of books by authors from USA
Use CASE WHEN to create a column that returns the count of books by authors from England

GROUP BY the birth_year bucket column that you created
Can you write Python code for it?

case_pivot_chall = widgets.Textarea(value='',width = '50em', height = '18em')
display(case_pivot_chall)

case_pivot_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(case_pivot_chall_b)
run_q(case_pivot_chall, case_pivot_chall_b)

case_pivot_chall_c ='''SELECT
    CASE WHEN A.birth_year < 1700 THEN "Before 1700"
        WHEN A.birth_year BETWEEN 1700 AND 1800 THEN "1700-1800"
        WHEN A.birth_year > 1800 THEN "After 1800"
        END AS era,
    COUNT(CASE WHEN A.country = 'USA' THEN book END) AS count_from_USA,
    COUNT(CASE WHEN A.country = 'England' THEN book END) AS count_from_England
FROM
    book_table B
    JOIN auth_table A ON B.author = A.last_name
GROUP BY
    era
'''
cheat(case_pivot_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:
Try out the GROUP_CONCAT function.  Write a query to select GROUP_CONCAT(last_name) from the auth_table, only return results where the author is NEITHER Austen NOR Shakespeare. After you get your query to work, change it to GROUP_CONCAT(last_name, ' / ') and rerun.
Can you write Python code for it?

function_chall2 = widgets.Textarea(value='',width = '50em', height = '10em')
display(function_chall2)

function_chall2_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(function_chall2_b)
run_q(function_chall2, function_chall2_b)

function_chall2_c ='''SELECT
    GROUP_CONCAT(last_name)
FROM
    auth_table
WHERE
    last_name NOT IN ('Austen', 'Shakespeare')
'''
cheat(function_chall2_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:
Write a query to find the average cost of goods for books whose authors are from the US (USA). Round the number to the nearest cent. Use an alias for your column.
Can you write Python code for it?

function_chall = widgets.Textarea(value='', width = '50em', height = '11em')
display(function_chall)

function_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(function_chall_b)
run_q(function_chall, function_chall_b)

function_chall_c ='''SELECT
    ROUND(AVG(B.cogs), 2) AS avg_cogs
FROM
    book_table B
    JOIN auth_table A ON B.author = A.last_name
WHERE
    A.country = 'USA'
'''
cheat(function_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:
Write a query to join the auth_table with the sales_table (remember that this requires multiple joins). Count the number of sales per country (author's country of origin in the auth_table), but don't include sales from Hemingway.
Can you write Python code for it?

have_chall3 = widgets.Textarea(value='', width = '50em', height = '16em')
display(have_chall3)

have_chall3_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(have_chall3_b)
run_q(have_chall3, have_chall3_b)

have_chall3_c ='''SELECT
    A.country,
    COUNT(*) AS count_of_sales
FROM
    sales_table S
    JOIN book_table B ON S.book = B.book
    JOIN auth_table A ON A.last_name = B.author
WHERE
    A.last_name != 'Hemingway'
GROUP BY
    A.country

NOTE: you can also use "B.author != 'Hemingway'" in the WHERE clause to get the same results'''
cheat(have_chall3_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:
Write a query to return all books from the book_table, and their revenue data whenever that information is available. Try to figure out how you might sort your results so that you see books with no sales first.
Can you write Python code for it?

join_drill4 = widgets.Textarea(value='',width = '50em', height = '12em')
display(join_drill4)

join_drill4_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(join_drill4_b)
run_q(join_drill4, join_drill4_b)

join_drill4_c ='''SELECT
    book_table.book,
    revenue
FROM
    book_table
    LEFT JOIN sales_table ON book_table.book = sales_table.book
ORDER BY
    revenue'''
cheat(join_drill4_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:
Write a query to return each author from the book_table without any names repeating.
Can you write Python code for it?

distinct_q_chall = widgets.Textarea(value='', width = '50em', height = '7em')
display(distinct_q_chall)

distinct_q_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(distinct_q_chall_b)
run_q(distinct_q_chall, distinct_q_chall_b)

REP ='''SELECT
    DISTINCT author
FROM
    book_table'''
cheat(REP)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:
Write a query to see authors who appear in the auth_table but don't show up in the book_table.
Can you write Python code for it?

nest_chall = widgets.Textarea(value='', width = '50em', height = '12em')
display(nest_chall)

nest_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(nest_chall_b)
run_q(nest_chall, nest_chall_b)

nest_chall_c ='''SELECT
    last_name
FROM
    auth_table
WHERE
    last_name NOT IN (SELECT author FROM book_table)
'''
cheat(nest_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:
Write a query to see the maximum and minimum prices that each book sold for, but don't include Macbeth or Hamlet in your result-set:
Can you write Python code for it?

group_chall3 = widgets.Textarea(value='',width = '50em', height = '15em')
display(group_chall3)

group_chall3_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(group_chall3_b)
run_q(group_chall3, group_chall3_b)

group_chall3_c ='''SELECT
    book,
    MAX(revenue),
    MIN(revenue)
FROM
    sales_table
WHERE
    book NOT IN ('Macbeth','Hamlet')
GROUP BY
    book'''
cheat(group_chall3_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:
Write a query to see the titles of all the books from the book_table, and the author's country when that information is available.
Can you write Python code for it?

join_drill3 = widgets.Textarea(value='', width = '50em', height = '13em')
display(join_drill3)

join_drill3_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(join_drill3_b)
run_q(join_drill3, join_drill3_b)

join_drill3_b ='''SELECT
    book,
    country
FROM
    book_table
    LEFT JOIN auth_table
        ON book_table.author = auth_table.last_name'''
cheat(join_drill3_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenge:
Write a query to view all columns from the book_table, but only where the author's name is something like "pear" or COGs are over $12. Sort your results by COGs with the cheapest book first.
Can you write Python code for it?

order_chall2 = widgets.Textarea(value=
'',
width = '50em', height = '16em')
display(order_chall2)

order_chall_b2 = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(order_chall_b2)
run_q(order_chall2, order_chall_b2)

order_chall_c2 ='''SELECT
    *
FROM
    book_table
WHERE
    author LIKE '%pear%'
    OR cogs > 12
ORDER BY
    cogs'''
cheat(order_chall_c2)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Challenges
After we've learned a new concept and you've practiced with some quick exercises, you'll be challenged to write your own query. Read each challenge carefully, and keep re-running it until you get the results you are looking for.
Need to cheat a little? Check the "Reveal" box to see the answer to a challenge.

Can you write Python code for it?

chall ='''When you click a checkbox, you reveal the answer to a challenge!
Uncheck it to hide the answer again.'''
cheat(chall)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
From the count of distinct books in each table, we see that there are two books in our book_table (our inventory) that haven't made a single sale. Imagine if both tables had thousands of rows - it'd be a nightmare to try to figure out which were the books with no sales.  However, a nested query can help us out.
The query below uses a nested query in the WHERE clause. In plain English, it says "Show me the books from the book_table, but not the ones that also show up in the sales_table":
Can you write Python code for it?

nested_q3 = widgets.Textarea(value=
'''SELECT
    book
FROM
    book_table
WHERE
    book NOT IN (SELECT book FROM sales_table)''',
width = '50em', height = '10em')
display(nested_q3)

nested_q3_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(nested_q3_b)
run_q(nested_q3, nested_q3_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
HAVING vs. WHERE
HAVING and WHERE both let you change the results you see in your result-set, but they operate quite differently. Take a look at the query below. It looks at the average cogs per author, but uses a WHERE clause to filter out 'Faulkner':
Can you write Python code for it?

hw_q1 = widgets.Textarea(value=
'''SELECT
    author,
    AVG(cogs)
FROM
    book_table
WHERE
    author != 'Faulkner'
GROUP BY
    author''',
width = '50em', height = '14em')
display(hw_q1)

hw_b1 = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(hw_b1)
run_q(hw_q1, hw_b1)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
It's much more common to use COUNT(*) than COUNT(column_name) when you are trying to get a count of the number of rows in your result-set.  This is because COUNT(*) will capture all rows, while COUNT(column_name) will skip over NULL values in that particular row.
Challenge:
We've just added new a new row to the auth_table that has some NULL (blank) values. Start by writing a query to view everything (SELECT *) in the auth_table and make a note of the new row.
Can you write Python code for it?

star_chall = widgets.Textarea(value='', width = '50em', height = '12em')
display(star_chall)

star_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(star_chall_b)
run_q(star_chall, star_chall_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
NOTE THAT YOU NEED TO HIT "RUN" AGAIN FOR THE QUERY BELOW
(It should return the number 13. If it doesn't, click the blue button above to update the book_table, then re-run the query below)
Can you write Python code for it?

nested_q2 = widgets.Textarea(value=
'''SELECT
    COUNT(DISTINCT(book)) AS Count_of_Distinct_Books
FROM
    book_table''',
width = '50em', height = '7em')
display(nested_q2)

nested_b2 = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(nested_b2)
run_q(nested_q2, nested_b2)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Next, take the output that you just produced and:

• comma-separate each book

• wrap each book in quotation marks

• paste your list between the parentheses in the WHERE clause below:

• rerun the query

Can you write Python code for it?

nest_q3_explained2 = widgets.Textarea(value=
'''SELECT
    SUM(revenue)
FROM
    sales_table
WHERE
    book IN ( )''',
width = '50em', height = '12em')
display(nest_q3_explained2)

nest_b3_explained2 = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(nest_b3_explained2)
run_q(nest_q3_explained2, nest_b3_explained2)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Note that while this CASE WHEN method will work in other RDBMSs, it's more common to use IF when you are only using a single logical test.  In MySQL, for instance, the line for Female_Rev would look like this instead, which would translate to "sum up the revenue for any row where the author is Austen, and the number 0 whenever the author is not Austen":

SUM(IF(B.author = 'Austen', revenue, 0))

Quick Exercises:
The query from above has been reproduced below for these exercises (so you don't have to keep scrolling up and down).
1.  Change the query so that you have separate columns for each individual author's revenue, rerun
2. Change SUM to AVG, rerun
3. Change AVG to COUNT - note that with conditional statements, you don't use an asterisk with COUNT. You need to stick with a specific column name
Can you write Python code for it?

case_pivot1 = widgets.Textarea(value=
'''SELECT
    date,
    SUM(CASE WHEN B.author = 'Austen' THEN revenue END) AS Female_Rev,
    SUM(CASE WHEN B.author != 'Austen' THEN revenue END) AS Male_Rev
FROM
    book_table B
    JOIN sales_table S ON B.book = S.book
GROUP BY
    date''',
width = '50em', height = '18em')
display(case_pivot1)

case_pivot_b1 = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(case_pivot_b1)
run_q(case_pivot1, case_pivot_b1)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Now auth_table has 2 authors listed (Tolstoy and Twain) that don't appear on the book_table, and the book_table has two books (Jude the Obscure and The Age of Innocence) whose authors don't appear in the auth_table.
Challenge:
Write a query to view the book titles, first names, and last names of authors that appear on both the auth_table and the book_table.
Can you write Python code for it?

join_drill2 = widgets.Textarea(value='', width = '50em', height = '12em')
display(join_drill2)

join_drill2_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(join_drill2_b)
run_q(join_drill2, join_drill2_b)

join_drill2_c ='''SELECT
    book,
    author,
    first_name
FROM
    book_table
    JOIN auth_table ON book_table.author = auth_table.last_name'''
cheat(join_drill2_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Now that we know the order in which SQL executes commands, we can see what went wrong. The rows with COGs under $11 were eliminated before SQL averaged COGs for each group:

Challenge:
Write a query to join book_table and sales_table. Select author and total revenue, but only return authors whose total revenue was over $200
Can you write Python code for it?

have_chall2 = widgets.Textarea(value='',width = '50em', height = '12em')
display(have_chall2)

have_chall2_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(have_chall2_b)
run_q(have_chall2, have_chall2_b)

have_chall2_c ='''SELECT
    B.author,
    SUM(S.revenue) AS total_rev
FROM
    book_table B
    JOIN sales_table S ON B.book = S.book
GROUP BY
    author HAVING total_rev > 200'''
cheat(have_chall2_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercise:

Add 'Faulkner' to the list, rerun.
Replace IN with NOT IN, rerun.
Delete the whole last line and replace it so that the query returns all books except for Emma and Macbeth.

WHERE & number values
The WHERE clause is useful with numbers as well. We can start throwing in comparisons like less than (<) and greater than (>):
Can you write Python code for it?

greater_q = widgets.Textarea(value=
'''SELECT
    *
FROM
    sales_table
WHERE
    revenue > 18''',
width = '50em', height = '10em')
display(greater_q)

greater_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(greater_b)
run_q(greater_q, greater_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercise:

Change the query above to use UNION ALL instead of UNION and re-run.  Make sure you understand how the output changes.
Delete AS selection in the second query and rerun.
Rename selection to something else in the first query and rerun.
Add cogs in the SELECT clause in the first query and country to the SELECT clause in the second query, rerun.
Delete cogs in the first query and rerun.  Can you think why you're hitting an error?

Useful applications for UNION
The above example is just a simple illustration of how UNION functions, but it's not very useful as a practical application.  Now lets try UNION in a more useful way.
Let's say that our imaginary book store decides to start stocking a few movies, so we've created a new table to manage this new inventory:

Now let's say we wanted to view all of our store inventory, COGs, and the author or director of each item.  We don't want to join the movie_table and the book_table - there's nothing to really join them on.  However, it'd be useful to stack them.
Challenge:
Use UNION to write a query to view the contents of both movie_table and book_table in a single table.  The column-headers should be: Item, COGs, and Creator.  Order by item title (hint: with UNION, the ORDER BY clause can only go after the second query).
Can you write Python code for it?

union_chall = widgets.Textarea(value='',width = '50em', height = '25em')
display(union_chall)

union_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(union_chall_b)
run_q(union_chall, union_chall_b)

union_chall_c ='''SELECT
    book as Item,
    cogs as COGs,
    author as Creator
FROM
    book_table

UNION

SELECT
    film,
    cogs,
    director
FROM
    movie_table
ORDER BY
    Item'''
cheat(union_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercise:

Change the query so that you only see results when the writer's first name is William, rerun
Change the query to sort the books in alphabethical order, rerun
Limit the number of rows to 3, rerun
Can you write Python code for it?

join_drill4_ex_c ='''SELECT
    first_name,
    book
FROM
    auth_table
    LEFT JOIN book_table ON auth_table.last_name = book_table.author
WHERE
    first_name = 'William'
ORDER BY
    book
LIMIT 3'''
cheat(join_drill4_ex_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercise:

Edit the query above so that you only see books by authors from England
Edit it so that you only see books by authors NOT from England.

Challenge
Write a query to see the first names of all authors in the auth_table, and the books they've written when that information is available.
Can you write Python code for it?

join_drill4 = widgets.Textarea(value='',width = '50em', height = '16em')
display(join_drill4)

join_drill4_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(join_drill4_b)
run_q(join_drill4, join_drill4_b)

join_drill4_c ='''SELECT
    first_name,
    book
FROM
    auth_table
    LEFT JOIN book_table ON auth_table.last_name = book_table.author'''
cheat(join_drill4_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercise:
Change the query above to show us all columns and their data from the auth_table instead of the book_table

Challenge:
Write a query to view all columns and their data from the sales_table
Can you write Python code for it?

select_c = widgets.Textarea(value='', width = '50em')
display(select_c)

select_c_button = widgets.Button(
    description='Run',
    width='10em',
    height='2.5em',
    color='white',
    background_color='black',
    border_color='black')
display(select_c_button)
run_q(select_c, select_c_button)

select_answer ='''SELECT
    *
FROM
    sales_table'''
cheat(select_answer)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercise:
Now let's say you want to filter out Faulkner AND you only want to see authors whose average COGs are over $11.  Your first thought might be to use the WHERE clause.  Add AND AVG(cogs) > 11, rerun. Why do you think you're hitting an error?

You hit an error because the AVG(cogs) column was created by a function, and SQL doesn't let you put function-generated columns in the WHERE clause. You have to use HAVING instead. The query below will accomplish what we're trying to do, and returns a result-set that doesn't include Faulkner AND only shows authors whose average COGs are over $11.
Can you write Python code for it?

hw_q2 = widgets.Textarea(value=
'''SELECT
    author,
    AVG(cogs)
FROM
    book_table
WHERE
    author != 'Faulkner'
GROUP BY
    author HAVING AVG(cogs) > 11''',
width = '50em', height = '14em')
display(hw_q2)

hw_b2 = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(hw_b2)
run_q(hw_q2, hw_b2)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercise:
Rewrite the SELECT clause so that you get the same results but only have to use SUM once.

Challenge:
Write a query to view gross margin for all transactions using functions in conjunction with operators.  Extra credit: round your results to the nearest cent.
Can you write Python code for it?

fun_op_chall = widgets.Textarea(value='', width = '50em', height = '8em')
display(fun_op_chall)

fun_op_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(fun_op_chall_b)
run_q(fun_op_chall, fun_op_chall_b)

fun_op_chall_c ='''SELECT
    SUM(S.revenue - B.cogs) / SUM(S.revenue) AS gross_margin
FROM
    book_table B
    JOIN sales_table S ON S.book = B.book

- OR - you can use AVG() instead of SUM() for all functions'''
cheat(fun_op_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercises:

Change 5 to 10, rerun
Add an ORDER BY clause so that you see the top 10 transactions in terms of revenue, rerun
Add a WHERE clause so you only see transactions relating to <u>Emma</u>

Challenge:
Write a query to view the title and cogs of the two books with the lowest cogs in the book_table
Can you write Python code for it?

limit_chall = widgets.Textarea(value=
'',
width = '50em', height = '12em')
display(limit_chall)

limit_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(limit_chall_b)
run_q(limit_chall, limit_chall_b)

limit_chall_c ='''SELECT
    book,
    cogs
FROM
    book_table
ORDER BY
    cogs
LIMIT 2'''
cheat(limit_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercises:

Change AVG() to SUM(), rerun
Give the book column the alias book_title, then use the alias in the GROUP BY clause, rerun
Sort the results so that the most profitable book is listed first, rerun
Add this to the SELECT clause: COUNT(*), rerun
Add a WHERE clause to only return results that are not written by Faulkner (hint: you'll have to join a table for this)

Challenge:
Write a query to count the number of books each author has listed in the book_table.
Can you write Python code for it?

group_chall = widgets.Textarea(value='', width = '50em', height = '12em')
display(group_chall)

group_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(group_chall_b)
run_q(group_chall, group_chall_b)

group_chall_c ='''SELECT
    author,
    count(*)
FROM
    book_table
GROUP BY
    author'''
cheat(group_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercises:

Change the query to rename the column to titles.
Delete the word AS and rerun. (You'll see that AS is totally optional when assigning aliases. It just make the query easier to read.)
Change the query so that you also pull the author, but rename the column author_name.
Change the query so that you only see books by Austen. Use the column's alias in your WHERE clause.
Order results by book in reverse alphabetical order. Use the column's alias in your ORDER BY clause.
Can you write Python code for it?

as_ex_c ='''SELECT
    book titles,
    author AS author_name
FROM
    book_table
WHERE
    author_name = 'Austen'
ORDER BY
    titles DESC'''
cheat(as_ex_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercises:

Delete AND cogs > 11 and rerun the query. Then replace it and run it again.
Change the word AND to OR, rerun. What's going on?

Challenge:
Write a query to pull the last_name, country, and birth_year of authors who were from England AND born after 1650
Can you write Python code for it?

and_chall = widgets.Textarea(value=
'',
width = '50em', height = '14em')
display(and_chall)

and_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(and_chall_b)
run_q(and_chall, and_chall_b)

and_chall_c ='''SELECT
    last_Name,
    country,
    birth_Year
FROM
    auth_table
WHERE
    country = 'England'
    AND birth_Year > 1650 '''
cheat(and_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercises:

Give the CASE WHEN column an alias (immediately after END), rerun
Change the query so that instead of "True", the query returns Austen's first name (use the first_name column), rerun
Add something to the CASE WHEN column so that the query returns Faulkner's first name as well (look at the example code above for help), rerun.

Using CASE WHEN to create categories
CASE WHEN allows you to set multiple logical tests, which can help you create buckets or categories.  In Excel, you'd have to nest multiple logical tests in an IF statement (ie. IF(logical_test, value_if_true, IF(other_logical_test, value_if_true, value_if_false))...very messy). With CASE WHEN, you can add an infinite number very easily.
Let's say that rather than caring about the exact revenue for each transaction, you only cared whether it was under \$10, between  \$10 and \$15, or over \$15. That's easy to do with CASE WHEN.  We'll include the revenue column as well so you can more easily see what's going on:
Can you write Python code for it?

case2_q = widgets.Textarea(value=
'''SELECT
    book,
    revenue,
    CASE WHEN revenue < 10 THEN "<$10"
        WHEN revenue BETWEEN 10 AND 15 THEN "$10-15"
        WHEN revenue > 15 THEN ">$15"
        END AS revenue_category
FROM
    sales_table
    ''',
width = '50em', height = '15em')
display(case2_q)

case2_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(case2_b)
run_q(case2_q, case2_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercises:

Give the calculated column an alias, rerun
Add a line to the SELECT clause to find the average revenue per transaction and give the column an alias, rerun
Edit the average column so that your results are rounded to the nearest cent, rerun
Add a line to the SELECT clause to count the total number of transactions and give the column an alias, rerun
Add a 2 lines to the SELECT clause to see the minimum and maximum revenue earned on a single transaction, rerun
Add a line to the SELECT clause to see a count of the number of distinct books that appear in sales_table
Add a WHERE clause to only see results for the books "For Whom the Bell Tolls" and "Emma"
Can you write Python code for it?

sum_q_c ='''SELECT
    SUM(revenue) AS total_rev,
    ROUND(AVG(revenue), 2) AS avg_rev,
    COUNT(revenue) AS total_transactions,
    MAX(revenue) AS max_rev,
    MIN(revenue) AS min_rev,
    COUNT(DISTINCT(book)) AS distinct_books
FROM
    sales_table
WHERE
    book IN ('For Whom the Bell Tolls', 'Emma') '''
cheat(sum_q_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercises:

In the SELECT clause, change S.book to just book, rerun. What's going wrong?
Now change book to B.book, rerun.
Change S.revenue to B.revenue, rerun.  What's going wrong?
Change B.revenue to just revenue and rerun. Note that when you're joining tables, it's standard to use table aliases even on columns that don't need them. This makes it easier for someone to read your query even if they are unfamaliar with the tables that you're working with.  However, as you see, it's not technically necessary.
Give each of these columns an alias (any alias) and rerun.

Challenge:
Write a query to view books and the author's country by joining auth_table and book_table.  Give auth_table the alias A and book_table the alias B. Use the aliases in the ON part of the JOIN clause.
Can you write Python code for it?

as_table_chall = widgets.Textarea(value='', width = '50em', height = '10em')
display(as_table_chall)

as_table_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(as_table_chall_b)
run_q(as_table_chall, as_table_chall_b)

as_table_chall_c ='''SELECT
    B.book,
    A.country
FROM
    book_table B
    JOIN auth_table A ON B.author = A.last_name'''
cheat(as_table_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Quick Exercises:

Replace > with <, rerun
Add an = directly after the <, rerun
Change the line to revenue BETWEEN 10 AND 12, rerun

Challenge:
Write a query that returns all columns from the auth_table for authors with a birth_year before 1800:
Can you write Python code for it?

born_chall = widgets.Textarea(value='',width = '50em', height = '10em')
display(born_chall)

born_chall_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(born_chall_b)
run_q(born_chall, born_chall_b)

born_chall_c ='''SELECT
    *
FROM
    auth_table
WHERE
    birth_year < 1800
'''
cheat(born_chall_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
SELECT

      table_x.column_a,

      table_y.column_b,

FROM

     table_x

     RIGHT JOIN table_y                  ➞ see all results from the second ("right") table, results where available from first table

           ON table_x.key_column = table_y.key_column

It's exactly the same as LEFT JOIN, except it prioritizes the second table (the "right" table) over the first ("left") table. We can't practice it because SQLite doesn't support it, and it's super redundant anyway because we can just use LEFT JOIN. Boom. We're done with RIGHT JOIN.

Previous        |       Table of Contents       |        Next

SELECT

      table_x.column_a,

      table_y.column_b,

FROM

     table_x

     OUTER JOIN table_y                  ➞ see all results from BOTH the first and second table

           ON table_x.key_column = table_y.key_column

What if you want to see all values from both tables? You can do this with an OUTER JOIN.  Unfortunately, MySQL and SQLite (what we're using right now!) doesn't support it, so we can't practice it.
If you are using Oracle or Microsoft SQL, then you'd use the example code above.  For MySQL and SQLite, there is a workaround. You don't need to understand what's going on in the code for now, just look at the output to make sure you understand what the OUTER JOIN output should look like.
Can you write Python code for it?

full_join_q = widgets.Textarea(value=
'''SELECT
    ceo,
    tech_cos.company,
    public_cos.company,
    share_price
FROM
    tech_cos
    LEFT JOIN public_cos ON tech_cos.company = public_cos.company
UNION ALL
SELECT
    ' ',
    ' ',
    public_cos.company,
    share_price
FROM
    public_cos
WHERE
    public_cos.company NOT IN (SELECT company FROM tech_cos)''',
width = '50em', height = '26em')
display(full_join_q)

full_join_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(full_join_b)
run_q(full_join_q, full_join_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
These results are inaccurate because instead of telling SQL to only return authors with average COGs over \$11, we've told SQL "only consider rows where COGs are over \$11".  SQL dropped the rows with COGs under \$11 before it started grouping and averaging.
SQL's Order of Execution
When we read, we start at the top of a page and work our way to the bottom. That's not how SQL works. It actually starts with the FROM clause and jumps around.  It's helpful to understand the order it follows to determine when to use HAVING and when to use WHERE.

We write the clauses in this order:

SELECT

FROM

      JOIN...ON

WHERE

GROUP BY...HAVING

ORDER BY

LIMIT

However, SQL reads and executes the clauses in this order:

FROM

      JOIN...ON

WHERE

SELECT

GROUP BY...HAVING

ORDER BY

LIMIT

Here's a query we've seen before, but now we've added a few more clauses so that we can see all of them in action:
Can you write Python code for it?

sql_order_q = widgets.Textarea(value=
'''SELECT
    author,
    AVG(cogs)
FROM
    book_table
WHERE
    author != 'Faulkner'
GROUP BY
    author HAVING AVG(cogs) > 11
ORDER BY
    AVG(cogs)
LIMIT 3''',
width = '50em', height = '17.5em')
display(sql_order_q)

sql_order_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(sql_order_b)
run_q(sql_order_q, sql_order_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
This seems relatively straightforward - but it's easy to forget and wind up with inaccurate results. Consider the following query. Why would this be wrong?
Can you write Python code for it?

hw_q3 = widgets.Textarea(value=
'''SELECT
    author,
    AVG(cogs)
FROM
    book_table
WHERE
    author != 'Faulkner'
    AND cogs > 11
GROUP BY
    author''',
width = '50em', height = '15em')
display(hw_q3)

hw_b3 = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(hw_b3)
run_q(hw_q3, hw_b3)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
This time, SQL can't find a match for every value in the two different company columns:

So it performs an "INNER JOIN". You can write either JOIN or INNER JOIN - SQL will interpret them as the same thing. This eliminates any rows that don't have matching values, then combined the tables:

Previous        |       Table of Contents       |        Next

SELECT

      table_x.column_a,

      table_y.column_b,

FROM

     table_x

     LEFT JOIN table_y                  ➞ see all results from the first ("left") table & results when available from second table

           ON table_x.key_column = table_y.key_column

If you want to make sure you see all the rows from a particular table - even if there's no match in the other table - you can do a LEFT JOIN instead. It lets you prioritize the results from one table over another.  Let's say your priority is to see all tech companies in your result-set, but you also want to see the share_price when that data is available:
Can you write Python code for it?

left_q = widgets.Textarea(value=
'''SELECT
    *
FROM
    tech_cos
    LEFT JOIN public_cos
        ON tech_cos.company = public_cos.company''',
width = '50em', height = '11em')
display(left_q)

left_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(left_b)
run_q(left_q, left_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Using CASE WHEN to create a pivot table
Say you want to see revenue broken out by gender and by date.  Right now, the only way we know how to do this is to add "date" to the GROUP BY clause. The query below is the same as the one from your last challenge, only we've added date to both the SELECT clause and the GROUP BY clause.
Can you write Python code for it?

case_pivot_ex = widgets.Textarea(value='''SELECT
    date,
    CASE WHEN
        B.author = 'Austen' THEN 'Female'
        ELSE 'Male'
        END AS gender,
    SUM(S.revenue) AS total_revenue
FROM
    book_table B
    JOIN sales_table S ON B.book = S.book
GROUP BY
    date, gender'''
, width = '50em', height = '18em')
display(case_pivot_ex)

case_pivot_ex_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(case_pivot_ex_b)
run_q(case_pivot_ex, case_pivot_ex_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Using UNION to add totals and subtotals:
Take a look at the query below. You'll see it pulls the COGs and book title for each book. It also uses UNION ALL to add a final line - a summary row averaging all cogs:
Can you write Python code for it?

union_q2 = widgets.Textarea(value=
'''SELECT
    book,
    cogs
FROM
    book_table

UNION ALL

SELECT
    'Average COGs',
    ROUND(AVG(cogs), 2)
FROM
    book_table''',
width = '50em', height = '19em')
display(union_q2)

union_b2 = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(union_b2)
run_q(union_q2, union_b2)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
Using this program
The content you're currently reading is written in Python, Markdown and HTML and runs in a Jupyter Notebook.  No need to know what any of that means, I only told you in case you were terribly curious.
You will not be using this interface at work - the point of this class is to teach you SQL the language, which can be typed into a variety of different software programs. You'll be able to learn the quirks of a different software program pretty easily as long as you know SQL.
Still, there are some things you should know about this program to help you with the class:
* If you accidentally double-click on a block of text, and suddenly it looks like code, hit SHIFT-RETURN or 'Cell ➤ Run'.
* If you try to run a query and the output doesn't refresh, select 'Cell ➤ Run All' to reboot the program.
* If you accidentally delete a cell, click 'Edit ➤ Undo Delete Cell'
* Nothing that you write in the challenges and exercises will save after you close this program.
* If you want to save something that you've written, follow the steps below:

Previous       |       Table of Contents       |       Next

Structure and Formatting Query Basics:
Below is an example of a query, SQL code that requests data from a database. Try to make a habit of writing queries by following these formatting conventions. Queries can get very long and complicated, and formatting makes them easier to read.

Previous       |       Table of Contents       |       Next

Microsoft SQL Server | MySQL | Oracle | SQLite
:------------------: | :---: | :----: | :----:
SP_Help...
Can you write Python code for it?

pragma = widgets.Textarea(
    value='''PRAGMA TABLE_INFO(book_table)''',
    width = '50em', height = '3em')
display(pragma)

prag_button = widgets.Button(description='Run',width='10em', height='2.5em', color='white', background_color='black', border_color='black')
display(prag_button)
run_q(pragma, prag_button)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
We've now added some rows to auth_table and book_table so we can practice our new join skills!
Challenge
Start simple by taking a look at the new rows we've added. Write a query to see all columns and rows from book_table, then change the query so you can take a look at auth_table instead:
Can you write Python code for it?

join_drill1 = widgets.Textarea(value='',width = '50em', height = '7em')
display(join_drill1)

join_drill1_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(join_drill1_b)
run_q(join_drill1, join_drill1_b)

join_drill1_c ='''SELECT
    *
FROM
    auth_table       [change to "book_table" for second part of challenge]'''
cheat(join_drill1_c)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
What just happened? SQL went through these steps:

First, SQL pulled up the two tables that we named in the FROM clause: FROM book_table JOIN auth_table.
Then it identified the "key" columns that we named with ON: ON book_table.author = auth_table.last_name.

Next, it matched up the values on the key columns:

Whenever it found a match, it made a kind of copy of the row from the auth_table and pasted it to the book_table

Finally, it literally "joined" the two tables by returning their columns in a single table

Quick Exercises:
The JOIN query we've been discussing has been reproduced in the box below for these exercises.
1. Change the query so you only see the book, first_name and the author's last name (you can do this with either author or last_name), and the birth_year, then rerun.
2. Add a WHERE clause so that you only see books by Hemingway and Austen, rerun.
3. Add an ORDER BY clause so that the author born first appears first, and so that their books appear in alphabetical order. Rerun.
Can you write Python code for it?

join_q2 = widgets.Textarea(value=
'''SELECT
    *
FROM
    book_table
    JOIN auth_table
        ON book_table.author = auth_table.last_name''',
width = '50em', height = '18em')
display(join_q2)

join_b2 = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(join_b2)
run_q(join_q2, join_b2)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
What's going on:
When you use COUNT(first_name), SQL skipped over the Homer row because there was no value in the first_name column. COUNT(*), on the other hand, looks across all columns, so as long as a row has a value in at least one column, it'll get included in the count. You might argue that you could just use country or last_name, but they fact is that * is just way easier and less time-consuming to type out.  Overwhelmingly, people opt for COUNT(*) instead of COUNT(column_name) unless they are interested in overlooking NULL values.

Functions + Operators
You can use functions together with operators to do more complex calculations.  Below, we've calculated our total gross profit using both the SUM() function and subtraction:
Can you write Python code for it?

fun_op_q = widgets.Textarea(value=
'''SELECT
    SUM(S.revenue) - SUM(B.cogs) AS gross_profit
FROM
    book_table B
    JOIN sales_table S ON S.book = B.book''',
width = '50em', height = '8em')
display(fun_op_q)

fun_op_b = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(fun_op_b)
run_q(fun_op_q, fun_op_b)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
With a LEFT JOIN, SQL still starts by looking for matching values.

When it fails to find a match, it will still keep the values on the "left" table, but get rid of the unmatched values on the "right" table.

Quick Exercise:
The query from above has been reproduced below for these exercises.
1. Figure out how to change the query above so that you only see the company column appear once.
2. Rewrite the query so that public_cos becomes the priority table instead.
Can you write Python code for it?

left_q2 = widgets.Textarea(value=
'''SELECT
    *
FROM
    tech_cos
    LEFT JOIN public_cos
        ON tech_cos.company = public_cos.company''',
width = '50em', height = '11em')
display(left_q2)

left_b2 = widgets.Button(description='Run', width='10em', height='2.5em', color='white',background_color='black', border_color='black')
display(left_b2)
run_q(left_q2, left_b2)


I am working on the file "Code/SQL/SQL_Bootcamp_Stern_2016.ipynb".
The first task is:
name tells us the names of each column in the table. So now we know that the book_table has columns headed Book, COGs, and Author
type tells us what type of data is in each column. So now we know that the Book column has TEXT data, and that COGs contains REAL numbers - numbers that can have a fractional value.
All other columns you can ignore. Seriously.

Quick Exercise:
Change the query above to look at the auth_table instead. Why is the author's birth_year data type not REAL like we saw with COGs?

Challenge:
Rewrite the query above to take a look at the sales_table structure. Judging from what your query returns, can you guess what you'll probably see once you actually look all the data in the sales_table?
Can you write Python code for it?

pragma_sales = widgets.Textarea(
    value='',
    width = '50em',
    height = '4em'
)
display(pragma_sales)

prag_sales_button = widgets.Button(description='Run',width='10em', height='2.5em', color='white', background_color='black', border_color='black')
display(prag_sales_button)
run_q(pragma_sales, prag_sales_button)

prag_sales_answer =  'PRAGMA TABLE_INFO(sales_table)'
cheat(prag_sales_answer)


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
A column with the name customer_id appears in both the cust_table and the sales_table. SQL doesn't know which one you want to see. You have to tell it from which table you want the customer_id.
This can be important when columns in different tables have the same names but totally unrelated values.
Look at the sales_table again:
Can you write Python code for it?

run('''
    SELECT
        *
    FROM
        sales_table
    LIMIT 5
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
ALIASES
SELECT
    T.column_a AS alias_a    # creates a nickname for column_a, and states that it's from table_name (whose alias is T)
FROM
    table_name AS T          # creates a nickname for table_name
WHERE
    alias_a = z           # refer to an alias in the WHERE clause
ORDER BY
    alias_a                  # refer to an alias in the ORDER BY clause

Aliases are optional, but save you time and make column headers cleaner
AS isn't necessary to create and alias, but commonly used
The convention is to use "AS" in the "SELECT" clause, but not in the "FROM" clause.
More on Aliases: http://www.w3schools.com/sql/sql_alias.asp

Change the aiases for model_id and revenue, or add extra columns to see how they work:
Can you write Python code for it?

run('''
    SELECT
        model_id AS Model_of_car,
        revenue AS Rev_per_car
    FROM
        sales_table
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Above, there's a column called "id".
Now look at the salesman_table again:
Can you write Python code for it?

run('''
    SELECT
        *
    FROM
        salesman_table
    LIMIT 5
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Add on the average amount of revenue made per sale:
Can you write Python code for it?

run('''
    SELECT
        first_name ||  ' ' || last_name as Salesperson,
        COUNT(*) as Cars_Sold,
        ROUND(AVG(revenue)) as Revenue_per_Sale
    FROM
        sales_table S
        JOIN salesman_table M ON S.salesman_id = M.id
    GROUP BY
        Salesperson
    ORDER BY
        Cars_Sold DESC
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
All model_ids are returned, but the averages are all much lower than they should be. That's because the query first drops all rows that have revenue greater than 18000, and then averages the remaining rows.
When you use HAVING, SQL follows these steps instead (this query should look like the one you wrote in the last challenge):
SELECT
    C.model as Car_Model,
    AVG(S.revenue) as Avg_Revenue
FROM
    sales_table S
    JOIN car_table C on S.model_id = C.model_id
GROUP BY
    Car_Model HAVING Avg_Revenue < 18000

Find the sales_table and join it to the car_table (same as before)
Pull the data from the 'model' column in car_table and 'revenue' column in sales_table (same as before)
Average the rows for each Car_Model
Return only the Car_Models whose averages are less than 18,000

And as you can see, there's a big difference in these results and the results of the query that used "WHERE" instead of HAVING:
Can you write Python code for it?

run('''
    SELECT
        C.model as Car_Model,
        AVG(S.revenue) as Avg_Revenue
    FROM
        sales_table S
        JOIN car_table C on S.model_id = C.model_id
    GROUP BY
        Car_Model HAVING Avg_Revenue < 18000
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Be careful with LIKE though - it can't deal with extra characters or mispellings:
Can you write Python code for it?

run('''
    SELECT
        *
    FROM
        sales_table
    WHERE
        payment_type LIKE 'ces%'
    LIMIT 5
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Because SQLite doesn't support ROLLUP, the query below is just intended to illustrate how ROLLUP would work. Don't worry about understanding the query itself, just get familiar with what's going on in the result-set:
Can you write Python code for it?

run('''
    SELECT
        C.model AS Car_Model,
        SUM(S.revenue) as Sum_Revenue
    FROM
        sales_table S
        JOIN car_table C on S.model_id = C.model_id
    GROUP BY C.model

    UNION ALL

    SELECT
        'NULL',
        SUM(S.revenue)
    FROM
        sales_table S
''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
CASE WHEN also lets you create new groups.  Start by looking at the cust_table grouped by age - remember that COUNT(***) tells you how many rows are in each group (which is the same as telling you the number of customers in each group):
Can you write Python code for it?

run('''
    SELECT
        age,
        COUNT(*) customers
    FROM
        cust_table
    GROUP BY
        age
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
CASE WHEN makes this same information a lot easier to read by letting you pivot the result set a little.
Write a query using CASE WHEN to look at total revenue per gender, grouped by each car model
Can you write Python code for it?

run('''
    SELECT NULL
    ''')
#print(case_cheat)


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
CONCATENATING:
Concatenating varies by RDBMS:
Can you write Python code for it?


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Conditional Expressions: IF & CASE WHEN
SELECT
    CASE WHEN column_a = x THEN some_value
         WHEN column_a = y THEN some_value2
         ELSE some_other_value
         END some_alias                         # alias optional after END
FROM
    table_name

Conditional expressions let you use IF/THEN logic in SQL
In SQLite, you have to use CASE WHEN, but in other RDBMS you may prefer to use IF, depending on your needs
More on CASE WHEN: http://www.dotnet-tricks.com/Tutorial/sqlserver/1MS1120313-Understanding-Case-Expression-in-SQL-Server-with-Example.html
Can you write Python code for it?


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
DO use LIKE on small tables and in simple queries:
LIKE is helpful if you know where to find something but you can't quite remember what it's called. Try to use a wildcard sparingly - don't use 2 when 1 will suffice:
Can you write Python code for it?

run('''
    SELECT
        model_id, model
    FROM
        car_table
    WHERE
        model LIKE '%undra'
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
DO use a UNION to look at result-sets that aren't mutually exclusive
Let's say you were interested in seeing all Toyotas as well as cars with COGs of more than 13000. Write a query for the first group, then a query for the second group, and unite them with UNION. The result set won't show you repeats - if a row matches both result sets, it will only display once.
Can you write Python code for it?

run('''
    SELECT
        make, model, cogs
    FROM
        car_table
    WHERE
        make = 'Toyota'

    UNION

    SELECT
        make, model, cogs
    FROM
        car_table
    WHERE
        cogs > 13000
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
DON'T use LIKE on large tables or when using JOINs:
Can you write Python code for it?

run('''
    SELECT
        C.model,
        AVG(revenue)
    FROM
        sales_table S
        JOIN car_table C on S.model_id = C.model_id
    WHERE
        C.model LIKE '%undra'
''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
DON'T use an asterisk unless you absolutely have to:
This can put a lot of strain on servers. Only use if you know for certain that your using a small table
Can you write Python code for it?

run('''
    SELECT
        *
    FROM
        sales_table
    ''').head()


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
DON'T use negative filters when a positive filter is possible
Let's say you want to look at cars made by Toyato and Honda, but you don't care about Subaru. It might be tempting to use a negative filter:
Can you write Python code for it?

run('''
    SELECT
        *
    FROM
        car_table
    WHERE
        make != 'Subaru'
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Different RDBMS have different datatypes available:
- Oracle: http://docs.oracle.com/cd/B10501_01/appdev.920/a96624/03_types.htm
- MySQL:
 - Numeric: http://dev.mysql.com/doc/refman/5.0/en/numeric-type-overview.html
 - Date/time: http://dev.mysql.com/doc/refman/5.0/en/date-and-time-type-overview.html
 - String/text: http://dev.mysql.com/doc/refman/5.0/en/string-type-overview.html
- SQLite: https://www.sqlite.org/datatype3.html
- Microsoft: http://msdn.microsoft.com/en-us/library/ms187752.aspx

SELECT & FROM:

Basically every "read" query will contain a SELECT and FROM clause
In the SELECT clause, you tell SQL which columns you want to see
In the FROM clause, you tell SQL the table where those columns are located
More on SELECT: http://www.w3schools.com/sql/sql_select.asp

SELECT * (ALL COLUMNS)
SELECT          # specifies which columns you want to see
    *           # asterisk returns all columns
FROM            # specifies the table or tables where these columns can be found
    table_name

Use an asterisk to tell SQL to return all columns from the table:
Can you write Python code for it?

run('''
    SELECT
        *
    FROM
        sales_table
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Essentially, in Venn Diagram terms, and outer join lets you see all contents of both circles. This join will let you see all pet owners, regardless of whether the own only a cat or only a dog

Using the "WHERE" Clause to Join Tables
SELECT
    *
FROM
    table_x X
    JOIN table y Y
WHERE
    X.column_a = Y.column_a         # tells SQL the key for the join

Some people prefer to use the WHERE clause to specify the key for a join
Fine if the query is short, but SUPER messy when the query is complex
We won't use this moving forward, but it's good to see it in case you run across someone else's code and you need to make sense of it

When it's simple, it's not so bad:
Can you write Python code for it?

run('''
    SELECT
        C.model,
        S.revenue
    FROM
        sales_table S, car_table C
    WHERE
        S.model_id = C.model_id
    LIMIT 5
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
GROUP BY, when used with joins and functions, can help you quickly see trends in your data. Parse out what's going on here:
Can you write Python code for it?

run('''
    SELECT
        C.model AS Car_Model,
        MIN(S.revenue) || ' - ' || MAX(S.revenue) AS Min_to_Max_Sale,
        MAX(S.revenue) - MIN(S.revenue) AS Range,
        ROUND(AVG(S.revenue), 2) AS Average_Sale
    FROM
        sales_table S
        JOIN car_table C on S.model_id = C.model_id
    GROUP BY
        Car_Model
    ORDER BY
        Average_Sale DESC
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
GROUP BY:
SELECT
    column_a,
    SUM(column_b)    # sums up the values in column_b
FROM
    table_name
GROUP BY             # creates one group for each unique value in column_a
    column_a

Creates a group for each unique value in the column you specify
Extremely helpful when you're using functions - it segments out results
More on GROUP BY: http://www.w3schools.com/sql/sql_groupby.asp

The query below creates a group for each unique value in the car_table's model column, then sums up the revenue for each group. Note that you can use an alias in the GROUP BY clause.
Can you write Python code for it?

run('''
    SELECT
        C.model AS Car_Model,
        SUM(revenue) AS Total_Revenue
    FROM
        sales_table S
        JOIN car_table C on S.model_id = C.model_id
    GROUP BY
        Car_Model
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
HAVING vs WHERE:
WHERE filters which rows will be included in the function, whereas HAVING filters what's returned after the function has been applied.
Take a look at the query below. It might look like the query you just wrote (above) if you'd tried to use WHERE instead of HAVING:
SELECT
    C.model as Car_Model,
    AVG(S.revenue) as Avg_Revenue
FROM
    sales_table S
    JOIN car_table C on S.model_id = C.model_id
WHERE
    S.revenue < 18000
GROUP BY
    Car_Model

Find the sales_table and join it to the car_table
Pull the data from the 'model' column in car_table and 'revenue' column in sales_table
Filter out all rows where revenue is less than 18000
Average remaining rows for each Car_Model

Even though AVG( ) appears early in the query, it's not actually being applied until after the WHERE statement has filtered out rows with less than 18,000 in revenue.
This is the result:
Can you write Python code for it?

run('''
    SELECT
        C.model as Car_Model,
        AVG(S.revenue) as Avg_Revenue
    FROM
        sales_table S
        JOIN car_table C on S.model_id = C.model_id
    WHERE
        S.revenue < 18000
    GROUP BY
        Car_Model
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
If you want to look at average revenue for car models that are like "%undra", run the LIKE query on the small table (car_table) first to figure out exacly what you're looking for, then use that information to search for the data you need from the sales_table
DO dip your toe in by starting with a small data set
Use WHERE to only view a few days of data at first. If the query runs quickly, add a few days at a time. If it starts to run slowly, run just a few days at a time and paste results into excel to combine results (or use Python...ask me later!!!).
The query below won't work because SQLite doesn't recognize dates, but remember these concepts when working with other RDBMS
Can you write Python code for it?

run('''
    SELECT
        revenue,
        date
    FROM
        sales_table
    WHERE
        date = '1/1/14'
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
JOINS
SELECT
    *
FROM
    table_x
    JOIN table_y                                 # use JOIN to add the second table
        ON table_x.column_a = table_y.column_a   # use ON to specify which columns correspond on each table

Joining tables is the most fundamental and useful part about relational databases
Use columns on different tables with corresponding values to join the two tables
The format "table_x.column_a" can be read as "column_a from table_x"; it tells SQL the table where it can find that column
More on JOINS: http://www.w3schools.com/sql/sql_join.asp

Start by looking at the first few rows of sales_table again:
Can you write Python code for it?

run('''
    SELECT
        *
    FROM
        sales_table
    LIMIT 5
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Join the sales_table (assign it the alias S) and salesman_table (alias SM) again.
- Select the id and salesman_id column from the sales_table
- Also, select the id column from the salesman_table
- Optional: assign aliases to the columns in the SELECT clause to make the result-set easier to read
Can you write Python code for it?

run('''
    SELECT NULL
    ''')
#print(join_cheat5)


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
LIMIT
SELECT
    column_a
FROM
    table_name
WHERE
    columna_a = x      # optional
ORDER BY
    column_a           # optional
LIMIT                  # Limits the result-set to N rows
    N

LIMIT just limits the number of rows in your result set
More on LIMIT: http://www.w3schools.com/sql/sql_top.asp
The ability to limit results varies by RBDSM. Below you can see the different ways to do this:
Can you write Python code for it?


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Look at what's going on in that query without the AVG( ) function and the GROUP BY clause:
Can you write Python code for it?

run('''
    SELECT
        C.Make as Car_Maker,
        payment_type,
        S.revenue
    FROM
        sales_table S
        JOIN car_table C on S.model_id = C.model_id
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Make it easier to compare the average revenue of Jared's sales to the average revenue of per sale overall by adding a column to see by what percent each salesperson's sales are more or less than average:
Can you write Python code for it?

run('''
    SELECT
        first_name ||  ' ' || last_name as Salesperson,
        COUNT(*) as Cars_Sold,
        ROUND(AVG(revenue), 2) as Rev_per_Sale,
        ROUND((((AVG(revenue)
            - (SELECT AVG(revenue) from sales_table))
            /(SELECT AVG(revenue) from sales_table))*100), 1) || ' %'
            as RPS_Compared_to_Avg
    FROM
        sales_table S
        JOIN salesman_table M ON S.salesman_id = M.id
    GROUP BY
        Salesperson
    ORDER BY
        Cars_Sold DESC
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
NESTING

Nested queries allow you to put a query within a query
Depending on your needs, you might put a nested query in the SELECT clause, the FROM clause, or the WHERE clause

Consider the following query. We're using a nested query in the SELECT clause to see the sum of all revenue in the sales_table, and then using it again to what percentage of total revenue can be attributed to each Car_Model.
Can you write Python code for it?

run('''
    SELECT
        C.model AS Car_Model,
        SUM(S.revenue) AS Revenue_Per_Model,
        (SELECT SUM(revenue) FROM sales_table) AS Total_Revenue,
        SUM(S.revenue) / (SELECT SUM(revenue) FROM sales_table) AS Contribution_to_Revenue
    FROM
        sales_table S
        JOIN car_table C ON C.model_id = S.model_id
    GROUP BY
        Car_Model
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Notice that the Owner_Name columns on each table have some corresponding values (Michael, Gilbert, May and Elizabeth and Donna are in both tables), but they both also have values that don't overlap.
JOINS or INNER JOINS
SELECT
    *
FROM
    table_x X
    JOIN table_y Y ON X.column_a = Y.column_a      # Returns rows when values match on both tables.

This is what we used in the initial example.  Simple joins, (also called Inner Joins) will combine tables only where there are corresponding values on both tables.
Write a query below to join the Cat_Table and Dog_Table using the same method we've used before:
Can you write Python code for it?

run('''
    SELECT NULL
    ''')
#print(inner_join_cheat)


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Notice that the result-set only includes the names that are in both tables. Think of inner joins as being the overlapping parts of a Venn Diagram. So, essentially we're looking at results only where the pet owner has both a cat and a dog.

LEFT JOINS or LEFT OUTER JOINS
SELECT
    *
FROM
    table_x X
    LEFT JOIN table_y Y ON X.column_a = Y.column_a   # Returns all rows from 1st table, rows that match from 2nd

LEFT JOINS will return all rows from the first table, but only rows from the second table if a value matches on the key column.

Rewrite your query from above, but instead of "JOIN", write "LEFT JOIN":
Can you write Python code for it?

run('''
    SELECT NULL
    ''')
#print(left_join_cheat)


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Now let's add back the ROUND() and AVG() functions and the GROUP BY statement:
Can you write Python code for it?

run('''
    SELECT
        C.Make as Car_Maker,
        ROUND(AVG(CASE WHEN payment_type = 'cash' THEN S.revenue END)) AS Avg_Cash_Revenue,
        ROUND(AVG(CASE WHEN payment_type = 'finance' THEN S.revenue END)) AS Avg_Finance_Revenue
    FROM
        sales_table S
        JOIN car_table C on S.model_id = C.model_id
    GROUP BY
        C.Make
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Now the first few rows of the car_table:
Can you write Python code for it?

run('''
    SELECT
        *
    FROM
        car_table
    LIMIT 5
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
ORDER BY
SELECT
    column_a
FROM
    table_name
WHERE                 # optional
    column_a = x
ORDER BY              # sorts the result-set by column_a
    column_a DESC     # DESC is optional. It sorts results in descending order (100->1) instead of ascending (1->100)

Without an ORDER BY clause, the default result-set will be ordered by however it appears in the database
By default, ORDER BY will sort values in ascending order (A→Z, 1→100). Add DESC to order results in desceding order instead (Z→A, 100→1)
More on ORDER BY:  http://www.w3schools.com/sql/sql_orderby.asp

The query below orders the result-set by revenue amount, starting with the smallest amount listed first:
Can you write Python code for it?

run('''
    SELECT
        *
    FROM
        sales_table
    ORDER BY
        revenue DESC
    LIMIT 5
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
On a big table, this will run much more slowly than if you use a positive filter. Try this instead - it might require a little extra typing, but it will run much faster:
Can you write Python code for it?

run('''
    SELECT
        *
    FROM
        car_table
    WHERE
        make in ('Toyota', 'Honda')
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Optimization:
Non-optimized queries can cause a lot of problems because tables frequently have thousands or millions of rows:
If you haven't optimized your query, it might:

Take several minutes (or even hours) to return the information you're requesting
Crash your computer
Muck up the server's processes, and you'll face the wrath of your company's system administrators once they figure out that you are the reason why the whole system has slowed down and everyone is sending them angry emails (this will probably happen to you no matter what. It's a rite of passage).

Find a few more useful optimization tips here: http://hungred.com/useful-information/ways-optimize-sql-queries/
Some of these seem strange, because we're going ling you NOT to do a bunch of things that you've learned how to do. Stick to this principal: if you're dealing with a small table, you can break a few of these rules. The larger the table, the fewer rules you can break.
DO name specific columns in the SELECT CLAUSE:
Can you write Python code for it?

run('''
    SELECT
        date,
        revenue
    FROM
        sales_table
    ''').head()


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
Practice applying this "table_x.column_a" format to all columns in the SELECT clause when you are joining multiple tables, since multiple tables frequenty use the same column names even when they don't correspond.
It's common to use single-letter aliases for tables to make queries shorter. Take a look at the query below and make sure you understand what's going on with the table aliases. It's the same query that you wrote earlier, but with aliases to help identify the columns
Can you write Python code for it?

run('''
    SELECT
        S.customer_id,
        C.gender,
        S.revenue
    FROM
        sales_table AS S
        JOIN cust_table AS C on S.customer_id = C.customer_id
    ''')


I am working on the file "Code/SQL/SQL_Intro.ipynb".
The first task is:
ROLLUP
SELECT
    column_a,
    SUM(column_b)
FROM
    table_x
GROUP BY
    ROLLUP(column_a)     # adds up all groups' values in a single final row

Rollup, used with GROUP BY, provides subtotals and totals for your groups
Useful for quick analysis
Varies by RDBMS
Can you write Python code for it?