I am working on the file "Python/35_Segmentation_Shape_Analysis.ipynb".
The first task is:
Object Analysis
Once we have the segmented objects we look at their shapes and the intensity distributions inside the objects.
Note that sizes are in nanometers. ITK and consequently SimpleITK are agnostic of the actual measurement units. It is up to you as the developer to explicitly use the correct units and more importantly, DO NOT MIX UNITS.
We first compute all of the measurements we are interested in.
Can you write Python code for it?

shape_stats = sitk.LabelShapeStatisticsImageFilter()
shape_stats.ComputeOrientedBoundingBoxOn()
shape_stats.Execute(non_border_seg)

intensity_stats = sitk.LabelIntensityStatisticsImageFilter()
intensity_stats.Execute(non_border_seg, img)


I am working on the file "Python/35_Segmentation_Shape_Analysis.ipynb".
The first task is:
Seed based watershed segmentation
The bacteria appear to be segmented correctly from the background but not from each other. Using the visualization and histogram above we see that in 3D many of them are connected, even if on a slice by slice inspection they appear separate.
Can you write Python code for it?

dist_img = sitk.SignedMaurerDistanceMap(
    cleaned_thresh_img != 0,
    insideIsPositive=False,
    squaredDistance=False,
    useImageSpacing=False,
)
radius = 10
# Seeds have a distance of "radius" or more to the object boundary, they are uniquely labelled.
seeds = sitk.ConnectedComponent(dist_img < -radius)
# Relabel the seed objects using consecutive object labels while removing all objects with less than 15 pixels.
seeds = sitk.RelabelComponent(seeds, minimumObjectSize=15)
# Run the watershed segmentation using the distance map and seeds.
ws = sitk.MorphologicalWatershedFromMarkers(dist_img, seeds, markWatershedLine=True)
ws = sitk.Mask(ws, sitk.Cast(cleaned_thresh_img, ws.GetPixelID()))


I am working on the file "Python/35_Segmentation_Shape_Analysis.ipynb".
The first task is:
Segmentation
To allow us to analyze the shape of whole bacteria we first need to segment them. We will do this in several steps:
1. Separate the bacteria from the embedding resin background.
2. Mark each potential bacterium with a unique label, to evaluate the segmentation.
3. Remove small components and fill small holes using binary morphology operators (opening and closing).
4. Use seed based watersheds to perform final segmentation.
5. Remove bacterium that are connected to the image boundary.
Separate the bacteria from the background
Based on the visualization of the data above, it intuitively appears that the background and foreground are separable using a single intensity threshold. Our first step towards validating this observation is to plot the intensity distribution.
Can you write Python code for it?

plt.figure()
plt.hist(sitk.GetArrayViewFromImage(img).flatten(), bins=100)
plt.show()


I am working on the file "Python/35_Segmentation_Shape_Analysis.ipynb".
The first task is:
The histogram above shows tens of thousands of very small labels which are not visually detected by looking at the segmentation.
Remove small islands and holes
Using binary morphological operations we remove small objects using the opening operation and fill small holes using the closing operation. The use of opening and closing by reconstruction maintains the boundary of the original objects.
Can you write Python code for it?

cleaned_thresh_img = sitk.BinaryOpeningByReconstruction(thresh_img, [10, 10, 10])
cleaned_thresh_img = sitk.BinaryClosingByReconstruction(
    cleaned_thresh_img, [10, 10, 10]
)

gui.MultiImageDisplay(
    image_list=[sitk.LabelOverlay(img, cleaned_thresh_img)],
    title_list=["Cleaned Binary Segmentation"],
    figure_size=(8, 4),
);


I am working on the file "Python/35_Segmentation_Shape_Analysis.ipynb".
The first task is:
The histogram is bi-modal with a clear separation, which we have manually identified as having an intensity value of 120.
We can also use one of several binary threshold selection filters available in SimpleITK.
Can you write Python code for it?

threshold_filters = {
    "Otsu": sitk.OtsuThresholdImageFilter(),
    "Triangle": sitk.TriangleThresholdImageFilter(),
    "Huang": sitk.HuangThresholdImageFilter(),
    "MaxEntropy": sitk.MaximumEntropyThresholdImageFilter(),
}

filter_selection = "Manual"
try:
    thresh_filter = threshold_filters[filter_selection]
    thresh_filter.SetInsideValue(0)
    thresh_filter.SetOutsideValue(1)
    thresh_img = thresh_filter.Execute(img)
    thresh_value = thresh_filter.GetThreshold()
except KeyError:
    thresh_value = 120
    thresh_img = img > thresh_value

print("Threshold used: " + str(thresh_value))
gui.MultiImageDisplay(
    image_list=[sitk.LabelOverlay(img, thresh_img)],
    title_list=["Binary Segmentation"],
    figure_size=(8, 4),
);


I am working on the file "Python/36_Microscopy_Colocalization_Distance_Analysis.ipynb".
The first task is:
Distance of colocalizations from nuclei
We now compute the distances between marker colocalization blobs to nuclei blobs, both edge (boundary) to edge distance and center to center distance. The centroid of a blob is defined as the location in physical space of the mean of the blob voxel locations. Consequentially, if the blob is not convex the centroid may lie outside the blob.
Edge to edge distances are computed using a distance map, this treats all nuclei as a single object.
Can you write Python code for it?

# Compute the edge to edge distances using the absolute distance map from the nuclei segmentation.
distance_map_from_all_nuclei = sitk.Abs(
    sitk.SignedMaurerDistanceMap(
        dapi_binary_segmentation, squaredDistance=False, useImageSpacing=True
    )
)

distance_stats_filter = sitk.LabelIntensityStatisticsImageFilter()
distance_stats_filter.Execute(
    colocolization_labeled_segmentation, distance_map_from_all_nuclei
)

labels_edge_distances = []
for label in distance_stats_filter.GetLabels():
    # Using minimum for each label gives us edge to edge distance
    labels_edge_distances.append(
        colocalization_labels_2_original_labels[label]
        + [distance_stats_filter.GetMinimum(label)]
    )

ee_distances_df = pd.DataFrame(
    labels_edge_distances, columns=markers_names + ["edge edge distance to DAPI [um]"]
)
ee_distances_df.to_csv(
    os.path.join(OUTPUT_DIR, "colocalization_edge_edge_distances.csv"), index=False
)

# Display the first N rows as HTML, sorted according to distance from nuclei
head_length = 20
display(
    HTML(
        ee_distances_df.sort_values(by="edge edge distance to DAPI [um]")
        .head(head_length)
        .to_html(index=False)
    )
)


I am working on the file "Python/36_Microscopy_Colocalization_Distance_Analysis.ipynb".
The first task is:
Distance of colocalizations from nuclei centers
Determining how close proteins (or genes if that is your interest) is to the center of the nucleus. Note that this distance is not symmetric, the distance between a protein to the closest nucleus is not the same as the distance between that nucleus and the closest protein.
Can you write Python code for it?

colocalization_labels, colocalization_centroids = zip(
    *[
        (coloc_label, colocolization_stats_filter.GetCentroid(coloc_label))
        for coloc_label in colocolization_stats_filter.GetLabels()
    ]
)
colocalization_centroids = np.array(colocalization_centroids)

nuclei_labels, nuclei_centroids = zip(
    *[
        (nucleus, dapi_stats_filter.GetCentroid(nucleus))
        for nucleus in dapi_stats_filter.GetLabels()
    ]
)
nuclei_centroids = np.array(nuclei_centroids)

# Compute minimal distances and matching labels
all_distances = -2 * np.dot(colocalization_centroids, nuclei_centroids.T)
all_distances += np.sum(colocalization_centroids**2, axis=1)[:, np.newaxis]
all_distances += np.sum(nuclei_centroids**2, axis=1)
all_distances = np.sqrt(all_distances)

min_indexes = np.argmin(all_distances, axis=1)

results = list(
    zip(
        colocalization_labels,
        tuple(np.array(nuclei_labels)[min_indexes]),
        all_distances[np.arange(len(min_indexes)), min_indexes],
    )
)
# Replace the colocalization labels with the original channel labels
results = [
    colocalization_labels_2_original_labels[coloc_label]
    + list((nucleus_label, distance))
    for coloc_label, nucleus_label, distance in results
]

cc_distances_df = pd.DataFrame(
    results, columns=markers_names + ["DAPI", "centroid centroid distance [um]"]
)
cc_distances_df.to_csv(
    os.path.join(OUTPUT_DIR, "colocalization_centroid_centroid_distances.csv"),
    index=False,
)

# Display the first N rows as HTML
head_length = 20
display(HTML(cc_distances_df.head(head_length).to_html(index=False)))


I am working on the file "Python/36_Microscopy_Colocalization_Distance_Analysis.ipynb".
The first task is:
Load data
Load the 3D multi channel structured illumination microscopy image, split it into the separate channels and display them.
Can you write Python code for it?

sim_image = sitk.ReadImage(fdata("microscopy_colocalization.nrrd"))

# Our original image was saved using mm as the unit size, but in microscopy the more common unit is um.
# We modify the size so that it is in um. If your image is in um you won't have to perform this modification.
sim_image.SetSpacing([spc * 1000 for spc in sim_image.GetSpacing()])
sim_image.SetOrigin([org * 1000 for org in sim_image.GetOrigin()])

# The channel name appears in the image's meta-data dictionary with key: channel_i_name
channel_titles = [
    sim_image.GetMetaData(f"channel_{i}_name")
    for i in range(sim_image.GetNumberOfComponentsPerPixel())
]
channels = [
    sitk.VectorIndexSelectionCast(sim_image, i)
    for i in range(sim_image.GetNumberOfComponentsPerPixel())
]
fitc_image = channels[channel_titles.index("FITC")]
cy3_image = channels[channel_titles.index("Cy3")]
dapi_image = channels[channel_titles.index("DAPI")]

gui.MultiImageDisplay(
    image_list=channels,
    title_list=channel_titles,
    shared_slider=True,
    intensity_slider_range_percentile=[0, 100],
    figure_size=(20, 10),
);


I am working on the file "Python/4 Automatic Theorem Proving/AST-2-Dot.ipynb".
The first task is:
The function assign_numbers takes three arguments:
- t is a nested tuple that is interpreted as a tree,
- Nodes2Numbers is dictionary,
- n is a natural number.
Given a tree t that is represented as a nested tuple, the function assign_numbers assigns a unique natural number
to every node of t.  This assignment is stored in the dictionary Nodes2Numbers. n is the first natural number
that is used.  The function returns the smallest natural number that is still unused.
Can you write Python code for it?

def assign_numbers(address, t, Nodes2Numbers, n=0):
    Nodes2Numbers[address] = str(n)
    if isinstance(t, str) or isinstance(t, int):
        return n + 1
    n += 1
    j  = 1
    for t in t[1:]:
        n = assign_numbers(address + (j,), t, Nodes2Numbers, n)
        j += 1
    return n


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Back to top
Critical Pairs
The central notion of the Knuth-Bendix algorithm is the notion of a critical pair.
Given two equations lhs1 = rhs1 and lhs2 = rhs2, a pair of terms (s, t) is a critical pair of these equations if we have the following:
- u is a non-trivial position in lhs1, i.e. lhs1/u is not a variable,
- The subterm lhs1/u is unifiable with lhs2, i.e.
  $$\mu = \texttt{mgu}(\texttt{lhs}_1 / \texttt{u}, \texttt{lhs}_2) \not= \texttt{None},$$
- $s = \texttt{lhs}_1\mu[\texttt{u} \mapsto \texttt{rhs}_2\mu]$ and $t = \texttt{rhs}_1\mu$.
The idea is then that the term $\texttt{lhs1}\mu$ can be rewritten into different ways:
- $\texttt{lhs1}\mu \rightarrow \texttt{rhs1}\mu = t$,
- $\texttt{lhs1}\mu \rightarrow \texttt{lhs}_1\mu[\texttt{u} \mapsto \texttt{rhs}_2\mu] = s$.
The function critical_pairs implemented in this section computes the critical pairs between two rewrite rules.
Given a term t, the function non_triv_positions computes the set $\mathcal{P}os(t)$ of all positions in t that do not point to variables.  Such positions are called non-trivial positions.  Given a term t, the set $\mathcal{P}os(t)$ of all positions in $t$ is defined by induction on t.
1. $\mathcal{P}os(v) := \bigl{()\bigr} \quad \mbox{if $v$ is a variable} $
2. $\mathcal{P}os\bigl(f(t_0,\cdots,t_{n-1})\bigr) :=
     \bigl{()\bigr} \cup
     \bigl{ (i,) + u \mid i \in{0,\cdots,n-1} \wedge u \in \mathcal{P}os(t_i) \bigr}
   $
Note that since we are programming in Python, positions are zero-based.  Given a position $v$ in a term $t$, we define $t/v$ as the subterm of $t$ at position $v$ by induction on $t$:
1. $t/() := t$,
2. $f(t_0,\cdots,t_{n-1})/u := t_{u\texttt{[0]}}/u\texttt{[1:]}$.
Given a term $s$, a term $t$, and a position $u \in \mathcal{P}os(t)$, we also define the replacement of the subterm at position $u$ by $t$, written $s[u \mapsto t]$ by induction on $u$:
1. $s\bigl[() \mapsto t\bigr] := t$.
2. $f(s_0,\cdots,s_{n-1})\bigl[\bigl((i,) + u\bigr) \mapsto t\bigr] := f\bigl(s_0,\cdots,s_i[u \mapsto t],\cdots,s_{n-1}\bigr)$.
Can you write Python code for it?

def non_triv_positions(t):
    if is_var(t):
        return set()
    _, *args = t
    Result = { () }
    for i, arg in enumerate(args):
        Result |= { (i,) + a for a in non_triv_positions(arg) }
    return Result

def test():
    t = parse_term('x * i(x) * 1')
    print(f'non_triv_positions({to_str(t)}) = {non_triv_positions(t)}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Back to top
Examples
In this section we present a number of examples where the Knuth-Bendix completion algorithm is able to produce a confluent system of equations.  In detail, we discuss the following examples:
1. Group Theory
2. Central Groupoids
3. Quasigroups
4. Quasigroups with Idempotence
5. Quasigroups with Unipotence
6. Loops
Group Theory
A structure $\mathcal{G} = \langle G, 1, *, i \rangle$ is a group iff
1. $G$ is a set.
2. $1 \in G$,
where $1$ is called the left-neutral element.
3. $*: G \times G \rightarrow G$,
where $$ is called the multiplication* of $\mathcal{G}$.
4. $i: G \rightarrow G$,
where for any $x \in G$ the element $i(x)$ is called the left-inverse of $x$.
5. The following equations hold for all $x,y,z \in G$:
   * $1 * x = x$, i.e. $1$ is a left-neutral element.
   * $i(x) * x = 1$, i.e. $i(x)$ is a left-inverse of $x$.
   * $(x * y) * z = x * (y * z)$, i.e. the multiplication is associative.
A typical example of a group is the set of invertible $n \times n$ matrices.
Given the axioms defining a group, the Knuth-Bendix completion algorithm is able to prove the following:
1. The left neutral element is also a right neutral element, we have:
   $$ x * 1 = x \quad \mbox{for all $x\in G$.} $$
2. The left inverse is also a right inverse, we have:
   $$ x * i(x) = 1 \quad \mbox{for all $x\in G$.} $$
3. The operations $i$ and $*$ commute as follows:
   $$ i(x * y) = i(y) * i(x) \quad \mbox{for all $x,y\in G$.}$$
Can you write Python code for it?

!cat Examples/group-theory-1.eqn || type Examples\group-theory-1.eqn

%%time
Rules = knuth_bendix_algorithm('Examples/group-theory-1.eqn')


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Back to top
Quasigroups
A structure $\mathcal{G} = \langle G, , /, \backslash \rangle$ is a quasigroup iff
1. $G$ is a non-empty set.
2. $: G \times G \rightarrow G$,
where $$ is called the multiplication* of $\mathcal{G}$.
3. $/: G \times G \rightarrow G$,
where $/$ is called the left division of $\mathcal{G}$.
4. $\backslash: G \times G \rightarrow G$,
where $\backslash$ is called the right division of $\mathcal{G}$.
5. The following equations hold for all $x,y \in G$:
   * $x * (x \backslash y) = y$,
   * $(x / y) * y = x$,
   * $x \backslash (x * y) = y$,
   * $(x * y) / y = x$.
Can you write Python code for it?

!cat Examples/quasigroups.eqn || type Examples\quasigroups.eqn

%%time
Rules = knuth_bendix_algorithm('Examples/quasigroups.eqn')


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Back to top
The Knuth-Bendix Ordering
In order to turn an equation $s = t$ into a rewrite rule, we have to check whether the term $s$ is more complex than the term $t$, so that $s$ should be simplified to $t$, or whether $t$ is more complex than $s$ and we should rewrite $t$ into $s$.  To this end, we implement the Knuth-Bendix ordering, which is a method to compare terms.
Given a term t and a variable name x, the function count(t, x) computes the number of times that x occurs in t.
Can you write Python code for it?

def count(t, x):
    match t:
        case '$var', y:
            return 1 if x == y else 0
        case _, *Ts:
            return sum(count(arg, x) for arg in Ts)

def test():
    t = parse_term('x * (i(x) * y)')
    print(f'count({to_str(t)}, "x") = {count(t, "x")}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Central Groupoids
A structure $\mathcal{G} = \langle G, \rangle$ is a central groupoid iff
1. $G$ is a a non-empty set.
2. $: G \times G \rightarrow G$,
3. The following equation holds for all $x,y,z \in G$:
   $$ (x * y) * (y * z) = y $$
Central Groupoids have been defined by Trevor Adams in his paper Products of Points—Some Simple Algebras and Their Identities and are also discussed by Donald E. Knuth in his paper notes on Central Groupoids.
Can you write Python code for it?

!cat Examples/central-groupoid.eqn || type Examples\central-groupoid.eqn

%%time
Rules = knuth_bendix_algorithm('Examples/central-groupoid.eqn')


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Given a file name that contains a set of equations and a dictionary encoding an ordering of the function symbols, the function knuth_bendix_algorithm implements the Knuth-Bendix algorithm:
1. The equations read from the file are oriented into rewrite rules.
2. These oriented equations are pushed onto the priority queue EquationQueue according to their complexity.
3. The set RewriteRules is initialized as the empty set.  The idea is that all critical pairs between
   equations in RewriteRules have already been computed and that the resulting new equations have been added
   to the priority queue EquationQueue.
4. As long as the priority queue EquationQueue is not empty, the least complex equation eq is removed from the
   priority queue and simplified using the known RewriteRules.
5. If the simplified version of eq is not trivial, all critical pairs between eq and the
   existing RewriteRules are computed.  The resulting equations are pushed onto the priority queue EquationQueue.
6. When no new critical pairs can be found, the set of RewriteRules is returned.
   This set is then guaranteed to be a confluent set of rewrite rules.
Can you write Python code for it?

def knuth_bendix_algorithm(file):
    Equations    = set()
    Axioms       = set(parse_file(file))
    RewriteRules = set()
    try:
        for eq in Axioms:
            ordered_eq = order_equation(eq)
            Equations.add(ordered_eq)
            print(f'given:   {to_str(ordered_eq)}')
        EquationQueue = []
        for eq in Equations:
            hq.heappush(EquationQueue, (complexity(eq), eq))
        while EquationQueue != []:
            _, eq = hq.heappop(EquationQueue)
            eq = normal_form(eq, RewriteRules)
            if eq[1] != eq[2]:
                lr = order_equation(eq)
                print(f'added:   {to_str(lr)}')
                Pairs = all_critical_pairs(RewriteRules | { lr }, lr)
                for eq, r1, r2 in Pairs:
                    new_eq = normal_form(eq, RewriteRules)
                    if new_eq[1] != new_eq[2]:
                        print(f'found:   {to_str(eq)} from {to_str(r1)}, {to_str(r2)}')
                        hq.heappush(EquationQueue, (complexity(new_eq), new_eq))
                RewriteRules = simplify_rules(RewriteRules, lr)
    except OrderException as e:
        print(e)
    print()
    print_equations(RewriteRules)
    return RewriteRules


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Given a set of syntactical equations $E$ and a substitution $\sigma$, the function $\texttt{solve}(E, \sigma)$ applies the rules of Martelli and Montanari to solve $E$.
Can you write Python code for it?

def solve(E, σ):
    while E != set():
        _, s, t = E.pop()
        if s == t: # remove trivial equations
            continue
        if is_var(s):
            _, x = s
            if occurs(x, t):
                return None
            else: # set x to t
                E = apply_set(E, { x: t })
                σ = compose(σ, { x: t })
        elif is_var(t):
            E.add(('≐', t, s))
        else:
            f    , g     = s[0]      , t[0]
            sArgs, tArgs = s[1:]     , t[1:]
            m    , n     = len(sArgs), len(tArgs)
            if f != g or m != n:
                return None
            else:
                E |= { ('≐', sArgs[i], tArgs[i]) for i in range(m) }
    return σ

def test():
    s = parse_term('x * i(x) * (y * z)')
    t = parse_term('a * i(1) * b')
    print(f'unify({to_str(s)}, {to_str(t)}) = {to_str(unify(s, t))}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Given a term p, a term t,  and a substitution σ, the function match_pattern(p, t, σ) tries to extend the
substitution σ so that the equation
$$ p \sigma = t $$
is satisfied.  If this is possible, the function returns True and updates the substitution σ so that
$p \sigma = t$ holds.  Otherwise, the function returns False.
Can you write Python code for it?

def match_pattern(pattern, term, σ):
    match pattern:
        case '$var', var:
            if var in σ:
                return σ[var] == term
            else:
                σ[var] = term           # extend σ
                return True
        case _:
            if pattern[0] == term[0] and len(pattern) == len(term):
                return all(match_pattern(pattern[i], term[i], σ) for i in range(1, len(pattern)))
            else:
                return False

def test():
    p = parse_term('i(x) * z')
    t = parse_term('i(i(y)) * i(y)')
    σ = {}
    match_pattern(p, t, σ)
    print(to_str(σ))

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Given a term s and a set of variables V, the function rename_variables(s, V) renames the variables in s so that they differ from the variables in the set V.  This will only work if the number of variables occurring in V times two is less than the number of letters in the latin alphabet, i.e. less than 26.  Therefore, the set V must have at most 13 variables.  For our examples, this is not a restriction.
Can you write Python code for it?

def rename_variables(s, Vars):
    assert len(Vars) <= 13, f'Error: too many variables in {Vars}.'
    NewVars = set(ascii_lowercase) - Vars
    NewVars = sorted(list(NewVars))
    σ       = { x: make_var(NewVars[i]) for (i, x) in enumerate(Vars) }
    return apply(s, σ)

def test():
    t = parse_equation('x * y * z = x * (y * z)')
    V = find_variables(t)
    print(f'rename_variables({to_str(t)}, {V}) = {to_str(rename_variables(t, V))}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Given a term t and a substitution σ that is represented as a dictionary of the form
$$ \sigma = { x_1: s_1, \cdots, x_n:s_n }, $$
the function apply(t, σ) computes the term that results from replacing the variables $x_i$ with the terms $s_i$ in t for all $i=1,\cdots,n$.  This term is written as $t\sigma$ and if $\sigma = { x_1: s_1, \cdots, x_n:s_n }$, then $t\sigma$ is defined by induction on t as follows:
- $x_i\sigma := s_i$,
- $v\sigma := v$ if $v$ is a variable and $v \not\in {x_1,\cdots,x_n}$,
- $f(t_1,\cdots,t_m)\sigma := f(t_1\sigma, \cdots, t_m\sigma)$.
Can you write Python code for it?

def apply(t, σ):
    "Apply the substitution σ to the term t."
    if is_var(t):
        _, var = t
        if var in σ:
            return σ[var]
        else:
            return t
    else:
        f, *Ts = t
        return (f,) + tuple(apply(s, σ) for s in Ts)

def test():
    p = parse_term('i(x) * x')
    t = parse_term('i(i(y)) * i(y)')
    σ = {}
    match_pattern(p, t, σ)
    print(f'apply({to_str(p)}, {to_str(σ)}) = {to_str(apply(p, σ))}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Given a term t the function weight(t) computes the weight $w(t)$, where $w(t)$ is defined by induction on $t$:
- $w(x) := 1$ for all variables $x$,
- $w\bigl(f(t_1,\cdots,t_n)\bigr) := \texttt{Weight}[f] + \sum\limits_{i=1}^n w(t_i)$.
Can you write Python code for it?

def weight(t):
    match t:
        case '$var', _:
            return 1
        case f, *Ts:
            return WEIGHT[f] + sum(weight(arg) for arg in Ts)

def test():
    t = parse_term('x * (i(x) * 1)')
    print(f'weight({to_str(t)}) = {weight(t)}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Given a term t, a position u in t and a term s, the function replace_at(t, u, s) replaces the subterm at position u with t.  The position u uses zero-based indexing.  Hence it returns the term
$$ t[u \mapsto s]. $$
Can you write Python code for it?

def replace_at(t, u, s):
    if len(u) == 0:
        return s
    i, *ur   = u
    f, *Args = t
    NewArgs  = []
    for j, arg in enumerate(Args):
        if j == i:
            NewArgs.append(replace_at(arg, ur, s))
        else:
            NewArgs.append(arg)
    return (f,) + tuple(NewArgs)

def test():
    t = parse_term('(x * i(x)) * 1')
    s = parse_term('a * b')
    print(f'replace_at({to_str(t)}, (0,1), {to_str(s)}) = {to_str(replace_at(t, (0,1), s))}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Given an equation eq and an Ordering of the function symbols occurring eq, the function order_equation orders the equation eq with respect to the Knuth-Bendix ordering, i.e. in the ordered equation, the right hand side is simpler than the left hand side.  If the left hand side and the right hand side are incomparable, the function raises an OrderException.
Can you write Python code for it?

def order_equation(eq):
    _, s, t = eq
    if is_simpler(t, s):
        return ('=', s, t)
    elif is_simpler(s, t):
        return ('=', t, s)
    else:
        Msg = f'Knuth-Bendix algorithm failed: Could not order {to_str(s)} = {to_str(t)}'
        raise OrderException(Msg)

def test():
    equation = 'i(i(a)) = i(i(i(i(a))))'
    eq = parse_equation(equation)
    print(f'order_equation({to_str(eq)}) = {to_str(order_equation(eq))}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Given an equation eq of the form eq = ('=', lhs, rhs), the function complexity(eq) computes a measure of complexity for the given equation.   This measure of complexity is the length of the string that represents the equation.  This measure of complexity is later used to choose between equations: Less complex equations are more interesting and should be considered first when computing critical pairs.
Can you write Python code for it?

def complexity(eq):
    return len(to_str(eq))


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Given two lists S and T of terms, the function is_simpler_list(S, T) checks whether S is lexicographically simpler than T if the elements of S and T are compared with the Knuth-Bendix ordering $\prec_{\textrm{kbo}}$.  It is assumed that S and T have the same length.
Can you write Python code for it?

def is_simpler_list(S, T):
    if S == [] == T:
        return False
    if is_simpler(S[0], T[0]):
        return True
    if S[0] == T[0]:
        return is_simpler_list(S[1:], T[1:])
    return False

def test():
    #l = parse_term('(x * y) * z')
    #r = parse_term('x * (y * z)')
    l = parse_term('i(a)')
    r = parse_term('i(i(a))')
    print(f'is_simpler({to_str(r)}, {to_str(l)}) = {is_simpler(r, l)}')
    print(f'is_simpler({to_str(l)}, {to_str(r)}) = {is_simpler(l, r)}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
If  $\sigma = \big{ x_1 \mapsto s_1, \cdots, x_m \mapsto s_m \big}$ and
$\tau = \big{ y_1 \mapsto t_1, \cdots, y_n \mapsto t_n \big}$
are two substitutions that are non-overlapping, i.e. such that ${x_1,\cdots, x_m} \cap {y_1,\cdots,y_n} = {}$ holds,
then we define the composition $\sigma\tau$ of $\sigma$ and $\tau$ as follows:
$$\sigma\tau := \big{ x_1 \mapsto s_1\tau, \cdots, x_m \mapsto s_m\tau,\; y_1 \mapsto t_1, \cdots, y_n \mapsto t_n \big}$$
This definition implies that the following associative law is valid:
$$ s(\sigma\tau) = (s\sigma)\tau $$
The function $\texttt{compose}(\sigma, \tau)$ takes two non-overlapping substitutions and computes their composition $\sigma\tau$.
Can you write Python code for it?

def compose(σ, τ):
    Result = { x: apply(s, τ) for (x, s) in σ.items() }
    Result.update(τ)
    return Result

def test():
    t1 = parse_term('i(y)')
    t2 = parse_term('a * b')
    t3 = parse_term('i(b)')
    σ = { 'x': t1 }
    τ = { 'y': t2, 'z': t3  }
    print(f'compose({to_str(σ)}, {to_str(τ)}) = {to_str(compose(σ, τ))}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Loops
A loop is a quasigroup that additionally has an identity element.  Therefore, a structure $\mathcal{G} = \langle G, 1, , /, \backslash \rangle$ is a loop iff
1. $G$ is a set.
2. $1 \in G$.
2. $: G \times G \rightarrow G$,
where $$ is called the multiplication* of $\mathcal{G}$.
3. $/: G \times G \rightarrow G$,
where $/$ is called the left division of $\mathcal{G}$.
4. $\backslash: G \times G \rightarrow G$,
where $\backslash$ is called the right division of $\mathcal{G}$.
5. The following equations hold for all $x,y \in G$:
   * $1 * x = x$,
   * $x * 1 = x$,
   * $x * (x \backslash y) = y$,
   * $(x / y) * y = x$,
   * $x \backslash (x * y) = y$,
   * $(x * y) / y = x$.
Can you write Python code for it?

!cat Examples/loops.eqn || type Examples\loops.eqn

%%time
Rules = knuth_bendix_algorithm('Examples/loops.eqn')


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Quasigroups with Idempotence
A quasigroup with idempotence is a quasigroup that additionally satisfies the identity $x * x = x$.  Therefore, a structure $\mathcal{G} = \langle G, , /, \backslash \rangle$ is a quasigroup with idempotence iff
1. $G$ is a set.
2. $: G \times G \rightarrow G$,
where $$ is called the multiplication* of $\mathcal{G}$.
3. $/: G \times G \rightarrow G$,
where $/$ is called the left division of $\mathcal{G}$.
4. $\backslash: G \times G \rightarrow G$,
where $\backslash$ is called the right division of $\mathcal{G}$.
5. The following equations hold for all $x,y \in G$:
   * $x * (x \backslash y) = y$,
   * $(x / y) * y = x$,
   * $x \backslash (x * y) = y$,
   * $(x * y) / y = x$,
   * $x * x = x$.
Can you write Python code for it?

!cat Examples/quasigroup-idempotence.eqn || type Examples\quasigroup-idempotence.eqn

%%time
Rules = knuth_bendix_algorithm('Examples/quasigroup-idempotence.eqn')


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
Quasigroups with Unipotence
A quasigroup with idempotence is a quasigroup that additionally satisfies the identity $x * x = 1$
where $1$ is a constant symbol. Therefore, a structure $\mathcal{G} = \langle G, 1, , /, \backslash \rangle$ is a quasigroup with idempotence iff
1. $G$ is a set.
2. $1 \in G$.
2. $: G \times G \rightarrow G$,
where $$ is called the multiplication* of $\mathcal{G}$.
3. $/: G \times G \rightarrow G$,
where $/$ is called the left division of $\mathcal{G}$.
4. $\backslash: G \times G \rightarrow G$,
where $\backslash$ is called the right division of $\mathcal{G}$.
5. The following equations hold for all $x,y \in G$:
   * $x * (x \backslash y) = y$,
   * $(x / y) * y = x$,
   * $x \backslash (x * y) = y$,
   * $(x * y) / y = x$,
   * $x * x = 1$.
Can you write Python code for it?

!cat Examples/quasigroup-unipotence.eqn || type Examples\quasigroup-unipotence.eqn

%%time
Rules = knuth_bendix_algorithm('Examples/quasigroup-unipotence.eqn')


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
RL Systems
Similarly, if we have a right neutral element and a left inverse the resulting structure need not be a group. Systems of this kind are called RL system.
Can you write Python code for it?

!cat Examples/rl-system.eqn || type Examples\rl-system.eqn

%%time
Rules = knuth_bendix_algorithm('Examples/rl-system.eqn')


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
The Knuth-Bendix order $s \prec_{\textrm{kbo}} t$ is defined for terms $s$ and $t$.  We have $s \prec_{\textrm{kbo}} t$ iff one of the following two conditions hold:
1. $w(s) < w(t)$ and $\texttt{count}(s, x) \leq \texttt{count}(t, x)$ for all variables $x$ occurring in $s$ .
2. $w(s) = w(t)$, $\texttt{count}(s, x) \leq \texttt{count}(t, x)$ for all variables $x$ occurring in $s$, and
   one of the following subconditions holds:
   * $t = f^n(s)$ where $n \geq 1$ and $f$ is the maximum w.r.t. the order $<$ on function symbols,
     i.e. we have $f = \texttt{max_fct}()$.
   * $s = f(s_1,\cdots,s_m)$, $t=g(t_1,\cdots,t_n)$, and $f<g$.
   * $s = f(s_1,\cdots,s_m)$, $t=f(t_1,\cdots,t_m)$, and
     $[s_1,\cdots,s_m] \prec_{\textrm{lex}} [t_1,\cdots,t_m]$.
 Here, $\prec_{\textrm{lex}}$ denotes the *lexicographic extension* of the ordering $\prec_{\textrm{kbo}}$ to
 lists of terms.  It is defined as follows:
 $$ [x] + R_1 \prec_{\textrm{lex}} [y] + R_2 \;\stackrel{_\textrm{def}}{\Longleftrightarrow}\;
  x \prec_{\textrm{kbo}} y \,\vee\, \bigl(x = y \wedge R_1 \prec_{\textrm{lex}} R_2\bigr)
 $$

Given two terms s and t the function is_simpler(s, t) returns True if $s \prec_{\textrm{kbo}} t$.
Can you write Python code for it?

def is_simpler(s, t):
    if is_var(t):
        return False
    if is_var(s):
        _, x = s
        return occurs(x, t)
    Vs = find_variables(s)
    for x in Vs:
        if count(t, x) < count(s, x):
            return False
    ws = weight(s)
    wt = weight(t)
    if ws < wt:
        return True
    if ws > wt:
        return False
    # ws == wt
    if is_tower(s, t):
        return True
    f, *Ss = s
    g, *Ts = t
    if ORDERING[f] < ORDERING[g]:
        return True
    if ORDERING[f] > ORDERING[g]:
        return False
    return is_simpler_list(Ss, Ts)


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
The function normal_form(t, E) takes a term t and a list (or set) of equations E and tries to simplify the term t as much as possible using the equations from E.
In the implementation, we have to be careful to rename the variables occurring in E so that they are different from the variables occurring in t.  Furthermore, we have to take care that we don't identify different variables in E by accident.  Therefore, we rename the variables in E so that they are both different from the variables in t and from the old variables occurring in E.
Can you write Python code for it?

def normal_form(t, E):
    Vars = find_variables(t) | find_variables(E)
    NewE = []
    for eq in E:
        NewE += [ rename_variables(eq, Vars) ]
    while True:
        s = simplify_step(t, NewE)
        if s == None:
            return t
        t = s

!cat Examples/group-theory-1.eqn || type Examples\group-theory-1.eqn

def test():
    E  = parse_file('Examples/group-theory-1.eqn')
    t  = parse_term('1 * (b * i(a)) * a')
    print(f'E = {to_str(E)}')
    print(f'normal_form({to_str(t)}, E) = {to_str(normal_form(t, E))}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
The function simplify_step(t, E) takes two arguments:
- t is a term,
- E is a set of equations of the form ('=', l, r).
The function tries to find an equation l = r in E and a subterm s in the term t such that the left hand side l of the equation matches the subterm s using some substitution $\sigma$, i.e. we have $s = l\sigma$.  Then the term t is simplified by replacing the subterm s in t by $r\sigma$.  More formally, if u is the position of s in t, i.e. t/u = s then t is simplified into the term
$$ t = t[u \mapsto l\sigma] \rightarrow_{{l=r}} t[u \mapsto r\sigma]. $$
If an appropriate subterm s is found, the simplified term is returned.  Otherwise, the function returns None.
If multiple subterms of t can simplified, then the function simplify_step(t, E) simplifies all subterms.
Can you write Python code for it?

def simplify_step(t, Equations):
    if is_var(t):
        return None      # variables can't be simplified
    for eq in Equations:
        _, lhs, rhs = eq
        σ = {}
        if match_pattern(lhs, t, σ):
            return apply(rhs, σ)
    f, *args   = t
    simpleArgs = []
    change     = False
    for arg in args:
        simple = simplify_step(arg, Equations)
        if simple != None:
            simpleArgs += [simple]
            change = True
        else:
            simpleArgs += [arg]
    if change:
        return (f,) + tuple(simpleArgs)
    return None

def test():
    E = { parse_equation('(x * y) * z = x * (y * z)') }
    t = parse_term('(a * b) * i(b)')
    print(f'simplify_step({to_str(t)}, {to_str(E)}) = {to_str(simplify_step(t, E))}')

test()


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm-KBO.ipynb".
The first task is:
We define the class OrderException to be able to deal with equations that can't be ordered into a rewrite rule.
Can you write Python code for it?

class OrderException(Exception):
    pass


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Back to top
Critical Pairs
The central notion of the Knuth-Bendix algorithm is the notion of a critical pair.
Given two equations lhs1 = rhs1 and lhs2 = rhs2, a pair of terms (s, t) is a critical pair of these equations if we have the following:
- u is a non-trivial position in lhs1, i.e. lhs1/u is not a variable,
- The subterm lhs1/u is unifiable with lhs2, i.e.
  $$\mu = \texttt{mgu}(\texttt{lhs}_1 / a, \texttt{lhs}_2) \not= \texttt{None},$$
- $s = \texttt{lhs}_1[a \leftarrow \texttt{rhs}_2]\mu$ and $t = \texttt{rhs}_1\mu$.
The function critical_pairs implemented in this section computes the critical pairs between two rewrite rules.
Given a term t, the function positions computes the set $\mathcal{P}os(t)$ of all positions in t that do not point to variables.  Such positions are called non-trivial positions.  Given a term t, the set $\mathcal{P}os(t)$ of all positions in $t$ is defined by induction on t.
1. $\mathcal{P}os(v) := \bigl{()\bigr} \quad \mbox{if $v$ is a variable} $
2. $\mathcal{P}os\bigl(f(t_0,\cdots,t_{n-1})\bigr) :=
     \bigl{()\bigr} \cup
     \bigl{ (i,) + u \mid i \in{0,\cdots,n-1} \wedge u \in \mathcal{P}os(t_i) \bigr}
   $
Note that since we are programming in Python, positions are zero-based.  Given a position $v$ in a term $t$, we define $t/v$ as the subterm of $t$ at position $v$ by induction on $t$:
1. $t/() := t$,
2. $f(t_0,\cdots,t_{n-1})/u := t_i/u\texttt{[1:]}$.
Can you write Python code for it?

def non_triv_positions(t):
    if is_var(t):
        return set()
    _, *args = t
    Result = { () }
    for i, arg in enumerate(args):
        Result |= { (i,) + a for a in non_triv_positions(arg) }
    return Result

t = parse_term('x * i(x) * 1')
f'non_triv_positions({to_str(t)}) = {non_triv_positions(t)}'


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Back to top
Matching
The function is_var(t) checks whether the term t is a variable. Variables are represented as nested tuples of the form ($var, name), where name is the name of the variable.
Can you write Python code for it?

def is_var(t):
    return t[0] == '$var'


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Back to top
Term Rewriting
Can you write Python code for it?

from string import ascii_lowercase
ascii_lowercase


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Back to top
The Completion Algorithm
Can you write Python code for it?

def print_equations(Equations):
    cnt = 1
    for _, l, r in Equations:
        print(f'{cnt}. {to_str(l)} = {to_str(r)}')
        cnt += 1


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given a file name that contains a set of equations and a dictionary encoding an ordering of the function symbols, the function knuth_bendix_algorithm performs the Knuth-Bendix algorithm:
1. The given equations are ordered.
2. The ordered equations are pushed into the priority queue EqtnQue according to their complexity.
3. The set RewriteRules is initialized as the empty set.
4. As long as the priority queue is not empty, the least complex equation lr is removed from the
   priority queue and simplified using the known RewriteRules.
5. If the simplified version of lr is non-trivial, all critical pairs between it and the
   RewriteRules are computed.  These critical pairs are pushed onto the priority queue.
6. When no new critical pairs can be found, the set of RewriteRules is returned.
   This set is then guaranteed to be a confluent set of rewrite rules.
Can you write Python code for it?

def knuth_bendix_algorithm(file, Ordering):
    Equations = set()
    Axioms    = set(parse_file(file))
    for _, s, t in Axioms:
        ordered_eq = order_equation(s, t, Ordering)
        Equations.add(ordered_eq)
        print(f'given:   {to_str(ordered_eq[1])} = {to_str(ordered_eq[2])}')
    EqtnQue = []
    for eq in Equations:
        hq.heappush(EqtnQue, (complexity(eq), eq) )
    RewriteRules = set()
    while EqtnQue != []:
        _, lr = hq.heappop(EqtnQue)
        l, r  = simplify_equation(lr, RewriteRules)
        if l != r:
            _, l, r = order_equation(l, r, Ordering)
            print(f'added:   {to_str(l)} → {to_str(r)}')
            NewEqs = all_critical_pairs(RewriteRules | { ('=', l, r) }, lr)
            for eq in NewEqs:
                s, t = simplify_equation(eq, RewriteRules)
                if s != t:
                    new_rule = order_equation(s, t, Ordering)
                    hq.heappush(EqtnQue, (complexity(new_rule), new_rule) )
            RewriteRules = simplify_rules(RewriteRules, ('=', l, r))
            RewriteRules.add( ('=', l, r) )
    return RewriteRules

%%time
RewriteRules = knuth_bendix_algorithm('Examples/group-theory-1.eqn', Ordering)
print()
print_equations(RewriteRules)


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given a new rewrite rule, the function simplify_rules(RewriteRules, rule) tries to simplify all rules in RewriteRules with rule.  If an equation eq from RewriteRules can be simplified with rule, it is further simplified with all rules in RewriteRules.
Can you write Python code for it?

def simplify_rules(RewriteRules, rule):
    SimpleEqs = set()
    for eq in RewriteRules:
        _, s, t = eq
        new_s = normal_form(s, { rule })
        if new_s != s:
            new_s = normal_form(new_s, RewriteRules | { rule })
        new_t = normal_form(t, { rule })
        if new_t != t:
            new_t = normal_form(new_t, RewriteRules | { rule })
        if new_s != new_t:
            simple = order_equation(new_s, new_t, Ordering)
            SimpleEqs.add(simple)
        else:
            print(f'removed: {to_str(s)} = {to_str(t)}')
    return SimpleEqs


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given a pair of term s and t and an Ordering of the function symbols occurring in s and t, the function order_equation orders the equation s = t with respect to the lexicographic path ordering, i.e. in the ordered equation, the right hand side is simpler than the left hand side.  If s and t are incomparable, the function raises an exception.
Can you write Python code for it?

def order_equation(s, t, Ordering):
    if is_simpler(t, s, Ordering):
        return ('=', s, t)
    elif is_simpler(s, t, Ordering):
        return ('=', t, s)
    else:
        assert False, f'Error: could not order {to_str(s)} = {to_str(t)}'


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given a set of syntactical equations $E$ and a substitution $\sigma$, the function $\texttt{solve}(E, \sigma)$ applies the rules of Martelli and Montanari to solve $E$.
Can you write Python code for it?

def solve(E, σ):
    while E != set():
        _, s, t = E.pop()
        if s == t: # remove trivial equations
            continue
        if is_var(s):
            _, x = s
            if occurs(x, t):
                return None
            else: # set x to t
                E = apply_set(E, { x: t })
                σ = compose(σ, { x: t })
        elif is_var(t):
            E.add(('≐', t, s))
        else:
            f    , g     = s[0]      , t[0]
            sArgs, tArgs = s[1:]     , t[1:]
            m    , n     = len(sArgs), len(tArgs)
            if f != g or m != n:
                return None
            else:
                E |= { ('≐', sArgs[i], tArgs[i]) for i in range(m) }
    return σ

s = parse_term('x * i(x) * (y * z)')
t = parse_term('a * i(1) * b')
f'unify({to_str(s)}, {to_str(t)}) = {to_str(unify(s, t))}'


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given a set of equations RewriteRules and a single rewrite rule eq, the function all_critical_pairs(RewriteRules, eq) computes the set of all non-trivial critical pairs that can be build by building critical pairs with an equation from RewriteRules and the equation eq.  It is assumed that eq is already an element of RewriteRules.
Can you write Python code for it?

def all_critical_pairs(RewriteRules, eq):
    Result = set()
    for eq1 in RewriteRules:
        Result |= { ('=', l, r) for l, r in critical_pairs(eq1, eq) }
        Result |= { ('=', l, r) for l, r in critical_pairs(eq, eq1) }
    return Result


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given a term s and a set of variables V, the function rename_variables(s, V) renames the variables in s so that they differ from the variables in the set V.  This will only work if the number of variables occurring in V times two is less than the number of letters in the latin alphabet, i.e. less than 26.  Therefore, the set V must have at most 13 variables.  For our examples, this is not a restriction.
Can you write Python code for it?

def rename_variables(s, Vars):
    assert len(Vars) <= 13, f'Error: too many variables in {Vars}.'
    NewVars = set(ascii_lowercase) - Vars
    NewVars = sorted(list(NewVars))
    σ       = { x: make_var(NewVars[i]) for (i, x) in enumerate(Vars) }
    return apply(s, σ)

Vars = { 'x', 'y', 'z' }
t = parse_term('x * y * z')
f'rename_variables({to_str(t)}, {Vars}) = {to_str(rename_variables(t, Vars))}'


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given a term t (or a set of terms), the function find_variables(t) computes the set of all variables occurring in t.
Can you write Python code for it?

def find_variables(t):
    if isinstance(t, set):
        return { var for term in t
                     for var  in find_variables(term)
               }
    if is_var(t):
        _, var = t
        return { var }
    _, *L = t
    return find_variables_list(L)


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given a term t and a substitution σ that is represented as a dictionary of the form
$$ \sigma = { x_1: s_1, \cdots, x_n:s_n }, $$
the function apply(t, σ) computes the term that results from replacing the variables $x_i$ with the terms $s_i$ in t for all $i=1,\cdots,n$.  This term is written as $t\sigma$ and if $\sigma = { x_1: s_1, \cdots, x_n:s_n }$, then $t\sigma$ is defined by induction on t as follows:
- $x_i\sigma := s_i$,
- $v\sigma := v$ if $v$ is a variable and $v \not\in {x_1,\cdots,x_n}$,
- $f(t_1,\cdots,t_n)\sigma := f(t_1\sigma, \cdots, t_n\sigma)$.
Can you write Python code for it?

def apply(t, σ):
    "Apply the substitution σ to the term t."
    if is_var(t):
        _, var = t
        if var in σ:
            return σ[var]
        else:
            return t
    else:
        f, *Ts = t
        return (f,) + tuple(apply(s, σ) for s in Ts)

p = parse_term('i(x) * x')
t = parse_term('i(i(y)) * i(y)')
σ = {}
match(p, t, σ)
to_str(apply(p, σ))


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given a variable name x, the function make_var(x) creates a variable with name x.
Can you write Python code for it?


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given an equation eq and a set of RewriteRules, the function simplify_equations simplifies the equation eq using the given rewrite rules. It returns a pair of terms.
Can you write Python code for it?

def simplify_equation(eq, RewriteRules):
    _, s, t = eq
    new_s = normal_form(s, RewriteRules)
    new_t = normal_form(t, RewriteRules)
    return new_s, new_t


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given two equations eq1 and eq2, the function critical_pairs(eq1, eq2) computes the set of all critical pairs between these equations.  A pair of terms (s, t) is a critical pair of eq1 and eq2 if we have
- eq1 has the form lhs1 = rhs1,
- eq2 has the form lhs2 = rhs2,
- a is a non-trivial position in lhs1,
- $\mu = \texttt{mgu}(\texttt{lhs}_1/a, \texttt{lhs}_2) \not= \texttt{None}$,
- $s = \texttt{lhs}_1[a \leftarrow \texttt{rhs}_2]\mu$ and $t = \texttt{rhs}_1\mu$.
Can you write Python code for it?

def critical_pairs(eq1, eq2):
    Vars = find_variables(eq1)
    eq2  = rename_variables(eq2, Vars)
    _, lhs1, rhs1 = eq1
    _, lhs2, rhs2 = eq2
    Result = set()
    Positions = non_triv_positions(lhs1)
    for u in Positions:
        if eq1 == eq2 and u == ():
            continue
        s = subterm(lhs1, u)
        𝜇 = unify(s, lhs2)
        if 𝜇 != None:
            lhs1_new = replace_at(lhs1, u, rhs2)
            lhs1_new = apply(lhs1_new, 𝜇)
            rhs1_new = apply(rhs1, 𝜇)
            Result.add( (lhs1_new, rhs1_new) )
    return Result

eq1 = parse_equation('i(x) * x = 1')
eq2 = parse_equation('(x * y) * z = x * (y * z)')
for s, t in critical_pairs(eq2, eq1):
    print(f'{to_str(s)} = {to_str(t)}')


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
Given two lists S and T of terms and a dictionary D, the function is_simpler_list(S, T, D) checks whether S is lexicographically simpler than T if the elements of S and T are compared with the lexicographical path ordering $\prec$.  It is assumed that S and T have the same length.
Can you write Python code for it?

def is_simpler_list(S, T, D):
    if S == [] == T:
        return False
    if is_simpler(S[0], T[0], D):
        return True
    if S[0] == T[0]:
        return is_simpler_list(S[1:], T[1:], D)
    return False

Ordering = { '1': 0, '*': 1, 'i': 2 }

l = parse_term('(x * y) * z')
r = parse_term('x * (y * z)')
f'is_simpler({to_str(r)}, {to_str(l)}, {Ordering}) = {is_simpler(r, l, Ordering)}'


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
The Knuth-Bendix Completion Algorithm
This notebook presents the Knuth-Bendix completion algorithm for transforming a set of equations into a confluent term rewriting system.  This notebook is divided into seven sections.
- Parsing
- Matching
- Term Rewriting
- Unification
- The Lexicographic Path Ordering
- Critical Pairs
- The Completion Algorithm
Parsing
To begin, we need a parser that is capable of parsing terms and equations. This parser is implemented in Parser.ipynb and can parse equations of terms and supports the binary operators +, -, *, /, %, and ^ (exponentiation) with the usual precedences.  Furthermore, function symbols are supported.  It also provides the function to_str for turning terms or equations into strings.  All together, it provides the following functions:
- parse_file(file_name) parses a file containing equations between terms.
- parse_equation(s) converts the string s into an equation.
- parse_term(s) converts the string s into a term.
- to_str(o) converts an object o into a string.  The object o either is
  * a term,
  * an equation,
  * a list of equations,
  * a set of equations, or
  * a dictionary representing a substitution.
Terms and equations are represented as nested tuples.  The parser is implemented using the parser generator Ply.
Can you write Python code for it?

%run Parser.ipynb

t = parse_term('x * y * z')
t

to_str(t)

eq = parse_equation('i(x) * x = 1')
eq

to_str(parse_file('Examples/group-theory-1.eqn'))


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
The algorithm implemented below takes a pair (E, σ) as its input.  Here E is a set of syntactical equations that need to be solved and σ is a substitution that is initially empty.  The pair (E, σ) is then transformed using the rules of Martelli and Montanari.  The transformation is successful if the pair (E, σ) can be transformed into a pair of the form ({}, μ).  Then μ is the solution to the system of equations E.
The rules that can be used to solve a system of syntactical equations are as follows:

 If $y\in\mathcal{V}$ is a variable that does not occur in the term $t$,
     then we perform the following reduction:
     $$ \Big\langle E \cup \big\{ y \doteq t \big\}, \sigma \Big\rangle \quad\leadsto \quad
         \Big\langle E[y \mapsto t], \sigma\big[ y \mapsto t \big] \Big\rangle
     $$

 If the variable $y$ occurs in the term $t$, then the system of syntactical equations
     $E \cup \big\{ y \doteq t \big\}$ is not solvable:
     $$ \Big\langle E \cup \big\{ y \doteq t \big\}, \sigma \Big\rangle\;\leadsto\; \texttt{None} \quad
        \mbox{if $x \in \textrm{Var}(t)$ and $y \not=t$.}$$

 If $y\in\mathcal{V}$ is a variable and $t$ is no variable, then we use the following rule:
     $$ \Big\langle E \cup \big\{ t \doteq y \big\}, \sigma \Big\rangle \quad\leadsto \quad
         \Big\langle E \cup \big\{ y \doteq t \big\}, \sigma \Big\rangle.
     $$

 Trivial syntactical equations of variables can be dropped:
     $$ \Big\langle E \cup \big\{ x \doteq x \big\}, \sigma \Big\rangle \quad\leadsto \quad
         \Big\langle E, \sigma \Big\rangle.
     $$

 If $f$ is an $n$-ary function symbol, then we have:
     $$ \Big\langle E \cup \big\{ f(s_1,\cdots,s_n) \doteq f(t_1,\cdots,t_n) \big\}, \sigma \Big\rangle
         \;\leadsto\;
         \Big\langle E \cup \big\{ s_1 \doteq t_1, \cdots, s_n \doteq t_n\}, \sigma \Big\rangle.
     $$

 The system of syntactical equations $E \cup \big\{ f(s_1,\cdots,s_m)...
Can you write Python code for it?

def unify(s, t):
    return solve({('≐', s, t)}, {})


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
The function normal_form(t, E) takes a term t and a list (or set) of equations E and tries to simplify the term t as much as possible using the equations from E.
In the implementation, we have to be careful to rename the variables occurring in E so that they are different from the variables occurring in t.  Furthermore, we have to take care that we don't identify different variables in E by accident.  Therefore, we rename the variables in E so that they are both different from the variables in t and from the old variables occurring in E.
Can you write Python code for it?

def normal_form(t, E):
    Vars = find_variables(t) | find_variables(E)
    NewE = []
    for eq in E:
        NewE += [ rename_variables(eq, Vars) ]
    while True:
        s = simplify_step(t, NewE)
        if s == None:
            return t
        t = s

l  = parse_term('i(b * a)')
eq = parse_equation('i(a * c) = i(c) * i(a)')
f'normal_form({to_str(l)}, {to_str(eq)}) = {to_str(normal_form(l, {eq}))}'

E = parse_file('Examples/group-theory-1.eqn')
t = parse_term('(x * i(y)) * y * z')
print(f'normal_form({to_str(t)}, {to_str(E)}) = \n{to_str(normal_form(t, E))}')


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
The function simplify_step(t, E) takes two arguments:
- t is a term,
- E is a set of equations of the form ('=', l, r).
The function tries to an equation l = r in E and a subterm s in the term t such that the left hand side l of the equation matches the subterm s using some substitution $\sigma$, i.e. we have $s = l\sigma$.  Then the term t is simplified by replacing the subterm s in t by $r\sigma$.  More formally, if u is the position of s in t, i.e. t/u = s then t is simplified into the term
$$ t[u \mapsto r\sigma] $$
If an appropriate subterm s is found, the simplified term is returned.  Otherwise, the function returns None.
Can you write Python code for it?

def simplify_step(t, Equations):
    if is_var(t):
        return None
    for eq in Equations:
        _, lhs, rhs = eq
        σ = {}
        if match(lhs, t, σ):
            return apply(rhs, σ)
    f, *args   = t
    simpleArgs = []
    change     = False
    for arg in args:
        simple = simplify_step(arg, Equations)
        if simple != None:
            simpleArgs += [simple]
            change = True
        else:
            simpleArgs += [arg]
    if change:
        return (f,) + tuple(simpleArgs)
    return None

E = { parse_equation('(x * y) * z = x * (y * z)') }
t = parse_term('(a * b) * i(b)')
f'simplify_step({to_str(t)}, {to_str(E)}) = {to_str(simplify_step(t, E))}'


I am working on the file "Python/4 Automatic Theorem Proving/Knuth-Bendix-Algorithm.ipynb".
The first task is:
The module heapq provides heap-based priority queues, which are implemented as lists.
Can you write Python code for it?


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
A Simple Parser for Term Rewriting
This file implements a parser for terms and equations.  It uses the parser generator Ply.  To install Ply, change the cell below into a code cell and execute it.  If the package ply is already installed, this command will only produce a message that the package is already installed.
!conda install -y -c anaconda ply
Specification of the Scanner
The scanner that is implemented below recognizes numbers, variable names, function names, and various operator symbols.  Variable names have to start with a lower case letter, while function names start with an uppercase letter.
Can you write Python code for it?

import ply.lex as lex

tokens = [ 'NUMBER', 'VAR', 'FCT', 'BACKSLASH' ]


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
Setting the optional argument write_tables to False is required to prevent an obscure bug where the parser generator tries to read an empty parse table.  As we have used precedence declarations to resolve all shift/reduce conflicts, the action table should contain no conflict.
Can you write Python code for it?

parser = yacc.yacc(write_tables=False, debug=True)


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
Single line comments are supported and work as in C.
Can you write Python code for it?

def t_COMMENT(t):
    r'//[^\n]*'
    t.lexer.lineno += t.value.count('\n')
    pass


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
Syntactically, newline characters are ignored. However, we still need to keep track of them in order to know which line we are in.  This information is needed later for error messages.
Can you write Python code for it?

def t_newline(t):
    r'\n'
    t.lexer.lineno += 1
    return


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
The arithmetic operators and a few other symbols are supported.
Can you write Python code for it?


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
The function parse takes a file_name as its sole argument.  The file is read and parsed.
The resulting parse tree is visualized using graphviz.  It is important to reset the
attribute lineno of the scanner, for otherwise error messages will not have the correct line numbers.
Can you write Python code for it?

def test_parse(file_name):
    lexer.lineno = 1
    with open(file_name, 'r') as handle:
        program = handle.read()
    ast = yacc.parse(program)
    print(ast)
    return tuple2dot(ast)


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
The function t_error is called for any token t that can not be scanned by the lexer.  In this case, t.value[0] is the first character that can not be recognized by the scanner.
Can you write Python code for it?

def t_error(t):
    column = find_column(t)
    print(f"Illegal character '{t.value[0]}' in line {t.lineno}, column {column}.")
    t.lexer.skip(1)


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
The next assignment is necessary to make the lexer think that the code given above is part of some file.
Can you write Python code for it?

__file__ = 'main'

lexer = lex.lex()

def test_scanner(file_name):
    with open(file_name, 'r') as handle:
        program = handle.read()
    print(program)
    lexer.input(program)
    lexer.lineno = 1
    return [t for t in lexer]


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
The start variable of our grammar is axioms.
Can you write Python code for it?

start = 'axioms'

precedence = (
    ('nonassoc', '='),
    ('left', '+', '-'),
    ('left', '*', '/', 'BACKSLASH', '%'),
    ('right', '^')
)

def p_axioms_one(p):
    "axioms : equation"
    p[0] = ('axioms', p[1])

def p_axioms_more(p):
    "axioms : axioms equation"
    p[0] = p[1] + (p[2],)

def p_equation(p):
    "equation : term '=' term ';'"
    p[0] = ('=', p[1], p[3])

def p_term_plus(p):
    "term : term '+' term"
    p[0] = ('+', p[1], p[3])

def p_term_minus(p):
    "term : term '-' term"
    p[0] = ('-', p[1], p[3])

def p_term_times(p):
    "term : term '*' term"
    p[0] = ('*', p[1], p[3])

def p_term_divide(p):
    "term : term '/' term"
    p[0] = ('/', p[1], p[3])

def p_term_backslash(p):
    "term : term BACKSLASH term"
    p[0] = ('\\', p[1], p[3])

def p_term_modulo(p):
    "term : term '%' term"
    p[0] = ('%', p[1], p[3])

def p_term_power(p):
    "term : term '^' term"
    p[0] = ('^', p[1], p[3])

def p_term_group(p):
    "term : '(' term ')'"
    p[0] = p[2]

def p_term_fct_call(p):
    "term : FCT '(' term_list ')'"
    p[0] = (p[1],) + p[3][1:]

def p_term_number(p):
    "term : NUMBER"
    p[0] = (p[1],)

def p_term_id(p):
    "term : VAR"
    p[0] = ('$var', p[1])

def p_term_list_empty(p):
    "term_list :"
    p[0] = ('.',)

def p_term_list_one(p):
    "term_list : term"
    p[0] = ('.', p[1])

def p_term_list_more(p):
    "term_list : term ',' ne_term_list"
    p[0] = ('.', p[1]) + p[3][1:]

def p_ne_term_list_one(p):
    "ne_term_list : term"
    p[0] = ('.', p[1])

def p_ne_term_list_more(p):
    "ne_term_list : term ',' ne_term_list"
    p[0] = ('.', p[1]) + p[3][1:]

def p_error(p):
    if p:
        column = find_column(p)
        print(f'Syntax error at token "{p.value}" in line {p.lineno}, column {column}.')
    else:
        print('Syntax error at end of input.')


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
The token Number specifies a natural number.  Syntactically, numbers are treated a function symbols.
Can you write Python code for it?

def t_NUMBER(t):
    r'0|[1-9][0-9]*'
    return t


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
Variables start with a letter, followed by letters, digits, and underscores. They must be followed by a character that is not an opening parenthesis (.
Can you write Python code for it?


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
White space, i.e. space characters, tabulators, and carriage returns are ignored.
Can you write Python code for it?


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
parse_equation('i(x) * x = 1')
Can you write Python code for it?

def parse_term(s):
    lexer.lineno = 1
    AST = yacc.parse(s + '= 1;')
    if AST:
        _, *L = AST
        return L[0][1]
    return None


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
parse_file('Examples/group-theory.eqn')
Can you write Python code for it?

def parse_equation(s):
    lexer.lineno = 1
    AST = yacc.parse(s + ';')
    if AST:
        _, *L = AST
        return L[0]
    return None


I am working on the file "Python/4 Automatic Theorem Proving/Parser.ipynb".
The first task is:
parse_term('i(x) * x')
Can you write Python code for it?

def to_str(t):
    if isinstance(t, set):
        return '{' + ', '.join({ f'{to_str(eq)}' for eq in t }) + '}'
    if isinstance(t, list):
        return '[' + ', '.join([ f'{to_str(eq)}' for eq in t ]) + ']'
    if isinstance(t, dict):
        return '{' + ', '.join({ f'{k}: {to_str(v)}' for k, v in t.items() }) + '}'
    if isinstance(t, str):
        return t
    if t[0] == '$var':
        return t[1]
    if len(t) == 3 and t[0] in ['=']:
        _, lhs, rhs = t
        return f'{to_str(lhs)} = {to_str(rhs)}'
    if t[0] == '\\':
        op, lhs, rhs = t
        return to_str_paren(lhs) + ' \\ ' + to_str_paren(rhs)
    if len(t) == 3 and t[0] in ['+', '-', '*', '/', '%', '^']:
        op, lhs, rhs = t
        return f'{to_str_paren(lhs)} {op} {to_str_paren(rhs)}'
    f, *Args = t
    if Args == []:
        return f
    return f'{f}({to_str_list(Args)})'

def to_str_paren(t):
    if isinstance(t, str):
        return t
    if t[0] == '$var':
        return t[1]
    if len(t) == 3:
        op, lhs, rhs = t
        return f'({to_str_paren(lhs)} {op} {to_str_paren(rhs)})'
    f, *Args = t
    if Args == []:
        return f
    return f'{f}({to_str_list(Args)})'

def to_str_list(TL):
    if TL == []:
        return ''
    t, *Ts = TL
    if Ts == []:
        return f'{to_str(t)}'
    return f'{to_str(t)}, {to_str_list(Ts)}'


I am working on the file "Python/5 Linear Regression/Corona.ipynb".
The first task is:
It seems that our model is a good approximation of the data.
Finally, let us make a prediction for the 16th of March. The logarithm of the number of cases $n$ on the 16th of March which is $20$ days after the 25th of February is predicted according to the formula:
$$ \ln(n) = \vartheta_1 \cdot 20 + \vartheta_0 $$
Hence, the number of cases $n$ for that day is predicted as
$$ n = \exp\bigl(\vartheta_1 \cdot 20 + \vartheta_0\bigr). $$
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Corona.ipynb".
The first task is:
Let us plot the line $y(x) = ϑ0 + ϑ1 \cdot x$ together with our data:
Can you write Python code for it?

xMax = max(X) + 0.2
plt.figure(figsize=(12, 10))
sns.set(style='darkgrid')
plt.xticks(ticks=X)
plt.scatter(X, Y, c='r')
plt.plot([0, xMax], [ϑ0, ϑ0 + ϑ1 * xMax], c='b')
plt.xlabel('date in number of days since the 1st of March')
plt.ylabel('log of the number of Covid-19 cases')
plt.title('Log of the Number of Covid-19 cases versus date.')


I am working on the file "Python/5 Linear Regression/Corona.ipynb".
The first task is:
Next, we compute the residual sum of squares RSS as follows:
$$ \mathtt{RSS} := \sum\limits_{i=1}^m \bigl(\vartheta_1 \cdot x_i + \vartheta_0 - y_i\bigr)^2 $$
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Corona.ipynb".
The first task is:
Now $R^2$ is calculated via the formula:
$$ R^2 = 1 - \frac{\mathtt{RSS}}{\mathtt{TSS}}$$
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Corona.ipynb".
The first task is:
The blue line is not too far of the data points.  In order to judge the quality of our model we compute both the
total sum of squares and the residual sum of squares.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Corona.ipynb".
The first task is:
The coefficient $\vartheta_0$ is computed according to the formula:
$$ \vartheta_0 = \bar{\mathbf{y}} - \vartheta_1 \cdot \bar{\mathbf{x}} $$
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Corona.ipynb".
The first task is:
The coefficient $\vartheta_1$ is computed according to the formula:
$$ \vartheta_1 = \frac{\sum\limits_{i=1}^m \bigl(x_i - \bar{\mathbf{x}}\bigr) \cdot \bigl(y_i - \bar{\mathbf{y}}\bigr)}{
                       \sum\limits_{i=1}^m \bigl(x_i - \bar{\mathbf{x}}\bigr)^2}
$$
Can you write Python code for it?

ϑ1 = np.sum((X - xMean) * (Y - yMean)) / np.sum((X - xMean) ** 2)
ϑ1


I am working on the file "Python/5 Linear Regression/Corona.ipynb".
The first task is:
This does not look exactly linear.  Rather, it might be exponential.  Hence, we take the logarithm of the number of cases and plot these logarithms with respect to the date.
Can you write Python code for it?

import math

Y = [math.log(y) for y in num_cases]

plt.figure(figsize=(12, 10))
sns.set(style='whitegrid')
plt.xticks(ticks=X)
plt.scatter(X, Y, c='r')
plt.xlabel('date in number of days since the 25th of Febuary')
plt.ylabel('log of the number of Covid-19 cases')
plt.title('Log of the Number of Covid-19 cases versus date.')


I am working on the file "Python/5 Linear Regression/Corona.ipynb".
The first task is:
We compute the average number of the logarithm of Covid-19 cases according to the formula:
$$ \bar{\mathbf{y}} = \frac{1}{m} \cdot \sum\limits_{i=1}^m y_i $$
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Corona.ipynb".
The first task is:
We take the number of days after the 25th of February as x-values.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Linear-Regression-Rounding.ipynb".
The first task is:
Again, we fit a linear model.
Can you write Python code for it?

model = lm.LinearRegression()
M = model.fit(XS, YS)
M.score(XS, YS)


I am working on the file "Python/5 Linear Regression/Linear-Regression-Rounding.ipynb".
The first task is:
As we want to study the effect of rounding, the values of the dependent variable X are rounded to the nearest integer.  To this end, the values are transformed to another unit, rounded and then transformed back to the original unkit.  This way we can investigate how the performance of linear regression  degrades if the precision of the measurements of the independent variable is low.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Linear-Regression-Rounding.ipynb".
The first task is:
Linear Regression: Rounding and Subclassing
In this notebook we investigate the influence of rounding and subclassing on linear regression.  To begin, we import all the libraries we need.
Can you write Python code for it?

import numpy                as np
import matplotlib.pyplot    as plt
import seaborn              as sns
import sklearn.linear_model as lm


I am working on the file "Python/5 Linear Regression/Linear-Regression-Rounding.ipynb".
The first task is:
Next, we investigate the effect of subclassing. We will only keep those values such that $X > 11$.
Can you write Python code for it?

X.shape

selectorX = (X > 11)
selectorY = np.reshape(selectorX, (N,))
XS = X[selectorX]
XS = np.reshape(XS, (len(XS), 1))
YS = Y[selectorY]


I am working on the file "Python/5 Linear Regression/Linear-Regression-Rounding.ipynb".
The first task is:
We build a linear model for X and Y.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Linear-Regression-Rounding.ipynb".
The first task is:
We can see that the performance of the linear model has degraded considerably.
Can you write Python code for it?

ϑ0 = M.intercept_
ϑ1 = M.coef_[0]
xMax = max(X) + 0.2
xMin = min(X) - 0.2
plt.figure(figsize=(12, 10))
sns.set(style='darkgrid')
plt.scatter(X, Y, c='b')
plt.plot([xMin, xMax], [ϑ0 + ϑ1 * xMin, ϑ0 + ϑ1 * xMax], c='r')
plt.xlabel('rounded X values')
plt.ylabel('true X values + noise')
plt.title('Influence of rounding on explained variance')
plt.show()


I am working on the file "Python/5 Linear Regression/Linear-Regression-Rounding.ipynb".
The first task is:
We create a new linear model, fit it to the data and compute its score.
Can you write Python code for it?

model = lm.LinearRegression()
M = model.fit(X, Y)
M.score(X, Y)


I am working on the file "Python/5 Linear Regression/Linear-Regression-Rounding.ipynb".
The first task is:
We plot Y versus X and the linear regression line.
Can you write Python code for it?

xMax = np.max(X) + 0.2
xMin = np.min(X) - 0.2
%matplotlib inline
plt.figure(figsize=(15, 10))
sns.set(style='darkgrid')
plt.scatter(X, Y, c='b') # 'b' is blue color
plt.xlabel('X values')
plt.ylabel('true values + noise')
plt.title('Influence of rounding on explained variance')
plt.show(plt.plot([xMin, xMax], [ϑ0 + ϑ1 * xMin, ϑ0 + ϑ1 * xMax], c='r'))


I am working on the file "Python/5 Linear Regression/Linear-Regression-Rounding.ipynb".
The first task is:
We see that the performance of linear regression has degraded considerably.  Let's plot this.
Can you write Python code for it?

ϑ0 = M.intercept_
ϑ1 = M.coef_[0]
xMax = max(XS) + 0.2
xMin = min(XS) - 0.2
plt.figure(figsize=(12, 10))
sns.set(style='darkgrid')
plt.scatter(XS, YS, c='b')
plt.plot([xMin, xMax], [ϑ0 + ϑ1 * xMin, ϑ0 + ϑ1 * xMax], c='r')
plt.xlabel('rounded X values')
plt.ylabel('true X values + noise')
plt.title('Influence of subclassing on explained variance')
plt.show()


I am working on the file "Python/5 Linear Regression/Linear-Regression-Rounding.ipynb".
The first task is:
We train the model and compute its score.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Linear-Regression.ipynb".
The first task is:
Below we read the file cars.csv and store the fuel consumption in the list mpg, while the number of cylinders, the engine displacement,
the power, the weight, the acceleration, and the building year are stored as the list of lists Features. Note also that we have added the constant feature $1$ to every list in Features.
Can you write Python code for it?

with open('cars.csv') as input_file:
    reader   = csv.DictReader(input_file, delimiter=',')
    Features = ['cyl', 'displacement', 'hp' ,'weight', 'acc', 'year']
    X        = []
    Y        = []
    for row in reader:
        Y.append(float(row['mpg']))
        X.append([float(row[f]) for f in Features] + [1.0])

X


I am working on the file "Python/5 Linear Regression/Linear-Regression.ipynb".
The first task is:
General Linear Regression
In this case study we investigate how much of the fuel consumption of a car can be explained by the

number of cylinders,
engine displacement,
horsepower,
weight,
acceleration, and
the year the car has been introduced into the market.

This data is given in the CSV file cars.csv.  In this file, the engine displacement is given in cubic inches and the weight is given in pounds.  The fuel consumption is specified as miles per gallon and the acceleration is given as the number of seconds until the car reaches 60 miles per hour.
The module csv offers a number of functions for reading and writing <tt>csv</tt> files.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Linear-Regression.ipynb".
The first task is:
Note that every row in this matrix contains the data corresponding to a single car.
Since miles per gallon is the inverse of the fuel consumption, the vector Y is defined as the reciprocal of the mpg values that are currently stored in Y.
Can you write Python code for it?

Y = np.array([1 / Y[i] for i in range(m)])


I am working on the file "Python/5 Linear Regression/Linear-Regression.ipynb".
The first task is:
Now the proportion of explained variance $R^2$ is calculated via the formula:
$$ R^2 = 1 - \frac{\mathtt{RSS}}{\mathtt{TSS}}$$
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Linear-Regression.ipynb".
The first task is:
The residual sum of squares is given by the following sum:
$$ \texttt{RSS} = \sum\limits_{i=1}^m \Bigl(\bigl(\textbf{x}^{(i)}\bigr)^\top \cdot \textbf{w} - y_i\Bigr)^2 $$
Here $\textbf{x}^{(i)}$ is the $i$-th row of the matrix $X$, while $y_i$ is the $i$-th component of the vector $\textbf{y}$.
The expression $\bigl(\textbf{x}^{(i)}\bigr)^\top \cdot \textbf{w}$ is the predicted value of the linear model, while $y_i$ is the actual value.
As the feature Matrix $X$ is defined as
$$ X = \left( \begin{array}{c}
              \bigl(\textbf{x}^{(1)}\bigr)^\top \
              \vdots \
              \bigl(\textbf{x}^{(m)}\bigr)^\top
              \end{array}
       \right)
$$
we can compute the variable RSS as follows:
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Linear-Regression.ipynb".
The first task is:
The number of data pairs of the form $\langle \textbf{x}, y \rangle$ that we have read is stored in the variable m.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Linear-Regression.ipynb".
The first task is:
We compute the average fuel consumption $\bar{\mathbf{y}}$ according to the formula:
$$ \bar{\mathbf{y}} = \frac{1}{m} \cdot \sum\limits_{i=1}^m y_i $$
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
In order to improve the explained variance of our model, we extend it with polynomial features, i.e. we add features like $x_1^2$ and $x_1\cdot x_2$ etc.
Can you write Python code for it?

from sklearn.preprocessing import PolynomialFeatures

quadratic = PolynomialFeatures(2, include_bias=False)

X_train_quadratic = quadratic.fit_transform(X_train)
X_test_quadratic  = quadratic.fit_transform(X_test)
quadratic.get_feature_names(['x1', 'x2'])

X_test_quadratic


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
In this notebook we want to discuss both polynomial regression and overfitting.
One possible reason causing overfitting is a correlation between features.  Let us create a dataset with two feature x1 and x2 that are more or less the same. Actually, x2 is x1 plus some random noise.  The dependent variable y is the square root of x1.
Can you write Python code for it?

np.random.seed(42)
N  = 20                     # number of data points
X1 = np.array([k for k in range(N)])
X2 = np.array([k + 0.2 * (np.random.rand() - 0.5) for k in range(N)])
Y  = np.sqrt(X1)            # Y is the square root of X1
X1 = np.reshape(X1, (N, 1)) # turn X1 into an N x 1 matrix
X2 = np.reshape(X2, (N, 1)) # turn X2 into an N x 1 matrix
X  = np.hstack([X1, X2])    # combine X1 and X2 into an N x 2 matrix
X


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
It turns out that we can get $100\%$ of explained variance, but only for the training set.  The explained variance of the test set has decreased and apparently the curve is starting to get wiggly.
Ridge Regression
Can you write Python code for it?

def ridge_regression(X_train, Y_train, X_test, Y_test, alpha):
    M = lm.Ridge(alpha, solver='svd')
    M.fit(X_train, Y_train)
    train_score = M.score(X_train, Y_train)
    test_score  = M.score(X_test , Y_test)
    return M, train_score, test_score

def plot_nth_degree_polynomial_ridge(n, alpha):
    poly         = PolynomialFeatures(n, include_bias=False)
    X_train_poly = poly.fit_transform(X_train)
    X_test_poly  = poly.fit_transform(X_test)
    M, s1, s2    = ridge_regression(X_train_poly, Y_train, X_test_poly, Y_test, alpha)
    print('The explained variance on the training set is:', s1)
    print('The explained variance on the test     set is:', s2)
    a, b = polynomial_vector(n, M)
    plt.figure(figsize=(15, 10))
    sns.set(style='darkgrid')
    plt.title('A Regression Problem')
    plt.axvline(x=0.0, c='k')
    plt.axhline(y=0.0, c='k')
    plt.xlabel('x axis')
    plt.ylabel('y axis')
    plt.xticks(np.arange(0.0, N + 1, step=1.0))
    plt.yticks(np.arange(0.0, 2*np.sqrt(N), step=0.25))
    plt.scatter(X_train[:,0], Y_train, color='b')
    plt.scatter(X_test [:,0], Y_test , color='g')
    plt.plot(a, b, c='r')
    #plt.savefig('sqrt-' + str(n) + 'ridge.pdf')


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
Let us plot the data.  We will use colors to distinguish between x1 and x2.  The pairs $(x_1, y)$ are plotted in blue, while the pairs  $(x_2, y)$ are plotted in yellow.
Can you write Python code for it?

import matplotlib.pyplot as plt
import seaborn           as sns

plt.figure(figsize=(15, 12))
sns.set(style='whitegrid')
plt.title('A Regression Problem')
plt.axvline(x=0.0, c='k')
plt.axhline(y=0.0, c='k')
plt.xlabel('x axis')
plt.ylabel('y axis')
plt.xticks(np.arange(0.0, N, step=1.0))
plt.yticks(np.arange(0.0, np.sqrt(N) + 1, step=0.25))
plt.scatter(X1, Y, color='b')
plt.scatter(X2, Y, color='y')


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
Let's try to use a polynomial of degree 6 but without regularization.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
Now the model is much smoother and the explained variance has also increased considerably on the test set.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
Polynomial Regression and Overfitting
Can you write Python code for it?

import numpy                as np
import sklearn.linear_model as lm


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
The accuracy on the training set and on the test set have both increased.  Let us plot the model.
Can you write Python code for it?

ϑ0                 = M.intercept_
ϑ1, ϑ2, ϑ3, ϑ4, ϑ5 = M.coef_


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
The function $\texttt{plot_nth_degree_boundary}(n)$ creates a polynomial regression model of degree $n$.  It plots both the data and the polynomial model.
Can you write Python code for it?

def plot_nth_degree_polynomial(n):
    poly         = PolynomialFeatures(n, include_bias=False)
    X_train_poly = poly.fit_transform(X_train)
    X_test_poly  = poly.fit_transform(X_test)
    M, s1, s2    = linear_regression(X_train_poly, Y_train, X_test_poly, Y_test)
    print('The explained variance on the training set is:', s1)
    print('The explained variance on the test     set is:', s2)
    a, b = polynomial_vector(n, M)
    plt.figure(figsize=(15, 10))
    sns.set(style='darkgrid')
    plt.title('A Regression Problem')
    plt.axvline(x=0.0, c='k')
    plt.axhline(y=0.0, c='k')
    plt.xlabel('x axis')
    plt.ylabel('y axis')
    plt.xticks(np.arange(0.0, N + 1, step=1.0))
    plt.yticks(np.arange(0.0, 2*np.sqrt(N), step=0.25))
    plt.scatter(X_train[:,0], Y_train, color='b')
    plt.scatter(X_test [:,0], Y_test , color='g')
    plt.plot(a, b, c='r')
    #plt.savefig('sqrt-' + str(n) + '.pdf')


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
This looks like the model that we had found before.  Let us try to add a bit of regularization.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
This seems to be working and the explained variance has improved. Let's try to use even higher order polynomials.  Hopefully, we can get a $100\%$ explained variance.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
We use the function linear_regression to build a model for our data.  Note that this function uses only the training data to create the model.  The test data is only used for evaluating the model.
Can you write Python code for it?

M, train_score, test_score = linear_regression(X_train, Y_train, X_test, Y_test)
train_score, test_score


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
We want to split the data into a training set and a test set.
The training set will be used to compute the parameters of our model, while the
testing set is only used to check the accuracy.  SciKit-Learn has a predefined method
sklearn.model_selection.train_test_split that can be used to randomly split data into a training set and a test set.
Can you write Python code for it?

from sklearn.model_selection import train_test_split


I am working on the file "Python/5 Linear Regression/Polynomial-Regression.ipynb".
The first task is:
We will split the data at a ratio of $4:1$, i.e. $80\%$ of the data will be used for training, while the remaining $20\%$ is used to test the accuracy.
Can you write Python code for it?

X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=1)
X_test


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-Function.ipynb".
The first task is:
It seems that about $75\%$ of the fuel consumption is explained by the engine displacement.  We can get a better model of the fuel consumption if we use more variables for explaining the fuel consumption.  For example, the weight of a car is also responsible for its fuel consumption.
Can you write Python code for it?

def coefficient_of_determination(name, Data):
    X  = np.array([float(line[name])    for line in Data])
    Y  = np.array([1/float(line['mpg']) for line in Data])
    R2 = simple_linear_regression(X, Y)
    print(f'coefficient of determination of fuel consumption w.r.t. {name:12s}: {round(R2, 2)}')

DependentVars = ['cyl', 'displacement', 'hp', 'weight', 'acc', 'year']

for name in DependentVars:
    coefficient_of_determination(name, Data)


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-Function.ipynb".
The first task is:
Since kilometres per litre is the inverse of the fuel consumption, the vector Y is defined as follows:
Can you write Python code for it?

def simple_linear_regression(X, Y):
    xMean = np.mean(X)
    yMean = np.mean(Y)
    ϑ1    = np.sum((X - xMean) * (Y - yMean)) / np.sum((X - xMean) ** 2)
    ϑ0    = yMean - ϑ1 * xMean
    TSS   = np.sum((Y - yMean) ** 2)
    RSS   = np.sum((ϑ1 * X + ϑ0 - Y) ** 2)
    R2    = 1 - RSS/TSS
    return R2


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-with-SciKit-Learn.ipynb".
The first task is:
In order to use SciKit-Learn we have to reshape the array X into a matrix.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-with-SciKit-Learn.ipynb".
The first task is:
Let's plot the regression line with the data.
Can you write Python code for it?

xMax = max(X) + 0.2
plt.figure(figsize=(12, 10))
sns.set(style='whitegrid')
plt.scatter(X, Y, c='b', s=4)
plt.plot([0, xMax], [ϑ0, ϑ0 + ϑ1 * xMax], c='r')
plt.xlabel('engine displacement in cubic inches')
plt.ylabel('fuel consumption in litres per 100 km')
plt.title('Fuel Consumption vs. Engine Displacement')
plt.show()


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-with-SciKit-Learn.ipynb".
The first task is:
Simple Linear Regression with SciKit-Learn
We import the module pandas.  This module implements so called data frames and is more convenient than the module csv when reading a <tt>csv</tt> file.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-with-SciKit-Learn.ipynb".
The first task is:
The data we want to read is contained in the <tt>csv</tt> file 'cars.csv'.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-with-SciKit-Learn.ipynb".
The first task is:
The model M represents a linear relationship between X and Y of the form
$$ \texttt{Y} = \vartheta_0 + \vartheta_1 \cdot \texttt{X} $$
We extract the coefficients $\vartheta_0$ and $\vartheta_1$.
Can you write Python code for it?

ϑ0 = M.intercept_
ϑ0

ϑ1 = M.coef_[0]
ϑ1


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-with-SciKit-Learn.ipynb".
The first task is:
The values for $\vartheta_0$, $\vartheta_0$, and $R^2$ are, no surprise there, the same values that we had already computed with the notebook Simple-Linear-Regression.ipynb.  We plot the data together with the regression line.
The next line is needed to suppress a deprecation warning from one of the libraries.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-with-SciKit-Learn.ipynb".
The first task is:
We convert cubic inches into litres.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-with-SciKit-Learn.ipynb".
The first task is:
We convert kilometer per litre into litre per 100 kilometer.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-with-SciKit-Learn.ipynb".
The first task is:
We convert miles per gallon into kilometer per litre.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression-with-SciKit-Learn.ipynb".
The first task is:
We plot fuel consumption versus engine displacement.
Can you write Python code for it?

import matplotlib.pyplot as plt
import seaborn           as sns
%matplotlib inline

plt.figure(figsize=(12, 10))
sns.set(style='whitegrid')
plt.scatter(X, Y, c='b', s=4) # 'b' is blue color
plt.xlabel('engine displacement in litres')
plt.ylabel('litre per 100 km')
plt.title('Fuel Consumption vs Engine Displacement')
plt.show()


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression.ipynb".
The first task is:
In order to be able to plot the fuel efficiency versus the engine displacement we turn the
lists displacement and mpg into numpy arrays.  This is also usefull in order to compute the coefficients $\vartheta_0$ and $\vartheta_1$ later.
Can you write Python code for it?

import numpy             as np
import matplotlib.pyplot as plt
import seaborn           as sns


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression.ipynb".
The first task is:
Let us plot the line $y(x) = ϑ0 + ϑ1 \cdot x$ together with our data:
Can you write Python code for it?

xMax = max(X) + 0.2
plt.figure(figsize=(12, 10))
sns.set(style='whitegrid')
plt.scatter(X, Y, c='b')
plt.plot([0, xMax], [ϑ0, ϑ0 + ϑ1 * xMax], c='r')
plt.xlabel('engine displacement in cubic inches')
plt.ylabel('fuel consumption in litres per 100 km')
plt.title('Fuel Consumption versus Engine Displacement')


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression.ipynb".
The first task is:
Next, we compute the residual sum of squares RSS as follows:
$$ \mathtt{RSS} := \sum\limits_{i=1}^m \bigl(\vartheta_1 \cdot x_i + \vartheta_0 - y_i\bigr)^2 $$
Can you write Python code for it?

RSS = np.sum((ϑ1 * X + ϑ0 - Y) ** 2)
RSS


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression.ipynb".
The first task is:
Now $R^2$ is calculated via the formula:
$$ R^2 = 1 - \frac{\mathtt{RSS}}{\mathtt{TSS}}$$
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression.ipynb".
The first task is:
Now kpl is a list of floating point numbers specifying the
fuel efficiency, while the list displacement
contains the corresponding engine displacements
measured in litres.  We display these values for the first 5  cars.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression.ipynb".
The first task is:
Since kilometres per litre is the inverse of the fuel consumption, the vector Y is defined as follows:
Can you write Python code for it?

X = np.array(displacement)

Y = np.array([100 / y for y in kpl])

plt.figure(figsize=(12, 10))
sns.set(style='whitegrid')
plt.scatter(X, Y, c='b', s=4) # 'b' is short for blue
plt.xlabel('engine displacement in litres')
plt.ylabel('litre per 100 km')
plt.title('Fuel Consumption vs. Engine Displacement')


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression.ipynb".
The first task is:
The coefficient $\vartheta_0$ is computed according to the formula:
$$ \vartheta_0 = \bar{\mathbf{y}} - \vartheta_1 \cdot \bar{\mathbf{x}} $$
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression.ipynb".
The first task is:
The coefficient $\vartheta_1$ is computed according to the formula:
$$ \vartheta_1 = \frac{\sum\limits_{i=1}^m \bigl(x_i - \bar{\mathbf{x}}\bigr) \cdot \bigl(y_i - \bar{\mathbf{y}}\bigr)}{
                       \sum\limits_{i=1}^m \bigl(x_i - \bar{\mathbf{x}}\bigr)^2}
$$
Can you write Python code for it?

ϑ1 = np.sum((X - xMean) * (Y - yMean)) / np.sum((X - xMean) ** 2)
ϑ1


I am working on the file "Python/5 Linear Regression/Simple-Linear-Regression.ipynb".
The first task is:
We compute the average engine displacement according to the formula:
$$ \bar{\mathbf{x}} = \frac{1}{m} \cdot \sum\limits_{i=1}^m x_i $$
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Underfitting-Overfitting.ipynb".
The first task is:
The function $\texttt{model_and_plot}$ takes a natural number $k$ as input and creates a model that uses all of the numbers
$$ x^0, x^1, x^2, \cdots, x^k $$
as features.  The resulting model is plotted together with the data.
Can you write Python code for it?

X_test = np.linspace(0, 1, 500)
X_test = np.reshape(X_test, (len(X_test), 1))

def model_and_plot(k, X, y, X_test):
    polynomial_features = PolynomialFeatures(degree=k, include_bias=False)
    X_train = polynomial_features.fit_transform(X)
    X_test  = polynomial_features.fit_transform(X_test)
    M = LinearRegression()
    M.fit(X_train, y)
    print('R2 score =', M.score(X_test, f(X_test[:,0])))
    y_test = M.predict(X_test)
    plt.figure(figsize=(15, 10))
    plt.plot(X_test[:,0], y_test, label="Model")
    plt.plot(X_test[:,0], f(X_test[:,0]), label="cos(3/2*pi*x)")
    plt.scatter(X, y, edgecolor='b', s=20, label="Samples")
    plt.xlabel("x")
    plt.ylabel("y")
    plt.xlim((0, 1))
    plt.ylim((-2, 2))
    plt.legend(loc="best")
    plt.show()

model_and_plot(1, X, y, X_test)

model_and_plot(4, X, y, X_test)

model_and_plot(35, X, y, X_test)


I am working on the file "Python/5 Linear Regression/Underfitting-Overfitting.ipynb".
The first task is:
The function $f(X)$ is defined such that we have
$$ f(x) = \cos\left(\frac{3}{2} \cdot\pi\cdot x\right). $$
Can you write Python code for it?

def f(X):
    return np.cos(1.5 * np.pi * X)


I am working on the file "Python/5 Linear Regression/Underfitting-Overfitting.ipynb".
The first task is:
Underfitting vs. Overfitting
Adapted from https://scikit-learn.org/stable/auto_examples/model_selection/plot_underfitting_overfitting.html.
This example demonstrates the problems of underfitting and overfitting and
how we can use linear regression with polynomial features to approximate
nonlinear functions. The plot shows the function that we want to approximate,
which is a part of the cosine function. In addition, the samples from the
real function and the approximations of different models are displayed. The
models have polynomial features of different degrees. We can see that a
linear function (polynomial with degree 1) is not sufficient to fit the
training samples. This is called underfitting. A polynomial of degree 4
approximates the true function almost perfectly. However, for higher degrees
the model will overfit the training data, i.e. it learns the noise of the
training data.
We evaluate quantitatively overfitting / underfitting by using
cross-validation. We calculate the mean squared error (MSE) on the validation
set, the higher, the less likely the model generalizes correctly from the
training data.
Can you write Python code for it?

import numpy             as np
import matplotlib.pyplot as plt

from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model  import LinearRegression


I am working on the file "Python/5 Linear Regression/Vectorization-with-Numpy.ipynb".
The first task is:
To compare this time with time that is needed if a and b are stored as lists instead, we convert a and b to ordinary Python lists.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Vectorization-with-Numpy.ipynb".
The first task is:
Vectorization is Fast!
This short notebook demonstrates that working with <tt>NumPy</tt> arrays is much faster than working with Python lists.
Can you write Python code for it?


I am working on the file "Python/5 Linear Regression/Vectorization-with-Numpy.ipynb".
The first task is:
We begin by defining two <tt>NumPy</tt> arrays a and b that are each filled with a million random numbers.
Can you write Python code for it?