# Foundations of Computer Science/Recursion Revisited

## Recursion Revisited

Recursive solutions provide another powerful way to solve self-similar
problems. The example that we will examine is the binary search
solution.

### How Binary Search Works?

The process for identifying a target item in a sorted list. You start
with the first base case which is to directly solve whether the middle
item is equal to the target. If the middle item equals the target then
the search is complete and the index is reported. If the list is empty
then the search reports -1 or not found. Otherwise, decide if the target
is greater or less than the middle item to dictate which half of the
list to search.

Next, if the middle item is greater than the target the first half of
the list is searched, else, we search the second half of the list. This
is the same problem that we originally started with making this a
recursive or self-similar problem.

The image below is the binary search code implementation. The base cases
and recursive cases are labeled.

!Binary search is used to find an item (target) in a sorted list. It
consists of bases cases and recursive cases that make up the recursive
solution. in a sorted list. It consists of bases cases and recursive cases that make up the recursive solution.")

### Binary Search: Abstraction Simplification

One excellent way to simplify the binary search solution is using
abstraction. The image previously shown uses a helper block called
\"middle between\" which is used to find the middle index when given the
first and last index of a sorted list (see below).

!The helper block used in a binary search to find the middle index
between the first and last index given as
input.

Another way to simplify the binary search solution is adding another
level of abstraction. The helper block below shows the target and list
being passed in as input into the \"binary search for\" block (see
below). The actual recursive solution is implemented when the recursive
search block detects the first and last index of a sorted list. The
\"binary search for\" block allows users to call the recursive search
without the user being required to provide the first and last indices.
The user will only see the information and/or blocks necessary to start
the binary search. The user will not see the recursive search or behind
the scenes of the solution).

!The binary search helper block that adds another level of abstraction
that searches for the target in a list by calling the recursive search
block.

### Binary Search: Tracing

We have previously discussed simplifying our interface for the recursive
search using \"binary search for\" which takes the target and a list as
inputs (see example below). Then, the \"recursive search for\" is called
and the first base case is immediately solved as our target, 9, is not
equal to 5. Since the target 9 is greater than 5 (element at the middle
index), we search the second half of the list (index 4 to index 5). The
process is repeated and the list is split checking the first element at
position and/or index 4. The target is greater than 7 and the recursive
search is repeated again searching the second half of the list again.
The base case 2 immediately solves that the target 9 is equal to 9 in
the list; which located at the middle index 5.

Now, since the target has been found at index 5, the recursive search
reports back to the second recursive search which in turn reports back
to the first recursive search. Finally, index 5 is reported back to the
user at top level of the \"binary search for\" block.

!This image shows how to trace the binary search when the target is
equal to 9. Step by step the base case and recursive cases are used for
reporting.

In the event the target is not found in a list the binary search reports
-1 (see below). The same recursive search calls are made, but on the
last call the start index is greater than the last index which indicates
the target is not found and the end of the list has occurred.

!This image shows how to trace the binary search when the target is not
found in the sorted list. The solution reports -1 or not found if the
target is not in the
list.

### Koch Curve

In 1904, Helge von Koch discovered the von Koch snowflake curve, \"a
continuous curve important in the study of fractal geometry\" (3). The
Koch curve starts with a single line segment that is 1 unit long. Then,
the segment is replaced with four new segments, each one-third the
length of the original, also called the generator. The process is
repeated forever and creates the Koch curve. The image below shows this
process for stages 0 to 2.

!This shows the generator (stage 0) and iterations of stage 1 and 2 of
the Koch
curve. and iterations of stage 1 and 2 of the Koch curve.")

The length of the Koch curve is infinite. The curve is another
interesting implementation of recursion where it is self-similar; the
curve copies itself over and over.

### Koch Snowflake

Using the same Koch curve generator on all three sides of an equilateral
triangle we see repeated iterations eventually start to look like a
snowflake (please see Koch
Snowflake). The snowflake
has infinite perimeter and finite area.

Examining each stage of the Koch Snowflake a pattern is created for the
number of segments per side, the length, and the total length of the
curve as seen below. The number of segments per sides is increased four
times the previous stage. The length of the segment is divided into
equal thirds each time giving us the length of each segment. The
original stage, an equilateral triangle, is why we take three times the
number of segments per side divided by the length of a segment raised to
the number of the stage currently being implemented.

!The Koch Snowflake pattern can be seen by tracking the different
iterations.

### Exploration Questions

Studying the Koch Curve and Snowflake patterns are established and show
how the same process is repeated to generate each stage. We can use the
recursive process to solve abstractions of the same problem.

-   What would the total length of the N-th iteration be? Look at the
    patterns made by the numbers both before and after simplifying.
-   What do you expect the Koch Curve to look like? In other words, what
    would you expect to happen if you repeated this infinitely many
    times?
-   What is the length of the Koch Curve?
-   Can you estimate the area at each stage? What is the area of the
    final snowflake?

Reviewing the previous questions we can start to observe the behavior of
the Koch Curve and Koch Snowflake.

1.  Based on the patterns established it is clear that the total length
    of the N-th iteration would be 3\*(4/3)^n^.
2.  As the Koch Curve is infinitely repeated it will start to look more
    smooth as the lines will appear closer and closer although never
    touching.
3.  The length of the Koch Curve is infinity (after applying the
    generator infinite number of times).
4.  The area of the final snowflake is bounded (finite).

[^1]

```{=html}
<references/>
```

[^1]: Niels Fabian Helge von
    Koch.
    (2014). In Encyclopædia Britannica. Retrieved from
    <http://www.britannica.com/EBchecked/topic/958515/Niels-Fabian-Helge-von-Koch>


# Foundations of Computer Science/Higher Order Functions

## Higher Order Functions

higher order functions offer a more powerful ways to generalize
solutions to problems by allowing blocks to take blocks as parameters
and returning a block as a return value. All other functions are called
first order functions. An example higher order function in math is the
derivative function which takes a function as the input and produces
another function (the derivative of the first function) as the output.
In computer science, a map function takes an arbitrary function and a
data set (e.g. a list) and applies the function to each and every data
item in the set. Another example is the reduce (or fold) function, which
takes a input function and data set and produces the aggregation of all
items in the data set using the function. For instance, if the input
function is the addition the reduce function returns the sum of all
items in the data set as the output. If the input function is
multiplication, the reduce function produces the product of all items in
the data set. Higher order function also allows us to create
compositions of functions using existing functions on the fly. For
example, given two functions $y=f(x)$ and $y=g(x)$ and $b=f(a)$ and
$c=g(b)$ we can create a function $y=fg(x)=f(g(x))$ so that $c=fg(a)$.

### Examples

#### map

The following script uses the built in map block/function to apply the
same block/function to each and every element in a list.

!The built in map block (function) is used to apply a block (function)
to each and every element in a
list. is used to apply a block (function) to each and every element in a list.")

To apply a different function we simply need to find or implement the
function and use it as the first parameter to the map block. The next
example uses the multiplication function that takes two parameters.
Snap! is smart enough to detect that and use each element of the list as
both parameters when the function is applied. So the result list should
contain the elements from the original list multiplied to themselves.

!The built in map block (function) is used to apply a block (function)
to each and every element in a list. Because the function passed in
takes two parameters when the function is applied to a element of the
list the same element is used as both
parameters. is used to apply a block (function) to each and every element in a list. Because the function passed in takes two parameters when the function is applied to a element of the list the same element is used as both parameters.")

This map block generalized this pattern of applying the same function to
multiple data items into a block. It doesn\'t simply programming because
someone has to write this map block (check the source code to see how
complicated it is), but it makes programmers happier because it keeps
the thinking part simpler. As programmers we are freed from worrying
about the iteration of lists so that we can focus on the function that
needs to be applied to the list.

#### reduce

The following two examples use the built-in reduce function in Snap! to
calculate the sum and the product of a list of numbers.

!This block applies the addition function to each and every item in the
list to calculate the
total.

!This block uses the built-in reduce function in Snap! to calculate the
product of a list of
numbers.

Note that the reduce (combine with) function can take any function with
two input parameters to aggregate a list of values to a single value. By
using higher order functions we can create generalized solutions that
can be customized to solve a larger set of problems.

#### return blocks as data

The following block demonstrates the use of blocks as data. In this
block two reporter blocks are taken in as parameters, which are then
used to form the report value - a new block. The new block applies the
two input report blocks to some unknown input value. Note that the
\"ring\" (gray frame) around the report value is very important.
Whatever enclosed in the \"ring\" will be treated as data not programs.
The application of the two functions will not be evaluated but will
simply be returned as data without evaluation, which is what we wanted
in this higher order function.

!This block takes two (reporter) blocks as input parameters and reports
a new block as the output. The function of the new block is the
composition of the two functions represented by the two input blocks -
when the new block is called it takes the input to the new block,
applies the two functions (specified when the new block is created) to
the input, and reports the
result. blocks as input parameters and reports a new block as the output. The function of the new block is the composition of the two functions represented by the two input blocks - when the new block is called it takes the input to the new block, applies the two functions (specified when the new block is created) to the input, and reports the result.")

To use the composed function we can call the compose function with the
two input functions and use the \"call\" block to execute the composed
block against an input value as shown in the following script.

!The compose block is called to form a composed block (function) from
the two input blocks. The composed block is then applied to the input
value of 3. The result of this block call is
1+2+3=6. from the two input blocks. The composed block is then applied to the input value of 3. The result of this block call is 1+2+3=6.")

With this \"compose\" block we define new functions by combining
existing functions on the fly - when we need them. This is a powerful
way of generalization we cannot achieve without using higher order
functions.


# Foundations of Computer Science/The Internet and the Web

## The Internet and the Web

The Internet and the Web give us the ability to connect to countless
resources and is molding the way our society utilizes technology for
online storage and services. We will use principles previously learned
to examine Internet and Web communication. The principles we will
examine are:

-   information can be encoded into messages
-   a coordination system is a set of agents interacting with each other
    toward a common objective
-   messages can hide information

### Computer Networks

A computer network is considered a communication sub-system that
connects a group of computers enabling them to communicate with each
other. When thinking of a computer network you must consider two parts
that make it possible:

**Hardware:**

-   network interface card (NIC) - required in order to connect to a
    local area network
-   cabling or antennas - required to carry signals for transmission
-   network switches - used to relay signals

**Software:**

-   Programs - used to process information (bits) using algorithms

### Network Standard

Similar to encoding procedures used for bits, the same idea of a
standard must be used for networks. In order for communication to occur
it requires a standard for devices, message format, and procedures of
interactions. These standards provide ordered processes for
communication.

Once we have the standards in place we can examine what actually makes a
network tick. As stated previously, a computer network is made of two
parts: hardware and software. The physical hardware sets the way for
communication to travel, but does not enable the network. The software
(programs) are the pieces that make a computer network to allow
software-to-software communication.

The focus of this chapter will be on following three software standards:

-   the Internet protocol suite
-   layers of software
-   abstraction being used for simplification

### Background Definitions

Knowing the definitions from the links provided will give you a
foundation for the material in this chapter.

### Stack of Protocols

When analyzing the protocols needed to allow communication over a
network, we see that different protocols are layered to create levels of
abstraction. These abstraction layers are used both for the upper and
lower layers (see image below).

!Shows the stack of network agents used to transmit a message from one
computer to
another.

**Message Analogy**

Let\'s say that Computer A wants to send a message to Computer B. Trace
through the steps below to see how a message is sent via the two stacks
of agents.

1.  Only A4 and B4 can access the physical mailboxes to send and receive
    packages
2.  A1 puts the message into packages
3.  A2 adds sequence numbers and tracking numbers to packages
4.  A3 adds address labels
5.  A4 puts the packages in the outbox
6.  packages arrive at B4\'s inbox
7.  B3 accepts packages addressed to B
8.  B2 checks use sequence numbers to put the packages in order and
    acknowledges the packages using the tracking numbers to A2
9.  A2 re-sends a package unless acknowledged
10. B1 opens the packages to reconstruct the original message

The network protocols work in the same way with A1 to A4 and B1 to B4
being software. The delivery mechanism used between A4 and B4 usually
consists of metal wires, fiber optic cables, or radio waves in the air.

**Delivery Mechanism**

Previously, we established how information is transmitted using Computer
A and Computer B. Two delivery mechanisms that are used today for
communication between networks are circuit switching and
packet-switching. When you think about a telephone network, this network
requires a connection is established before communication can occur. For
example, when you call someone, the phone rings until the other person
picks up or voicemail initiates; this type of communication is known as
synchronous communication.

The opposite is true for computer networks which use packet-switching.
When using packet-switching, each packet (which is a small package of
information) is individually addressed and delivered separately. The
process mimics how mail packages are delivered via shared media, i.e.
trucks, trains, ships, and airplanes. For instance, when you send a
letter, you do not wait until the recipient is ready. This type of
communication is called asynchronous communication.

### The Internet

We have seen different standards and/or protocols of the Internet.The
following describes the different characteristics of the Internet which
will be important when distinguishing the Internet from the Web.

-   An infrastructure for communication (information highway)
-   A global connection of computer networks using the Internet Protocol
    (IP)
-   Uses layers of communication protocols: IP,TCP, HTTP/FTP/SSH
-   Built on open standards: anyone can create a new internet device
-   Lack of centralized control (mostly)
-   Everyone can use it with simple, commonly available software

## The World Wide Web

The World Wide Web is often confused with the Internet as it is used in
conjunction with the Internet. The web is only one of the services
provided through the Internet. It is important to know the
characteristics of the Web (see below):

-   A collection of distributed web pages or documents that can be
    fetched using the web protocol (HTTP-Hyper-Text Transfer Protocol)
-   A service (application) that uses the Internet as a delivery
    mechanism
-   Only one of the services that run on the Internet along with other
    services: email, file transfer, remote login, and etc.

### The Web

There are two roles that work together to make up the web: Web servers
and Web clients (browsers).

**Web servers**

-   Software that listens for web page requests and has access to stored
    web pages
-   Apache, MS Internet Information Server (IIS)

**Web clients (browsers)**

-   Software that fetches/displays documents fetched from web servers
-   Firefox, Internet Explorer, Safari, Chrome

### Uniform Resource Locator (URL)

The Uniform Resource Locator (URL) is an identifier for the location of
a page on the web. The system of URLs is hierarchical (see image below).

!An example of the different pieces of a
URL._example.PNG "An example of the different pieces of a URL.")

-   **edu**: a URL for a school (not .com or .org)
-   **www.sbuniv.edu**: a URL for the Southwest Baptist University (SBU)
    website
-   **www.sbuniv.edu/COBACS/CIS/index.html**: a URL to a page on SBU\'s
    website under the path

## Hyper-Text Markup Language (HTML)

The language used to define web pages is known as HTML. In order to view
an example, open another tab and navigate to the Southwest Baptist
University CIS
Department website.
Once you have the page open, right click on the page and select \"View
Source\", this will allow you to see the HTML code that was used to
create the web page. The web page itself may content
hypertext (clickable text
that serves as links). A link is just a defined URL that points to
another web page. Web pages and links are what combine to form the Web.

## Finding Information on the Web

It is important to note how to find information on the Web. Follow the
steps below to see how this process works: Use a hierarchical system
(directory) to find the URLs to pages that may have the information

-   Use our knowledge to guess, e.g. start from apple.com to navigate to
    the page for iPhone 5s
-   Use a search engine

`   -we look for information (wherever it is located) not pages`\
`   -we may find information we did not know existed`

## How a Search Engine Works

One of the main sources for locating resources can be found using a
search engine. However, have you ever thought about how they actually
work? There is a series of steps that describe exactly what happens when
a search engine is used:

1.  Gather information: crawl the web
2.  Keep copies: cache web pages
3.  Build an index
4.  Understand the query
5.  Determine the relevance of each possible result to the query
6.  Determine the ranking of the relevant results
7.  Present the results

**Measure of Important Pages**

Once a search is performed relevant pages are provided. However, not all
relevant pages displayed are considered important. A web page does not
gain importance until it has been ranked by credible sources. One of
Google\'s innovations is page rank - a measure of the "importance" of a
page that takes into account the external references to it. A page is
considered more important based on the number of important pages that
link to that page. For example, an electronic article from the New York
Times would have a higher level of importance or page rank than a
personal blog due to the number of important pages linked to that online
article.


# Foundations of Computer Science/Encryption

## Encryption

In order to ensure secure communication takes place encryption methods
must be used. Secure communication over the web is important for areas
such as e-commerce. Encryption is used to encode messages ensuring no
one, but the intended recipient knows the content of the message.

The messages that are transferred over the Internet in the form of
packets. When you think about packets, they are more like postcards than
letters. The content of each packet are plaintext (exposed for all to
see) as the bits are transmitted.

The best way to protect these packets during transmission and after
reception is using encryption techniques. Encryption is simply the
process of converting information (plaintext) into unintelligible text
(ciphertext) to avoid unwanted parties from intercepting the message. In
order for the recipient to understand the ciphertext they must use a
decryption method. Decryption reverses the process of ciphertext back
into plaintext.

The two parts: encryption and decryption are part of what is known as
cryptography. Cryptography (secret writing) is the practice and study of
techniques for secure communication in the presence of third parties. It
is not a new practice and has been around since early 2000 B.C..

### Caesar Cipher

The Caesar cipher is an example of a substitution cipher. This cipher
uses a letter-by-letter translation to encrypt messages. A cipher is
simply a method (algorithm) used to transform a message into an obscured
form and reversing the transformation. An example of this particular
cipher can be seen below where you replace each letter in the top row by
the corresponding letter on the bottom row:

!Caesar Cipher
example.

With the Caesar cipher there are 25 possible variations representing one
for each different amount of shifting. The key to remember about the
encryption and decryption rule is the amount of the shift. If we know
the Caesar cipher is used then we could try all possible 25 shifts of
the alphabet to decrypt the message. However, tools have been created to
encrypt and
decrypt
messages created using this cipher.

## Substitution Ciphers

Substitution ciphers are ciphers that use one symbol is substituted for
another according to a uniform rule. The example below shows a
substitution table that defines a rule for reordering letters in the
alphabet. How many possible reordering possibilities can be performed
using the example below?

!Substitution cipher
!Number of methods
possible. These
type of ciphers appear to be unbreakable, but that is not true.
Frequency analysis is used to decode substitution ciphers . This
technique used to break general substitution ciphers uses frequencies
letters that appear in a language.The image below shows the original
message with symbols. We will use frequency analysis to decode the
cipher.

!Original encoded
ciphertext.

After we have replaced the most used characters with E and T we can
begin using other common symbols and sentence structure to fill in the
gaps.

!Process of using conjectural
decoding.

Finally, after replacing symbols with frequently used letters we see the
entire message displayed below.

!Complete ciphertext
message.

### Vigenère Cipher

The Vigenere cipher is similar to the Caesar cipher, but it uses
multiple Caesar ciphers to encode a message. For a long time the
Viegenere cipher was considered unbreakable until the 1800s when Charles
Babbage discovered a way.

!This table shows the key to the cipher thomasbbryan. This cipher was
used by an attorney named Thomas B. Bryan in 1894 to communicate with
his
client.
!Key description

The substitution table used above encrypts and decrypts messages. We use
the second column, \"thomasbbryan\", to uniquely identify the table.
This key is used to specify which cipher is used.

The Vigenere cipher was unbreakable until the method to decode this
cipher was discovered in 1863. Although the cipher is no longer secure,
it was at the time a great enhancement to secure communications.

### Vernam Cipher

The weakness discovered with the Vigenere cipher is the repeated use of
the same key. In order to combat this problem the Vernam cipher was
created. The key is as long as the plaintext so that no repetition is
needed. For example, if we wanted to use the Vernam cipher to encrypt
the message the length of 100, we might use 100 Caesar ciphers extended
to 100 rows. This was a one-time pad used to encrypt messages. The
Vernam cipher was used widely during World War II and the Cold War.

In principle, the one-time pad is as good as it gets when it comes to
cryptography. This process is mathematically provable. The Vernam cipher
operates in a similar fashion to the Caesar cipher. Wherein the Caesar
cipher one number key is used as the shift cipher, the Vernam cipher
operates through the use of many different shift ciphers being used, a
unique one for each letter in the key. This is done by shifting the
character according to the value of the letter it corresponds to in the
alphabet. For example if the letter of the key was \'A\' that would lead
to a shift of 1.

When used correctly, the one-time pad is unbreakable, but it is
difficult to transmit the one-time pad between the parties without
interception. Another challenge is that the cipher (one-time pad) is
impractical. If there is a way to transmit the message, then the person
might as well send the message itself due to the length and complexity.

Today, we use more innovative practices to secure communication.
Sophisticated ciphers (programs) use shorter keys and these keys are
sequences of bits on which both parties agree to keep secret. This
process works because computers divide ASCII-coded plaintext messages
into blocks. The bits that make up that block are transformed according
to a specific method that depends on the secret key created.

There are no known shortcuts for breaking secret key ciphers. Even using
a brute-force attack is difficult because it requires guessing all
possible keys but as the attack occurs the process grows exponentially
in time based on the size of the key. Increasing the key length by only
one bit doubles the work required to break the cipher. By creating
longer keys it makes it possible to have the work outgrow the actual
computing power. Due to this, breaking these ciphers is possible, but
computationally infeasible, taking hundreds of years or more to crack.

The challenges that come with using secret key encryption is that the
number of keys required increases as the number of network members
increases. For each pair of members a new shared secret key must be
created. Creating unique keys becomes more complicated as more
combinations are needed. Another challenge is securely establishing a
secret key between two parties when a secure channel does not exist
between them.

### Public Key Encryption

In 1976, Whitfield Diffie and Martin Hellman proposed the idea of
public-key encryption. The idea is of two mathematically related keys a
public key and a private key. The keys are paired, but computationally
infeasible to connect to each other. A message encrypted using the
private key can only be decrypted by the public key and vice versa.

When a user picks a secret key and encrypts the message with the
recipient\'s public key and sends the ciphertext to the recipient. The
recipient then uses their private key to decrypt the ciphertext to get
the secret key. The private keys are kept secret and never sent to the
other user. The two can communicate using the secret key (also known as
a session key). The confidentiality of the message is ensured due to no
one except the recipient being able to decrypt the message from the
initiator.

The way to ensure the message is from the sender is to use digital
signature schemes. Signatures should be easy for a user to produce, but
difficult for anyone else to forge. Digital signatures can also be tied
to the content of the message being signed. Authenticity of the message
is verified due to the ciphertext only being decrypted with the intended
party\'s private key.


# Foundations of Computer Science/Simulation

## Simulation

Simulation can be a very powerful way to represent real-world systems,
scenarios, and experiments. Simulation is the recreation of a real-world
system in a prepared and controlled environment. As we study different
objects and environments we see the complexity of measuring, analyzing,
and emulating events such as nature. Although it is impossible to
produce 100% accuracy and detail; the two are used to attempt to create
an environment as close as possible to mimicking real-world systems. In
order to achieve a higher level of accuracy and detail we focus on the
most important aspects of the system we hope to simulate. The higher the
granularity in detail the more likely we are to accurately predict what
will happen in a real world environment.

The three motivating factors behind using simulation instead of real
world experimentation are:

-   **Control** - Gives us the ability to explore problems that were
    utterly out of reach due to our lack of control over a real world
    situation. For example, if we are attempting to simulate a storm or
    hurricane these are events we have no control over, but studying the
    paths could be beneficial in predicting future effects and climate
    changes.
-   **Cost** - Conducting experiments in real system can be costly both
    in regards to time and money. For example, instead of automotive
    manufacturers crashing several cars for testing they can use
    simulations to mimic different car crashes, angles, and scenarios
    without spending time and money on an actual car.
-   **Safety** - Certain experiments can be dangerous or harmful and
    simulations. Scientists are able to simulate events such as virus
    outbreaks, aircraft engine failure, and even testing nuclear bomb
    material (more info on atomic bomb
    simulations).

## Modeling

Modeling is the process of describing how the components of a simulation
looks and behaves. During this process we model the behavior and
interactions of all components. Although we are using loose
approximations these simulated versions can be a surprisingly accurate
recreation of the real world system.


# Foundations of Computer Science/Artificial Intelligence

## Artificial Intelligence

### What is A.I.?

Artificial Intelligence (AI) is the idea of building a system that can
mimic human intelligence. How we determine intelligence is based on how
people plan, learn, natural language processing, motion and
manipulation, perception, and creativity. These various areas are used
in the process of engineering and developing Artificial Intelligence.

### The Turing Test & A.I.

One concept that is important to note is that computers only perform
algorithms and programs given, they cannot inherently create algorithms
on their own. We have seen in previous chapters where algorithms can
morph or change, but not without being given an algorithm.

Previous chapters have breached the topic of the Turing Test. The Turing
Test is used as a theoretical standard to determine whether a human
judge can distinguish via a conversation with one machine and one human
which is a human and which is a machine. If a machine can trick the
human judge into thinking it is human then it passes the Turing Test.

Although there are several innovative developments in the field of AI
there are still areas that are being tested in improved. Once such
example can be found with CAPTCHAs (Completely Automated Public Turing
test to tell Computers and Humans apart). The CAPTCHAs are used to
distinguish between human and computer. Even today, computers are unable
to identify images such as those generated by CAPTCHAs as well as
humans. Those developing CAPTCHAs are using these as a tool to teach
computers to recognize and learn words and/or images that humans are
able to identify. CAPTCHA is considered a reverse Turing
test because a computer is
determining whether a human user is indeed human.

### Intelligent Agent Approach

The intelligent agent approach was first introduced in the quest for
\"artificial flight\". The Wright brothers and others stopped their
attempts to imitate birds and to instead embrace the idea of
aerodynamics. The goal is not to duplicate, but to use what is known
about flying and manipulate that knowledge.

A major part of this approach is a rational agent, which is an an agent
used to achieve the best outcome. Therein lies the connection to the
intelligence test of rationality. Basically, can this machine or
computer mimic the rational behavior of a human.

### History of A.I.

The concept of AI began around 1943 and became a field of study in 1956
at Dartmouth. AI is not limited to the Computer Sciences disciplines,
but can be seen in countless disciplines such as Mathematics,
Philosophy, Economics, Neuroscience, psychology and various other areas.
The areas of interest in the Computer Science and Engineering field are
focused on how we can build more efficient computers. Great advancements
have been made in the area of hardware and software.

### A.I. Knowledge-based Expert System

An A.I. system often times will use a rule-based system to capture
knowledge in the form of if-then statements. Another way to think about
these rule-based systems is as decision trees. Decision trees use these
preset rules to determine the decision path to follow based on input
provided. An example of a decision tree or rule-based system is a single
player game. In the game the player imagines an animal (real or
imaginary) and answers a series of questions, which are designed for the
computer to guess what the animal is assuming the player always answers
the questions truthfully.

### Machine Learning

There are two types of machine learning: formal and informal. The
informal involves giving computers the ability to learn without
explicitly programming the capability (Arthur Samuel, 1959). The formal
type of machine learning is a computer program that learns from
experience in respect to some task and increases performance based on
that experience (Tom Mitchell, 1998).

### Supervised Learning

The supervised learning is based on giving the correct answers and
having the computer mapping inputs to outputs. Examples of supervised
machine learning are:

-   United States Postal Service using computers to read zip codes on
    envelopes and automatically sort mail (i.e. handwritten zip codes)
-   Spam filters - software is trained to learn and distinguish between
    spam and non-spam messages (i.e. email filters)
-   Facial recognition- used by cameras to focus and via photo editing
    software to tag persons (i.e. Facebook)

### Unsupervised Learning

Unsupervised learning is simply the reverse of supervised learning where
the correct answers are unknown. The goal or objective of unsupervised
learning is to discover structure in data, which is also known as data
mining. The computer looks at data to find trends to make and/or assist
in decision making. Below are examples of unsupervised learning:

-   Clustering algorithm - Used to find patterns in datasets and then
    group that data into different coherent clusters.
-   Market segmentation - targeting customers based off regions, likes,
    dislikes, when the consumer makes purchases, etc. This is considered
    targeted marketing.
-   Recommendation systems - systems or software that make
    recommendations to the consumer as to what they may like based off
    their preferences (i.e. Netflix, Hulu, etc.).
-   Statistical Natural Language Processing - used to guess the next
    word or auto-complete words based off of correcting/guessing
    techniques, suggesting news stories, or translating texts)

### Genetic Programming

Genetic Programming is an idea that
uses evolutionary process to improve algorithms.

### Future of A.I.

There are many challenges in mimicking human intelligence. Humans
acquire common senses that are intuitive but hard to reason rationally,
e.g. the color of a blue car is blue. Deep
learning is a branch A.I. that aim to
create algorithms that can acquire intuition.


# Foundations of Computer Science/Limits of Computing

## Limits of Computing

We have studied some big ideas of computing, which allows us to perform
amazing tasks through information process. You might have gotten the
impression that if we can quantify information and design an algorithm,
we can solve any problem using computing. In this chapter we will study
the theory about the limits of computing. There are theoretical and
practical limits on what computing can do for us.

### Turing Machine Revisited

When we talked about the formal definition of \"algorithm\", we
introduced the Turing
machine, which is a
mathematical model for computation. With this model we can study the
nature of computing including what it is can possibly do. Alan Turing
contributed greatly to the computability theory with his model machine.
He proved that Turing machine is as powerful as any computer we can ever
build (Turing
completeness). If we
can find a problem the Turing machine cannot solve, we just proved the
solution is not computable, i.e., the problem is not solvable through
computing on any computer.

### Halting Problem

In 1936 Alan Turing proved that a general algorithm to solve the
halting problem for all
possible program-input pairs cannot exist, i.e., the halting problem is
unsolvable using computing. More specifically, the halting problem is
undecidable because it is the simplest type of question that answers a
yes or no (decision) question: whether a program will eventually halt
(stop) given an input. Obviously, it is infeasible to actually run the
program with the input to see whether it will halt because it may take
forever if there is an infinite loop in the program. The idea is to
analyze a program with given input to determine whether it will halt or
not.

Alan Turing proved the halting problem is undecidable by a proof
technique called proof by
contradiction. It
would be very helpful to review some of the sample
proofs on
Wikipedia. Another classic example is the proof of Euclid\'s
theorem
in number theory, which asserts that there are infinitely many prime
numbers. All proofs by contradiction start with an assumption that the
proposition we are trying to proof is false, follow a logical sequence
of valid steps, and arrive at a conclusion that is clearly false or
contradicts, which is also false because a statement can be either true
or false but never both.

If we assume the halting problem is decidable/solvable we should be able
to design an algorithm and implement it as a program that solve the
problem for us. The program should take a program and the input to the
program as input parameters and return the answer - whether the program
will halt or not on the input. Such a program may sound strange because
it takes a program as input, but we have seen such programs, e.g., the
higher order function blocks in Snap! takes a block (program) as input
and an interpreter
program) takes
the source code of a program as data and runs the program. Programs are
data and there no intrinsic difference between the tow. The following
proof shows that a program that answers the halting question cannot
exist.

1.  Assume such a program A exists. A(P, D) -\> whether program P halt
    on input data D.
2.  We can create another program B that takes a program as input.
    Inside program B we can call program A with the given input to
    determine whether the input program will halt on itself as input
    data and if the answer is no (will not halt) we halt (return) and if
    the answer is yes (will halt) loop forever.

` B(P):`\
`   if A(P, P) = yes `\
`     infinite loop`\
`   else`\
`     return `\

1.  What happens if we feed program B to itself as the input? or more
    simply, would program B halt on itself? There are two possible
    outcomes of the exercise: program B halts on itself or program B
    runs forever when itself is the input. The actual outcome/result
    depends on the outcome of program A called inside program B. If
    program B halts on itself, according to the design of program B it
    should run forever because the program A with program B as both
    inputs should run an infinite loop. On the other hand, if program B
    will not halt on itself, the output from program B with itself as
    input should return right away. Either way it is a contradiction.
2.  So far, we have made an assumption, followed a set of logically
    sound steps, and finally arrived at contradictions. What went wrong?
    The contradictions cannot be true because they are logically
    impossible. The steps are logically sound and the only part that
    could be wrong is the assumption. Therefore, the assumption cannot
    be true, i.e., the halting problem is not decidable.

Here is a YouTube video illustrating the proof:
<https://www.youtube.com/watch?v=92WHN-pAFCs>

### Intractable Problems

The Halting problem is hard because it is not solvable algorithmically
even in principle. There are other hard problems that are solvable in
principle but in practice they are close to being impossible to solve.
As you can see, we can categorize problems by the performance of the
best-known algorithms. If a problem can be solved using a fast
algorithm, the problem is easy because we can use a computer to solve it
fast. On the contrary if the best-known algorithm we know takes a long
time to solve a problem, it is hard because computers cannot solve it
fast.

Using algorithm complexity
theory
we can put each algorithm into a particular category according to the
algorithm\'s complexity. If the big-O notation of a category contains
only polynomial terms, the problems solvable using algorithms in this
category are called P problems (Polynomial time solutions exist), such
as $O(1)$, $log_2(N)$$O(N)$, and $O(N^2)$. The P problems are the easy
problems to computers. Among the problems without a polynomial time
solution there are problems that if we can guess the solution it can be
verified in polynomial time. For example, the integer
factorization
(or prime decomposition) problem has no known polynomial time solution
but given an answer we can verify it very quickly by simply multiplying
the factors and comparing the result to the integer. These types of
problems are called NP (Non-deterministic Polynomial) problems.

Collectively we call problems that take too long to solve intractable
problems, which include problems with best algorithm in exponential time
($O(2^N)$) or those with polynomial time solutions but the exponent is
too larger, e.g. $O(N^{15})$.

If a problem\'s best algorithmic solution is in the $O(2^N)$, when
$N=100$ and a computer does $10^{12}$ operations per second it would
take $4 \times 10^{10}$ years (the age of the universe) to solve the
problem on the computer.

### P v.s. NP

Obviously P is a subset of NP because NP is defined only by polynomial
answer verification time and being able to solve a problem in polynomial
time (P) certainly qualifies for that. Whether P is a proper subset of
NP or, in other words, whether $P=NP$ remains one of the open questions
in computer science. Nobody knows the answer. You can win a million
dollars if you can solve it as one of the Millennium Prize
Problems.

To attack this P v.s. NP problem theoretical computer scientist have
defined another category called
NP-complete problems. The
relationships among the three categories are illustrated in the
following figure.

!Diagram of complexity classes provided that P ≠ NP. The existence of
problems within NP but outside both P and NP-complete, under that
assumption, was established by Ladner\'s
theorem.\[1\]

All problems in this category are NP problems sharing one special
property - ALL other NP-complete problems can be translated/reduced to
each of the NP-complete problems in polynomial time. Because of the
nature of NP-completeness if we can solve ANY single problem in this
category, we have proved all NP problems are solvable in polynomial
time, i.e., $P=NP$. We can take any NP problem, translate it to the
solved NP-complete problem, and solve the problem in polynomial time.
The overall time is still polynomial because polynomial + polynomial is
polynomial. Thousands of NP-complete
problems
have been discovered but none of them has been solved. NP-complete
problems are, in a sense, the most difficult known problems.

Most computer scientist believe $P \ne NP$ because the implications
otherwise. The \"creative leaps\" will disappear because solving a
problem is as easy as being able to recognize the right answer. Most
encryption algorithms are computationally secure because breaking them
are NP-problems - there is no known efficient polynomial time solution.
If $P=NP$ then all encryptions are broken.

There are other unsolved problems in computer science. You can find a
list of such problems at
<https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_computer_science>


# Foundations of Computer Science/Computing Machinery

## Computing Machinery

We have studied some fundamental principles of computing and seen the
power of computing demonstrated in powerful technologies that operate on
these principles. At the beginning we imagined that computing can be
done purely mechanically and blindly through symbol manipulation if we
can build a simple device that is capable of some simple tasks. In this
lesson we will study the history of the development of
computers and the principles
on which all modern computer hardware operate. We will see on the
physical layer a computer is nothing but a simple device that follows
simple rules to manipulate two symbols.

### Computer Hardware

Computer hardware that can carry out computation automatically has been
around for too long. There is a short history from early
computers
to modern computers we have today.

#### Mechanical Computers

Charles Babbage invented the concept of a programmable computer and
designed the first mechanical computer in the early 19th century. His
difference engine and
analytical engine are
mechanical devices designed to tabulate polynomial functions. The input
to the machines are programs and data on punched cards and the output
number can be pouched onto cards or directed to a printer, a curve
plotter and a bell. The machines use ordinary base-10 fixed-point
arithmetic. Charles Babbage\'s engine is the first design for a
general-purpose computer that is
Turing-complete.

#### Analog Computers

In early 20th century mechanical analog computers are designed to
perform increasingly sophisticated computing, e.g. solving differential
equations by integration. Analog
computers use
continuously changeable aspects of physical phenomena such as
electrical, mechanical, or hydraulic quantities to model/quantify
computation, which lack the accuracy of modern digital computers.

#### Digital Computers

The world\'s first fully automatic digital computer is the
electromechanical programmable computer
Z3) made by Konrad Zuse in
1941. Z3 uses electric switches that drive mechanical relays to perform
computation. It replaces the decimal system with the binary system and
pioneered the use of floating point numbers. Program and data are stored
on punched film. The distinction between a digital device and an analog
device is whether the representations of values are discrete or
continuous. For example, black and white are discrete values but their
infinite number of grayish colors (a grayscale).

#### Electronic Digital Computers

The world\'s first electronic digital programmable computer is Colossus,
built with a large number of valves (vacuum tubes) in 1943. The design
was all electronic and was used to break the German Enigma code.

#### Transistor Computers

From 1955 vacuum tubes were replaced by transistors in computer designs
resulting in smaller, more reliable and more power efficient second
generation computers, giving rise to the \"second generation\" of
computers.

#### Integrated Circuit Computers

The invention of integrated circuit in 1952 ushered in a new era of
computing machinery - micro-computers. Microprocessors with integrated
circuit are used to build common computing devices you see today:
desktop computers, laptop computers, phones, and greeting cards.

### Principles of Digital Computing

The mathematical foundation of digital computing is Boolean logic
invented by George Boole.
Claude Shannon proved in
the 1930s that electronic circuits can compute in binary using Boolean
logic, which becomes the fundamental principle/idea behind all modern
computing devices.

#### Boolean Algebra

Boolean algebra has three operations AND, OR, and NOT on two values:
true and false. The rules for evaluating the three operations are shown
in the figure.

!The truth table showing the rules for the three basic Boolean
operations.

A Boolean operation operates on Boolean values and always result in a
Boolean value. For the AND operation the result is true only when both
operands are true. The OR operation, on the other hand, only results in
a false value if either of the two operands is false. The NOT operation
takes one operand and simply negates it. We will see if we can build
electronic circuits that implement the three operations we can build
circuits that can perform all kinds of arithmetic and logic functions.

The three boolean operations can be implemented physically using
transistors. A transistor is fundamentally a tiny switch as shown in the
figure.

!A transistor is fundamentally a tiny switch with three pins. When a
logical 1 is applied to the control pin the switch is closed connecting
in to
out.

When a high voltage (logical 1) is applied to the control pin the switch
is closed connecting the in pin directly to the out pin. Transistor
operates on two voltages: high and low, which can be used to represent
two different logical values: true and false or two binary values: 1 and
0. We will use a high voltage to physically represent a logical 1 and a
low voltage a logical 0.

#### Transistors and Logic Gates

Transistors are simple devices that are tiny, but it is fundamental
building block of electronic circuits. For example we can build a device
called NOT gate using a single transistor as shown in the figure.

!A NOT gate constructed using a single
transistor.

If we treat what\'s inside the red box as a unit it behaves like
negator, which is known as an NOT gate in digital logic design. As shown
in the truth table for this device (next to the figure) when the input
is logical 1 the switch is closed connecting the output to the ground
dragging the output voltage to a low signifying a logical 0. On the
other hand, when the input is logical 0 the switch stays open which
result in a high voltage on the output line because it is connected to
the power through a resister and without a current the resister will not
cause any drop in power. Once this device is built we can use it as a
building block to construct more complicated circuits. Will use the
following symbol to represent a NOT gate.

!NOT gate

With transistors and the NOT gate we can build a device that performs
the AND operation.

!An AND gate built using two transistors and a NOT
gate.

As shown in the figure the device performs exact an AND operation. The
output is logical 1 (high voltage) only when both inputs are logical 1,
which causes both switches to close dragging the output before the NOT
gate to low. The NOT gate then negates the output to be high voltage or
logical 1.

Similarly we can build a device for performing the OR operation.

!An OR gate built using two transistors and a NOT
gate.

A shown in the figure this kind of parallel structure guarantees that
the output is high voltage as long as any transistor is closed. In other
words, the relationship between the two inputs and the output is a OR
logical function.

#### Gates to Circuits

With the three basic gates (AND, OR, NOT) we can build any combinational
logic circuit. A circuit consists of input wires, gates connected by
wires, and output wires. Once a circuit is designed it can be viewed as
a black box that implements some logic mapping input to output. Here is
the standard circuit construction algorithm:

1.  build a truth table from the desired logic
2.  build a logic expression in sum-of-products form
3.  convert the expression to circuit design using gates

We want to construct a circuit that test the equality of two bits. The
two inputs are the two bits represented physically by a high voltage
(logical 1) or a low voltage (logical 0). According to the desired logic
of the circuit we can draw the following truth table:

!This logic tests whether two bits are the
same/equal.

The first two columns enumerates all possible value combinations of the
two input lines. The output is logical 1 (true) only when the two inputs
are the same. Based on the truth table, we can derive the following
logic expression (sum-of-products form):

`(a AND b) OR ((NOT a) AND (NOT b))`

To derive the sum-of-products form we check lines in the truth table
with a logical 1 for the output. We know the input combinations shown in
these lines are supposed to cause the output to become logical 1. We can
represent each of these lines using a logic expression, for instance, (a
AND b) represents the last line because when both a and b are logical 1
the expression evaluates to a logical 1 according to the definition of
the AND operation. Similarly ((NOT a) AND (NOT b)) represents the first
line. If we combine the two cases, we can represent the desired logic
using a single expression: (a AND b) OR ((NOT a) AND (NOT b)). If we
plugin the inputs for the cases in the truth table this expression
should evaluate to the same desired output values for the corresponding
cases. Because we know how to build the devices (gates) that implement
AND, OR, and operations we can build a device that can compare whether
two bits are equal. This is device will be able to perform this kind of
operation purely mechanically (blindly) because it doesn\'t know the
meaning of the operation.

Using the same methodology we can gradually build more and more
complicated circuits. For example we could build a device that can add
two binary digits: one-bit adder. It is just a matter of figuring out
the desired logic and construct the device using the building blocks we
already know how to make.

!An adder circuit that adds two binary
bits.{width="400"}

Once we get the sum-of-products form of the logic expression from the
truth table it is straight forward for us to construct the circuit
because all we need are the three types of logic gates and wire
connection. The following figure shows the design of a one-bit adder
(with carry-in) circuit using the three basic logic gates.

!The circuit for producing the sum bit of a one bit
adder.{width="400"}

We can construct multi-bit adders by connecting multiple one-bit adders.
The following figure shows that a two-bit adder can be formed by
connecting the carry-out of the first one-bit adder to the carry-in of
the second one-bit adder.

!A 2-bit adder circuit constructed from two 1-bit
adders.{width="400"}

!A Simple Flow Chart for a Lamp Diagnosis
Algorithm

!A Flow Chart for the Algorithm that Computes N! (factorial of
N)")

Computer data
storage


# Foundations of Computer Science/Parallel Processing

Computing is fundamentally about information processes. On a digital
computer such processes are carried out via symbol manipulations in
binary logic. With the advancement in semiconducting technology we have
been able to keep making computers run faster---manipulate bits at a
higher speed---by cramming more transistors into computer chips. This is
known as the Moore\'s law
originated around 1970\'s. The trend of increase has slowed down and
will eventually flatten due to limits in physics as predicted by some
physicist, who also
predicted potential new technologies that may replace semiconductors
(silicon) in computer hardware manufacturing.

In the meantime, hardware companies have tweaked their technologies to
maintain the growth in hardware capacity. Multicore technology replaces
one fast CPU (Central Processing
Unit---the brain
of a computer) with many slower ones (called cores) to avoid overheating
the chip. Even though each core is a slower but we get more of them and
could get more done if we can arrange the work properly. For instance, a
strong worker can lift 100 bricks a minute and a normal worker can only
lift 34 bricks. Three normal workers can outperform one strong worker
even though they are much slower individually. This is the idea of
parallel processing.

Traditionally computer program has been written to describe sequential
processes, which means the steps can only be carried out one at a time
and one after another in a sequence. This type of program works fine on
a computer with a single processor because the computer can perform one
symbol manipulation at a time any ways. In fact we have been reaping the
benefit of Moore\'s law: every two computer hardware double it speed
causing our program run twice as fast without us doing anything. This
trend has stopped. Each individual processor (core) is not getting
faster but we have more of them in a computer. As a result our existing
sequential program will run slower even though the hardware\'s capacity
has become larger. Before the next generation of computers are invented
we can parallel
computing/processing
to solve problems faster computationally.

The idea of parallel processing is not new. For example, a car assembly
line allows multiple cars to be built at the same time. Even though
different parts of the car are being assembled at a given time this
assembly line keeps all the workers busy increasing the throughput
(number of cars built per unit time) of the whole system. We can make
the workers work faster to further increase the throughput or we could
add another assembly and hire more workers. This is one form of parallel
processing - pipelining. Another form of parallelism divides the whole
computing task into parts that can be computed simultaneously and run
them physically on different CPU (computers). This is similar to putting
a jigsaw puzzle together with friends. As you can imagine having some
extra help will definitely help solve the puzzle faster, but do it mean
the more the better. The answer is no. As the number of helpers increase
the amount of coordination and communication increases faster. When you
have too many people they may start stepping on each other\'s toes and
competing with each other for resources (space and puzzle pieces). This
is known as the overhead of parallel processing, which causes the
diminishing return on investment. We can see this pattern clearly when
we measure the improvement in performance as a function of the workers
involved.

In the context of parallel processing/computing we use a metric called
speedup to measure the
improvement in performance. The achieved speedup equals the
solution/execution time of a program with out parallel processing
divided by the execution time of the same task with parallel processing.

$$S = \frac{T_{old}}{T_{new}}$$ **where:**

-   $S \$ is the speedup.
-   $T_{old} \$ is the old execution time without the parallel
    processing.
-   $T_{new} \$ is the new execution time with the parallel processing.

If parallel processing makes a program run twice as fast the speedup is
two (a.k.a two-fold speedup). Theoretically as we double the number of
workers or resources we can expect a two-fold speed up. Practically it
is hard to achieve this optimal speedup because some tasks are not
always parallelizable. For example you can not usually lay the carpet
before the floor of house is constructed and you cannot always add more
painters to get the painting job down faster. Computational tasks often
have similar dependency and resources constraints to keep us from fully
utilize the parallel processing systems (e.g. multi-core computers) we
have.

Exercise:

With a washing machine and a dryer you could work on one load of laundry
at time. You wash it first and then put it into the dryer. Assume the
whole task takes an hour. This works perfectly when you have only one
load to do and there is nothing you can do to make it go faster. What if
you have many loads of laundry to do? You can at least at one load done
every hour. Can you \"speed it up\"? If the number of loads can be
arbitrary larger what is shortest average per load laundry time you can
achieve?


# C Programming/Why learn C?

C "wikilink") is the most commonly
used programming language for writing operating
systems. The first operating
system written in C was Unix. Later
operating systems like GNU/Linux were all
written in C. Not only is C the language of operating systems, it is the
precursor and inspiration for almost all of the most popular high-level
languages available today. In fact, Perl,
PHP,
Python "wikilink") and
Ruby "wikilink") are all written
in C.

By way of analogy, let\'s say that you were going to be learning
Spanish, Italian, French, or Romanian. Do you think knowing Latin would
be helpful? Just as Latin was the basis of all of those languages,
knowing C will enable you to understand and appreciate an entire family
of programming languages built upon the traditions of C. Knowledge of C
enables freedom.

### Why C and not assembly?

The biggest reason to learn C over
assembly is because it\'s much easier
and faster to write code in C than in assembly for a given programming
task. With C, you will write far fewer lines of code, complete the job
much quicker, and with far less mental effort than if you wrote it in
assembly. And with today\'s modern compilers, an executable file
compiled from C source code will typically run faster than one written
\"by hand\" using assembly. Only in rare edge cases, and only if you
really know what you are doing, can assembly offer important speed
advantages over C code compiled with a decent compiler.

And with C, you do not have to sacrifice a lot of low level control over
how your code is executed. A typical C statement translates into just a
few assembly instructions. But C also provides you with a large software
library to help you execute low-level tasks that you\'d rather not be
bothered programming.

Another huge advantage of C is portability. Different processors have
different instruction sets. Having to rewrite and maintain assembly code
for each computer architecture you wish to execute your code on is an
onerous task. And so one of the main strengths of C is that it combines
universality and portability across various computer architectures while
still giving you the same kind of low level hardware control you get
with assembly. This means you can write your C source code once and
easily compile it into binaries for use on a wide variety of machines.

For example, C programs can be compiled and run on the HP 50g calculator
(ARM processor), the TI-89 calculator
(68000 processor), Palm OS Cobalt
smartphones (ARM processor), the original iMac
(PowerPC), the Arduino (Atmel
AVR), and the Intel iMac
(Intel Core 2 Duo). Each of these devices has its
own assembly that is completely incompatible with the assembly of any
other. C makes it possible to run your code on these machines with much
less effort.

So is it any wonder that C is such a popular language?

Like toppling dominoes, the next generation of programs follows the
trend of its ancestors. Operating systems designed in C always have
system libraries designed in C. Those system libraries are in turn used
to create higher-level libraries (like
OpenGL, or
GTK), and the designers of those libraries
often decide to use the language the system libraries used. Application
developers use the higher-level libraries to design word processors,
games, media players and the like. Many of them will choose to program
in the language that the higher-level library uses. And the pattern
continues on and on and on\...

That said, learning assembly can be fun and worthwhile because it can
give you a deep understanding of how your computer works at very low
levels. And learning assembly will definitely help you become a more
skilled C programmer. So, by all means, we encourage you learn assembly,
but when it comes time to do real work, you\'ll definitely want to get
it done with C.

### Why C, and not another language?

The primary design of C is to produce portable code while maintaining
performance and minimizing footprint (CPU
time,
memory usage, disk I/O, etc.).
This is useful for operating
systems, embedded
systems or other programs where
performance matters a lot ("high-level" interface would affect
performance). With C it's relatively easy to keep a mental picture of
what a given line really does, because most of the things are written
explicitly in the code. C has a big codebase for low level applications.
It is the "native" language of UNIX, which makes it
flexible and portable. It is a stable and mature language which is
unlikely to disappear for a long time and has been ported to most, if
not all, platforms.

One powerful reason is memory allocation. Unlike most programming
languages, C allows the programmer to write directly to memory. Key
constructs in C such as structs, pointers and arrays are designed to
structure and manipulate memory in an efficient, machine-independent
fashion. In particular, C gives control over the memory layout of data
structures. Moreover dynamic memory allocation is under the control of
the programmer (which also means that memory deallocation has to be done
by the programmer). Languages like
Java "wikilink") and Perl shield
the programmer from having to manage most details of memory allocation
and pointers (except for memory leaks and
some other forms of excess memory usage). This can be useful since
dealing with memory allocation when building a high-level program is a
highly error-prone process. However, when dealing with low-level code
such as the part of the OS that controls a device, C provides a uniform,
clean interface. These capabilities just do not exist in most other
languages.

While Perl, PHP, Python and Ruby may be powerful and support many
features not provided by default in C, they are not normally implemented
in their own language. Rather, most such languages initially relied on
being written in C (or another high-performance programming language),
and would require their implementation be ported to a new platform
before they can be used.

As with all programming languages, whether you want to choose C over
another high-level language is a matter of opinion and both technical
and business requirements could dictate which language is required.


# C Programming/History

The field of computing as we know it today started in 1947 with three
scientists at Bell Telephone Laboratories---William
Shockley, Walter
Brattain, and John
Bardeen---and their groundbreaking
invention: the transistor. In 1956, the first
fully transistor-based computer, the TX-0, was
completed at MIT. The first integrated
circuit was created in 1958 by Jack
Kilby at Texas Instruments, but the first
high-level programming language existed even before then.

The Fortran project was developed in 1954 by
IBM. A shortening of \"*The IBM Mathematical **For**mula **Tran**slating
System*\", the project had the purpose of creating and fostering
development of a procedural, imperative programming language that was
especially suited to numeric computation and scientific computing. It
was a breakthrough in terms of productivity and programming ease
(compared to assembly language) and
speed (Fortran programs ran nearly as fast as, and in some cases, just
as fast as, programs written in assembly). Furthermore, Fortran was
written at a high-enough level (and thus was machine independent enough)
to become the first widely adopted programming language. The Algorithmic
Language (Algol 58) was derived from Fortran in
1958 and evolved into Algol 60 in 1960. The
Combined Programming Language
(CPL) was then created out
of Algol 60 in 1963. In 1967, it evolved into Basic
CPL (BCPL), which was the basis for
*B* "wikilink"), which was created in 1971,
and served as the basis of *C*.

Created by Ken Thompson at Bell Labs, B was
a stripped-down version of BCPL that was also a compiled
language (see User\'s Reference to
B) used in early
internal versions of the UNIX operating system. As
Dennis Ritchie noted in his *Development
of the C Language* :

> The B compiler on the PDP-7 did not generate machine instructions, but
> instead \'threaded code\', an interpretive scheme in which the
> compiler\'s output consists of a sequence of addresses of code
> fragments that perform the elementary operations. The operations
> typically --- in particular for B --- act on a simple stack machine.

Thompson and Ritchie improved B, and called the result NB. Further
extensions to NB created its logical successor, C. Most of UNIX was
rewritten in NB, and then C, which resulted in a more portable operating
system. The portability of UNIX was
the main reason for the initial popularity of both UNIX and C. Rather
than creating a new operating system for each new machine, system
programmers could simply write the few system-dependent parts required
for the machine, and then write a C compiler for the new system. Since
most of the system utilities were thus written in C, it simply made
sense to also write new utilities in C.

The American National Standards Institute began work on standardizing
the C language in 1983, and completed the standard in 1989. The
standard, ANSI X3.159-1989 \"Programming Language C\", served as the
basis for all implementations of C compilers. The standards were later
updated in 1990 and 1999, allowing for features that were either in
common use, or were appearing in C++.

de:C-Programmierung:
Grundlagen es:Programación
en C/Historia de C
fr:Programmation
C/Introduction
it:C/Linguaggio/Panoramica
pt:Programar em C/História da linguagem
C


# C Programming/What you need before you can learn

## Getting Started

This book introduces and teaches the basics of the C programming
language and touches upon some advanced topics as well. This section
outlines the required skills and tools you\'ll need to get the most out
of this book.

### Skills and Prior Experience You\'ll Need

This book is for beginning programmers, so don\'t worry if you have no
formal computer training or prior programming experience. It\'s assumed
you know how to turn your computer on, start and stop applications, and
perform other basic operations like installing software. It\'s also
assumed you have some experience interacting with your operating system
through a terminal window using its **command line interface.** If you
aren\'t sure what this means, consider seeking out a tutorial for your
chosen platform that can get you comfortable with getting around your
computer\'s command line. At a minimum, you should know the basic
commands for navigating to different directories and performing simple
file management operations. This book will spell out any other commands
you\'ll need to run from the command line to get your C code working on
your machine.

### Software You\'ll Need

No one ever became a musician just by reading sheet music. Musicians
have to constantly play and practice on their instruments to get good.
Similarly, the only way to become a programmer is to write and execute
lots of code. To do that, you will need two different pieces of
software: a **compiler** and a **text editor**. Both can be had for no
cost.

###### Compilers

A compiler is a sophisticated piece of software for converting the C
source code you write with your text editor into the machine
code[^1] that you can execute on your
computer. Below is a list of some popular C compilers. Note that some of
the compilers listed below come as part of an **integrated development
environment
(IDE).** However, if you are brand new to programming, it\'s best if you
can install and run the compiler from the command line instead of
through an IDE. This book uses the GNU C Compiler (GCC) in its examples
so we recommend installing this compiler for use with this book. The
next section in this chapter will explain how to download and install
the GCC software to your machine.

**Popular C compilers/IDEs include:**

  Name                                                                                Website                                                       Platform                                                                                License                                                                              Details
  ------------------------------------------------------------------------------------- ------------------------------------------------------------- --------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------------------
  Microsoft Visual Studio Community   Visual Studio   Windows                                                                                Proprietary, free of charge                                                          Powerful and student-friendly version of an industry standard compiler.
  Xcode                                                   Xcode                                                         macOS, OSX                                                                                Proprietary, free of charge                                                          Available free of charge at Mac App Store.
  Tiny C Compiler (TCC)                                 tinycc                            GNU/Linux, Windows                                                                                LGPL                               Small, fast and simple compiler.
  Clang                                                           clang                               GNU/Linux, Windows, Unix, OS X                                                                      University of Illinois/NCSA License   A free, permissively licensed front-end using a LLVM backend.
  GNU C Compiler                                gcc                                    GNU/Linux, MinGW or mingw-w64 (Windows), Unix, OS X.   GPL                                       The De facto standard. Ships with most Unix-like systems.

###### Text Editors and IDEs

Aside from a compiler, the only other software requirement is a text
editor for writing and saving your C code.
Note that a text editor is different from a word
processor, a piece of software with many
features for creating visually appealing documents. Unlike word
processors, text editors are primarily designed to create plain text
files. On Windows, the Notepad text editor can be used but it does not
offer any advanced capabilities such as syntax highlighting and code
completion. There are hundreds of text editors (see List of Text
Editors). Among the most popular are
Notepad++ for Windows as well as Sublime
Text, gedit,
Vim "wikilink") and Emacs
which are also available on other operating systems ("cross-platform").
These text editors come with syntax
highlighting and line numbers, which
makes code easier to read at a glance, and to spot syntax errors. Many
text editors have features for increasing your coding speed, such as
keystroke macros and code snippets, that you can take advantage of as
you gain skill as a programmer.

You may also be considering the use of an **integrated Development
Environment** (**IDE**) to help you write code. An IDE is a suite of
integrated tools and features in one convenient package, usually with a
graphical user interface. These programs include a text editor and file
browser and are also sometimes bundled with an easily accessible
compiler. They also typically include a debugger, a tool that will
enable you to do such things as step through the program you develop
manually one source code line at a time, or alter data as an aid to
finding and correcting programming errors.

However, many IDEs do not offer a command line interface to the compiler
and/or offer only graphical buttons or a menu for executing programs. So
for new programmers, an IDE is not ideal. Instead, a simple text editor
will suffice along with the ability to issue simple commands on the
command line to help you gain a hands-on familiarity and understanding
of core development tools. Of course, an IDE may still be useful to you
if you have experience with one. But as a general guideline: Do not use
an IDE unless you know what the IDE is doing for you!

**Other popular compilers/IDEs include:**

  Name                                                     Website                                                     Platform                       License                                                                                Details
  -------------------------------------------------------- ----------------------------------------------------------- ------------------------------ ------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------
  Eclipse CDT "wikilink")           Eclipse   Windows, Mac OS X, GNU/Linux   Free/Libre and Open Source                                                                                Eclipse "wikilink") IDE for C/C++ development, a popular open source IDE.
  Netbeans                        Netbeans                            Cross-platform                 CDDL and GPL 2.0   A Good comparable matured IDE to Eclipse.
  GNOME Builder              Builder              GNU/Linux                      GPL                                                                            A feature-rich but simple IDE for the GNOME desktop environment.
  Anjuta                            Anjuta                                 GNU/Linux                      GPL                                                                            An extensible GTK+3 IDE for the GNOME desktop environment.
  Geany                              geany                               Cross-platform                 GPL                                                                            A lightweight cross-platform GTK+ notepad based on Scintilla, with basic IDE features.
  KDevelop                        KDevelop                       Cross-platform                 GPL                                                                            A cross-platform IDE for the KDE project.
  Little C Compiler (LCC) "wikilink")   lcc               Windows                        Open Source but not Libre                                                                                Small open source compiler.
  Pelles C                        Pelles C                Windows, Pocket PC             Proprietary, free of charge                                                                                A complete C development kit for Windows.
   Dev-C++                         Dev C++   Windows                        GPL                                                                            Updated version of the formerly popular Bloodshed Dev-C++.
  CodeLite                        CodeLite                           Cross-platform                 GPL 2                                                                          Free IDE for C/C++ development.
  Code::Blocks                Code::Blocks                      Cross-platform                 GPL 3.0                                                                        Built to meet users\' most demanding needs. Very extensible and fully configurable.

On **GNU/Linux**, GCC is almost always included by default.

On **Microsoft Windows**, Dev-C++ is recommended for beginners because
it is easy to use, free, and simple to install. Although the initial
developer (Bloodshed) hasn't updated it since 2005, a new version
appeared in 2011, made by an independent programmer, and is being
actively developed.[^2] An alternate option for those working only in
the Windows environment is the proprietary Microsoft Visual Studio
Community which is free of charge and has an excellent debugger.

On **Mac OS X**, the Xcode IDE provides the compilers needed to compile
various source files. The newer versions do not include the command line
tools. They need to be downloaded via Xcode-\>Preferences-\>Downloads.

## Footnotes

pl:C/Czego potrzebujesz

[^1]: Actually, GCC's (GNU C Compiler) **cc** (C Compiler) translates
    the input .c file to the target CPU's
    assembly, output is written to an
    .s file. Then **as** (assembler) generates a machine code file from
    the .s file. Pre-processing is done by another sub-program **cpp**
    (C PreProcessor), which is not to be confused with **c++** (a
    compiler for another programming language).

[^2]: <http://orwelldevcpp.blogspot.com/>


# C Programming/Obtaining a compiler

## Dev-C++

{{ Wikipedia \| Dev-C++ }}

Dev C++ is an Integrated Development
Environment (IDE) for the C++ programming language, available from
Bloodshed Software. An updated version is
available at Orwell Dev-C++.\
C++ is a programming language which contains within itself most of the C
language, plus extensions. Most C++ compilers will compile C programs,
sometimes with a few adjustments (like invoking them with a different
name or command line switch). Therefore, you can use Dev C++ for C
development.

However, Dev C++ is not the compiler. It is designed to use the
MinGW or Cygwin versions of
GCC - both of which can be obtained as part
of the Dev C++ package, although they are completely different
projects.\
Dev C++ simply provides an editor, syntax highlighting, some facilities
for the visualisation of code (like class and package browsing) and a
graphical interface to the chosen compiler. Because Dev C++ analyses the
error messages produced by the compiler and attempts to distinguish the
line numbers from the errors themselves, the use of other compiler
software is discouraged since the format of their error messages is
likely to be different.

The latest version of Dev-C++ is a
beta for version 5. However, it
still has a significant number of bugs. All the features are there, and
it is quite usable. It is considered one of the best free software C
IDEs available for Windows.

A version of Dev C++ for Linux is in the pipeline. It is not quite
usable yet, however. Linux users already have a wealth of IDEs
available. (e.g. KDevelop and
Anjuta.) Most of the graphical text editors, and
other common editors such as *emacs* and *vim*, support syntax
highlighting.

### Windows

1.  Go to <https://sourceforge.net/projects/orwelldevcpp/> and pick the
    download option.
2.  The setup is pretty straight forward. Make sure the compiler option
    is ticked.
3.  You can now use the environment provided by the software to write
    and run your code.
4.  OPTIONALLY: \"C:\\Program Files (x86)\\Dev-Cpp\\MinGW64\\bin\" can
    be added to the global PATH variable of the operating system to
    compile with GCC from a command prompt.

## GCC

The GNU Compiler Collection
(GCC) is a free/libre set of compilers
developed by the Free Software
Foundation and can be installed
on a wide variety of operating systems. GCC commands are used throughout
this book to demonstrate how to compile C code so you are encouraged to
take the time to install GCC on your machine.

### GNU/Linux

On **GNU/Linux,** Installing the GNU C Compiler can vary in method from
distribution to distribution. (Type
in **cc -v** to see if it is installed already.)

-   For Ubuntu, install the GCC compiler (along
    with other necessary tools) by running
    `sudo ``apt`` install build-essential`
    in the terminal.
-   For Debian, install the GCC compiler (as
    root) by running
    `apt`` install gcc` in the
    terminal.
-   For Fedora "wikilink"), install the
    GCC compiler (as root) by running
    `dnf` "wikilink")` install gcc` in the terminal.
-   For RHEL, install the GCC
    compiler (as root) by running
    `dnf` "wikilink")` install gcc` in the terminal.
-   For Mandrake, install the GCC compiler (as
    root) by running `urpmi`` gcc` in the
    terminal.
-   For Slackware, the package is available on
    their website - simply download, and
    type `installpkg gcc-xxxxx.tgz` in the terminal.
-   For Gentoo, you should already have GCC
    installed as it will have been used when you first installed. To
    update it run (as root) `emerge -uav gcc` in the terminal.
-   For Arch Linux, install the GCC compiler
    (as root) by running `pacman -S gcc` in the terminal.
-   For  Void Linux, install the GCC
    compiler (as root) by running `xbps-install -S gcc` in the terminal.
-   If you cannot become root, get the GCC tarball from
    <ftp://ftp.gnu.org/> and follow the instructions in it to compile
    and install in your home directory. Be warned though, you need a C
    compiler to do that - yes, GCC itself is written in C.
-   You can use a commercial C compiler/IDE.

### macOS

The simplest method for obtaining a compiler is to install Apple\'s
proprietary IDE, Xcode, available for
free.

Xcode comes bundled with a GCC-compatible compiler called
clang which replaced GCC as Xcode\'s default C
compiler a number of years ago. But because Xcode aliases the `gcc`
command to the clang compiler, GCC installation isn\'t necessary to
compile the example code in this book.

If you prefer using the GCC compiler, the third-party package manager,
Homebrew, provides an easy installation process.
You\'ll first need to install
Homebrew, and then issue the
`brew install` command to install the desired GCC Homebrew
formulae. You may want to find a
recent tutorial that will step you through this process as other
commands may be necessary to get GCC set up flawlessly on your system,
especially if you already have Xcode installed.

For hardcore computer enthusiasts, GCC can be compiled directly from the
source code. We highly recommend searching out and following an
up-to-date tutorial for installing GCC from source files.

### BSD Family Systems

-   For FreeBSD, NetBSD,
    OpenBSD, DragonFly
    BSD the port of GNU GCC is available in
    the base system, or it could be obtained using the ports collection
    or pkgsrc.

### Windows

There are three ways to use GCC on Windows: Cygwin, MinGW and Windows
Subsystem for Linux (WSL). Applications compiled with Cygwin will not
run on any computer without Cygwin, so MinGW is recommended. MinGW is
simpler to install, and takes less disk space.

#### MinGW

1.  Go to <http://sourceforge.net/projects/mingw/> download and save
    this to your hard drive.
2.  Once the download is finished, open it and follow the instructions.
    You can also choose to install additional compilers, or the tool
    Make, but these aren\'t necessary.
3.  Now you need to set your PATH. Right-click on \"My computer\" and
    click \"Properties\". Go to the \"Advanced\" tab and click on
    \"Environment variables\". Go to the \"System variables\" section
    and scroll down until you see \"Path\". Click on it, then click
    \"edit\". Add \"C:\\MinGW\\bin\\\" (without the quotes) to the end.
4.  To test if GCC works, open a command prompt and type \"gcc\". You
    should get the message \"gcc: fatal error: no input files
    compilation terminated.\". If you get this message, GCC is installed
    correctly.

#### Cygwin

1.  Go to <http://www.cygwin.com> and click on the \"Install Cygwin
    Now\" button in the upper right corner of the page.
2.  Click \"run\" in the window that pops up, and click \"next\" several
    times, accepting all the default settings.
3.  Choose any of the Download sites (\"ftp.easynet.be\", etc.) when
    that window comes up; press \"next\" and the Cygwin installer should
    start downloading.
4.  When the \"Select Packages\" window appears, scroll down to the
    heading \"Devel\" and click on the \"+\" by it. In the list of
    packages that now displays, scroll down and find the \"gcc-core\"
    package; this is the compiler. Click once on the word \"Skip\", and
    it should change to some number like \"3.4\" etc. (the version
    number), and an \"X\" will appear next to \"gcc-core\" and several
    other related packages that will now be downloaded.
5.  Click \"next\" and the compiler as well as the Cygwin tools should
    start downloading; this could take a while. While you\'re waiting
    for the installation to finish, download any text-editor designed
    for programming. While Cygwin does include some, you may prefer
    doing a web search to find other alternatives. While using a stock
    text editor is possible, it is not ideal.
6.  Once the Cygwin downloads are finished and you have clicked
    \"next\", etc. to finish the installation, double-click the Cygwin
    icon on your desktop to begin the Cygwin \"command prompt\". Your
    home directory will automatically be set up in the Cygwin folder,
    which now should be at \"C:\\cygwin\" (the Cygwin folder is in some
    ways like a small unix/linux computer on your Windows machine \--
    not technically of course, but it may be helpful to think of it that
    way).
7.  Type \"gcc\" at the Cygwin prompt and press \"enter\"; if \"gcc: no
    input files\" or something like it appears you have succeeded and
    now have the GCC compiler on your computer (and congratulations \--
    you have also just received your first error message!).

#### Windows Subsystem for Linux

1.  Go to <http://aka.ms/wsldocs> and follow the steps to install
    WSL
2.  Go to <https://aka.ms/vscode> and follow the steps to install
    VSCode
3.  Follow the
    guide
    and choose Get Started with C++ and
    WSL
4.  As a result you will need to install possibly
    Ubuntu
    and set-up accordingly installing GCC like the Linux guide above.

The current stable (usable) version of GCC is 4.9.1 published on
2014-07-16, which supports several platforms. In fact, GCC is not only a
C compiler, but a family of compilers for several languages, such as
C++, Ada, Java, and
Fortran.

## Embedded systems

-   Most CPUs are microcontrollers in embedded systems, often programmed
    in C, but most of the compilers mentioned above (except GCC) do not
    support such CPUs. For specialized compilers that do support
    embedded systems, see Embedded Systems/C
    Programming.

## Other C compilers

We have a long  list of C
compilers in a much later section
of this Wikibook. *Which of those compilers would be suitable for
beginning C programmers, that we should say a few words about getting
started with that particular compiler in this section of this Wikibook?*

pl:C/Używanie kompilatora