NeuralNetworkPY 0.0.2
An easy-to-use Neural Network library I'm building from scratch.

# NeuralNetworkPY
A basic, easy-to-use, neural network library built from scratch in python. Only dependency is numpy.

## Examples
I find examples are what I want when I go to a readme so I'm going to start with it. References for everything in this package can be found below.

### Creating a Network to Understand XOR
```python
    #Initialize the network with 2 inputs, 1 hidden layer with 5 neurons, 2 outputs
    nn = NeuralNetwork(2, [5], 2) 

    #Setting up the training data
    data = []
    data.append(np.array([0, 1]))
    data.append(np.array([1, 0]))
    data.append(np.array([1, 1]))
    data.append(np.array([0, 0]))
    out = [[0, 1], [0, 1], [1, 0], [1, 0]] #The expected output for every input added above 
    #(i.e. 0 xor 1 = 1  which in array form is [0, 1] because the guess function returns the index of the highest value, 1 in this case)

    #Train the network
    import random as rand
    for i in range(50000):
        index = rand.randint(0, 3) #randomizes which value we train with on each iteration
        nn.train(data[index], out[index])

    #Print the guess for every input
    print('')
    for ar in data:
        print(str(ar[0]) + " xor " + str(ar[1]) + " = " + str(nn.guess(ar)))
```

### NeuralNetwork and MNIST Example ("The hello world! of machine learning")
```python
#This may take a long time to run, 2-10 minutes, depending on the speed of your cpu
#I know it isn't commented the best, I've been writing documentation for hours at this point lol

import NeuralNetworkPY
import numpy as np
import struct
import random as rand

#Data obtained from http://yann.lecun.com/exdb/mnist/. These files MUST be in same directory

#Function obtained from https://gist.github.com/tylerneylon/ce60e8a06e7506ac45788443f7269e40
def read_idx(filename):
    with open(filename, 'rb') as f:
        zero, data_type, dims = struct.unpack('>HBB', f.read(4))
        shape = tuple(struct.unpack('>I', f.read(4))[0] for d in range(dims))
        return np.frombuffer(f.read(), dtype=np.uint8).reshape(shape)

def prepImgs(raw_images):
    images = np.array([None for i in range(len(raw_images))])
    for i in range(len(raw_images)):
        images[i] = raw_images[i].flatten()
        images[i] = images[i] / 255
    return images

def createOutput(val):
    out = []
    for i in range(10):
        if (i == val):
            out.append(1.0)
        else:
            out.append(0.0)
    return np.array([out]).transpose()

def guessToNum(guess):
    max = guess[0]
    index = 0
    for i in range(len(guess)-1):
        if (guess[i+1] > max):
            max = guess[i+1]
            index = i+1
    return index

#Load images and labels from files
raw_images = read_idx('train-images.idx3-ubyte')
labels = read_idx('train-labels.idx1-ubyte')

#Prep the images into a big array (need to normalize all values and flatten the 2D array to a 1D)
images = prepImgs(raw_images)

#Setup network with 784 inputs (the images are 28*28 = 784), one hidden layer with 50 neurons, and 10 outputs (there are 10 possible values)
digitRecog = NeuralNetworkPY.NeuralNetwork(784, [50], 10)
digitRecog.setLearningRate(0.05)

for n in range(10): #train the network in 10 epochs
    for i in range(len(images)): #in each epoch, train the network with a random image n times where n is the number of images
        index = rand.randint(0, len(images)-1) #choose random image
        digitRecog.train(images[index], createOutput(labels[index])) #train the network for one iteration

#get and setup testing data
test_raw_images = read_idx('t10k-images.idx3-ubyte')
test_images = prepImgs(test_raw_images)
test_labels = read_idx('t10k-labels.idx1-ubyte')
correct = 0
total = len(test_images)

#loop through all test images, count how many the network got right
for i in range(len(test_images)):
    guess = digitRecog.feedForward(test_images[i])
    if (guessToNum(guess) == test_labels[i]):
        correct = correct + 1

#print results
print("The Network guessed " + str(correct) + "/" + str(total) + " (" + str(100*correct/total) + "%) correct")
```


## Object References
### NeuralNetwork:
  - Description: The NeuralNetwork object which is what is trained and where inputs are passed in. This is what you'll want
  - Functions:
    - `init(inputNum, hiddenNum, outputNum)` (the constructor):
      - Description: Creates a NeuralNetwork object with the structure given as inputs
      - Inputs:
        - `inputNum`
          - Description: The number of inputs to the network (size of input array)
          - Type Required: int
        - `hiddenNum`
          - Description: The size of every hidden layer in the network
          - Type Required: list of int
        - `outputNum`
          - Description: The number of outputs from the network (size of output array)
          - Type Required: int
    - `feedForward(inputData)`:
      - Description: Computes the feed forwards algorithm for the network. Will propagate a given input array through every layer to produce an output array.
      - Inputs:
        - `inputData`
          - Description: The input array to be propogated through the network.
          - Type Required: 1D python list or numpy array (such as [1, 2, 3])
      - Outputs:
        - `probabilities`
          - Description: The confidence values for every possible output.
          - Type: Column vector in a numpy array. Will have shape (n, 1) where n is the number of elements.
    - `train(inputData, expectedOutput)`:
      - Description: Processes one iteration of network training, will adjust the weights and bias based on the error between the expected output and the current output for a given input.
      - Inputs:
        - `inputData`
          - Description: The input array to be trained with
          - Type Required: 1D python list or numpy array (such as [1, 2, 3])
        - `expectedOutput`
          - Description: What the given input array should produce in the network
          - Type Required: 1D python list or numpy array (such as [1, 2, 3])
    - `guess(inputData)`:
      - Description: Produces a guess using forward propagation for a given input. The guess is the highest confidence value in the output array from feedForward()
      - Inputs:
        - `inputData`
          - Description: The input array to get a guess from
          - Type Required: 1D python list or numpy array (such as [1, 2, 3])
      - Outputs:
        - `output`
          - Description: The index of the highest confidence value in the feedForward output.
          - Type: int
    - `setLearningRate(newRate)`:
      - Description: Sets the learning rate of the network
      - Inputs:
        - `newRate`
          - Description: The new learning rate
          - Type Required: float
    - `getLearningRate()`:
      - Description: Gets the learning rate of the network
      - Outputs:
        - `output`
          - Description: The learning rate of the current network.
          - Type: float
    - `save(fileName)`:
      - Description: Saves the network to a text file for later use relative to the current directory. Format is provided below.
      - Inputs:
        - `fileName`
          - Description: The name of the file (ex. "test.txt").
          - Type Required: str
    - `setLayers(weights, biases)`:
      - Description: Sets the given network based on a list of weights and biases.
      - Inputs:
        - `weights`
          - Description: A python list (must be the same size as self.layers) containing the new weight matricies for each layer.
          - Type Required: Python list of 2D numpy arrays
        - `biases`
          - Description: A python list (must be the same size as self.layers) containing the new bias matricies for each layer.
          - Type Required: Python list of 2D numpy arrays

### Layer:
  - Description: Used by the NeuralNetwork object for every layer (the inputs not being a layer). Contains information for every neuron in the layer which includes the weight connections and biases.
  - Functions:
    - `init(inputSize, selfSize)` (the constructor):
      - Description: Creates a Layer object with the structure as given by parameters
      - Inputs:
        - `inputSize`
          - Description: The number of inputs from the previous layer. Could also be thought of as the number of neurons on the previous layer.
          - Type Required: int
        - `selfSize`
          - Description: The number of neurons on the current layer.
          - Type Required: int
    - `compute(inputData)`:
      - Description: Propagates a given array of inputs through the current layer. This involves computing the necessary weighted sums for every neuron and adding the bias. Matrix math is used.
      - Inputs:
        - `inputData`
          - Description: The input array to propagate through this layer
          - Type Required: A column vector of inputs between -1 and 1. The shape of the numpy array is (n, 1) where n is the number of elements.
      - Outputs:
        - `out`
          - Description: The output from the current layer with the given input.
          - Type: A column vector of inputs between -1 and 1. The shape of the numpy array is (n, 1) where n is the number of elements.
    - `stringify()`:
      - Description: Gives a string representation of the layer. Used for saving the network to a text file.
      - Outputs:
        - `weightsStr`
          - Description: The string of the weights matrix, stripped of newline characters.
          - Type: str
        - `biasStr`
          - Description: The string of the bias matrix, stripped of newline characters.
          - Type: str
    - `setVals(weights, bias)`:
      - Description: Sets the weights and bias of the current layer to the given inputs.
      - Inputs:
        - `weights`
          - Description: A matrix of the weights, needs to match the size of the current one.
          - Type Required: 2D numpy array
        - `bias`
          - Description: A matrix of the bias, needs to match the size of the current one.
          - Type Required: 2D numpy array

## Helper Functions
### `checkReform(data)`
  - Description: Checks an input data list or array, ensures it is a column vector, reshapes it if neccessary, returns the clean output.
  - Inputs:
    - `data`
      - Description: A list or array that is going to be propagated through the array.
      - Type Required: python list or numpy array
  - Outputs:
    - `out`
      - Description: The cleaned input, is ready for use by a NeuralNetwork object
      - Type: A column vector with the shape of the numpy array being (n, 1) where n is the number of elements.

### `writeLine(file, text)`
  - Description: Writes a line to an open file object with write permissions.
  - Inputs:
    - `file`
      - Description: The file object to write to, must have write permissions.
      - Type Required: _io.TextIOWrapper object
    - `text`
      - Description: The text to write to the line
      - Type Required: str

### `readLine(file)`
  - Description: Reads the next line from an open file object with read permissions evaluated as a python variable.
  - Inputs:
    - `file`
      - Description: The file object to read from, must have read permissions.
      - Type Required: _io.TextIOWrapper object
  - Outputs:
    - `output`
      - Description: Evaluates the line recieved as a python value. (i.e. the string "array([1, 2, 3])" would evaluate as a numpy array containing [1, 2, 3])
      - Type: depends on the string evaluated

### `loadNetwork(fileName)`
  - Description: Reads a network info file with the given name and recreates a Neural Network out of it.
  - Inputs:
    - `fileName`
      - Description: The name of the saved Neural Network data file. (i.e. "test.txt")
      - Type Required: str
  - Outputs:
    - `out`
      - Description: A Neural Network with the same specifications as outlined by the file data.
      - Type: NeuralNetwork object


## Save File Format
**line**    **value**
1. inputNum 
2. hiddenNum
3. outputNum
4. learningRate 
5. number of layers (layerNum) 
6. layer[0] Weights 
7. layer[0] Bias 

......... 

(5+2xlayerNum-1). layer[layerNum-1] Weights 

(5+2xlayerNum). layer[layerNum-1] Bias 


## Dev Notes
I am currently working on building a Convolutional Neural Network from scratch modelled after the Neural Network created here. The code that is being worked on can be found here as well.