I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Attributes & Methods
Classes contain attributes (also called fields, members etc...) and methods (a.k.a functions). Attributes defines the characteristics of the object and methods perfom action on the object. For example, the class definition below has firstname and lastname attributes and fullname is a method.
Can you write Python code for it?

class Person:
    '''Simple Person Class

       Attributes:
           firstname: String representing first name of the person
           lastname: String representing last name of the person
    '''
    def __init__(self,firstname,lastname):
        '''Initialiser method for Person'''
        self.firstname = firstname
        self.lastname = lastname

    def fullname(self):
        '''Returns the full name of the person'''
        return self.firstname + ' ' + self.lastname


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Built In Class Attributes
Can you write Python code for it?

class Employee:
   'Common base class for all employees'
   empCount = 0

   def __init__(self, name, salary):
      self.name = name
      self.salary = salary
      Employee.empCount += 1

   def displayCount(self):
     print("Total Employee", Employee.empCount)

   def displayEmployee(self):
      print("Name : ", self.name,  ", Salary: ", self.salary)

print ("Employee.__doc__:", Employee.__doc__)
print ("Employee.__name__:", Employee.__name__)
print ("Employee.__module__:", Employee.__module__)
print ("Employee.__bases__:", Employee.__bases__)
print ("Employee.__dict__:", Employee.__dict__)


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Class Inheritance
Inheritance is a way of arranging objects in a hierarchy from the most general to the most specific. An object which inherits from another object is considered to be a subtype of that object.
We also often say that a class is a subclass or child class of a class from which it inherits, or that the other class is its superclass or parent class. We can refer to the most generic class at the base of a hierarchy as a base class.
Inheritance is also a way of reusing existing code easily. If we already have a class which does almost what we want, we can create a subclass in which we partially override some of its behaviour, or perhaps add some new functionality.
Can you write Python code for it?

# Simple Example of Inheritance

class Person:
    pass

# Parent class must be defined inside the paranthesis

class Employee(Person):
    pass

e1 = Employee()
print(dir(e1))

class Person:

    def __init__(self,firstname,lastname):
        self.firstname = firstname
        self.lastname = lastname

    def __str__(self):
        return "[{},{}]".format(self.firstname,self.lastname)

class Employee(Person):
    pass

john = Employee('John','Doe')
print(john)

class Person:

    def __init__(self, firstname, lastname):
        self.firstname = firstname
        self.lastname = lastname

    def __str__(self):
        return "{},{}".format(self.firstname, self.lastname)

class Employee(Person):

    def __init__(self, firstname, lastname, staffid):
        super().__init__(firstname, lastname)
        self.staffid = staffid

    def __str__(self):
        return super().__str__() + ",{}".format(self.staffid)

john = Employee('Jack','Doe','12345')
print(john)


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Create Class Using Key Value Arguments
Can you write Python code for it?

class Student:
    def __init__(self, **kwargs):
        for k,v in kwargs.items():
            setattr(self,k,v,)

    def __str__(self):
        attrs = ["{}={}".format(k, v) for (k, v) in self.__dict__.items()]
        return str(attrs)
        #classname = self.__class__.__name__
        #return "{}: {}".format((classname, " ".join(attrs)))

s1 = Student(firstname="John",lastname="Doe")
print(s1.firstname)
print(s1.lastname)
print(s1)

def print_values(**kwargs):
    for key, value in kwargs.items():
        print("The value of {} is {}".format(key, value))

print_values(my_name="Sammy", your_name="Casey")


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Here the class statement did not create anything, it just the blueprint to create "Person" objects. To create an object we need to instantiate the "Person" class.
Can you write Python code for it?


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Inside the class body, we define two functions – these are our object’s methods. The first is called _init_, which is a special method. When we call the class object, a new instance of the class is created, and the _init_ method on this new object is immediately executed with all the parameters that we passed to the class object. The purpose of this method is thus to set up a new object using data that we have provided.
The second method is a custom method which derives the fullname of the person using the firstname and the lastname.

_init_ is sometimes called the object’s constructor, because it is used similarly to the way that constructors are used in other languages, but that is not technically correct – it’s better to call it the initialiser. There is a different method called _new_ which is more analogous to a constructor, but it is hardly ever used.

You may have noticed that both of these method definitions have self as the first parameter, and we use this variable inside the method bodies – but we don’t appear to pass this parameter in. This is because whenever we call a method on an object, the object itself is automatically passed in as the first parameter (as self). This gives us a way to access the object’s properties from inside the object’s methods.
Instance Attributes
All the attributes that are defined on the Person instance are called instance attributes. They are added to the instance when the _init_ method is executed.
Class Attributes
We can, however, also define attributes which are set on the class. These attributes will be shared by all instances of that class. In many ways they behave just like instance attributes, but there are some caveats that you should be aware of.
We define class attributes in the body of a class, at the same indentation level as method definitions (one level up from the insides of methods)
Can you write Python code for it?

class Person:
    '''Simple Person Class

       Attributes:
           firstname: String representing first name of the person
           lastname: String representing last name of the person
    '''
    TITLES = ['Mr','Mrs','Master']

    def __init__(self,title,firstname,lastname):
        '''Initialiser method for Person'''
        if title not in self.TITLES:
            raise ValueError("%s is not a valid title.", title)
        self.firstname = firstname
        self.lastname = lastname

    def fullname(self):
        '''Returns the full name of the person'''
        return self.firstname + ' ' + self.lastname

John = Person('Mister','John','Doe') # this will create an error

class Employee:
   '''Common base class for all employees'''

   empCount = 0

   def __init__(self, name, salary):
      self.name = name
      self.salary = salary
      Employee.empCount += 1

   def displayCount(self):
     print("Total Employee %d",Employee.empCount)

   def displayEmployee(self):
      print("Name : ", self.name,  ", Salary: ", self.salary)

"This would create first object of Employee class"
emp1 = Employee("Zara", 2000)
"This would create second object of Employee class"
emp2 = Employee("Manni", 5000)
emp1.displayEmployee()
emp2.displayEmployee()
print("Total Employee ", Employee.empCount)


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Inspecting an Object
Can you write Python code for it?

class Person:
    def __init__(self, name, surname):
        self.name = name
        self.surname = surname

    def fullname(self):
        return "%s %s" % (self.name, self.surname)

jane = Person("Jane", "Smith")

print(dir(jane))


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Mixins
If we use multiple inheritance, it is often a good idea for us to design our classes in a way which avoids the kind of ambiguity described above. One way of doing this is to split up optional functionality into mix-ins. A Mix-in is a class which is not intended to stand on its own – it exists to add extra functionality to another class through multiple inheritance.
Can you write Python code for it?

class Person:
    def __init__(self, name, surname, number):
        self.name = name
        self.surname = surname
        self.number = number

class LearnerMixin:
    def __init__(self):
        self.classes = []

    def enrol(self, course):
        self.classes.append(course)

class TeacherMixin:
    def __init__(self):
        self.courses_taught = []

    def assign_teaching(self, course):
        self.courses_taught.append(course)

class Tutor(Person, LearnerMixin, TeacherMixin):
    def __init__(self, *args, **kwargs):
        super(Tutor, self).__init__(*args, **kwargs)

jane = Tutor("Jane", "Smith", "SMTJNX045")
#jane.enrol(a_postgrad_course)
#jane.assign_teaching(an_undergrad_course)


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Multiple Inheritance
Can you write Python code for it?

class Person:
    pass

class Company:
    pass

class Employee(Person,Company):
    pass

print(Employee.mro())


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Now we created a "Person" object and assigned it to "P1". We can create any number of objects but please note there will be only one "Person" class.
Can you write Python code for it?

# Doc string for class
class Person:
    '''Simple Person Class'''
    pass

print(Person.__doc__)


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
Static Methods
A static method doesn’t have the calling object passed into it as the first parameter. This means that it doesn’t have access to the rest of the class or instance at all. We can call them from an instance or a class object, but they are most commonly called from class objects, like class methods.
If we are using a class to group together related methods which don’t need to access each other or any other data on the class, we may want to use this technique.
The advantage of using static methods is that we eliminate unnecessary cls or self parameters from our method definitions.
The disadvantage is that if we do occasionally want to refer to another class method or attribute inside a static method we have to write the class name out in full, which can be much more verbose than using the cls variable which is available to us inside a class method.
Can you write Python code for it?

class ClassGrades:

    def __init__(self, grades):
        self.grades = grades

    @classmethod
    def from_csv(cls, grade_csv_str):
        grades = grade_csv_str.split(', ')
        cls.validate(grades)
        return cls(grades)

    @staticmethod
    def validate(grades):
        for g in grades:
            if int(g) < 0 or int(g) > 100:
                raise Exception()

try:
    # Try out some valid grades
    class_grades_valid = ClassGrades.from_csv('90, 80, 85, 94, 70')
    print('Got grades:', class_grades_valid.grades)

    # Should fail with invalid grades
    class_grades_invalid = ClassGrades.from_csv('92, -15, 99, 101, 77, 65, 100')
    print(class_grades_invalid.grades)
except:
    print('Invalid!')


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
The difference between a static method and a class method is:

Static method knows nothing about the class and just deals with the parameters.
Class method works with the class since its parameter is always the class itself.

Property
Sometimes we use a method to generate a property of an object dynamically, calculating it from the object’s other properties. Sometimes you can simply use a method to access a single attribute and return it. You can also use a different method to update the value of the attribute instead of accessing it directly. Methods like this are called getters and setters, because they “get” and “set” the values of attributes, respectively.
The @property decorator lets us make a method behave like an attribute.
Can you write Python code for it?

class Person:
    '''Simple Person Class

       Attributes:
           firstname: String representing first name of the person
           lastname: String representing last name of the person
    '''
    def __init__(self,firstname,lastname):
        '''Initialiser method for Person'''
        self.firstname = firstname
        self.lastname = lastname

    @property
    def fullname(self):
        '''Returns the full name of the person'''
        return self.firstname + ' ' + self.lastname

p1 = Person('John','Doe')
print(p1.fullname)


I am working on the file "Classes+and+Objects.ipynb".
The first task is:
There are also decorators which we can use to define a setter and a deleter for our attribute (a deleter will delete the attribute from our object). The getter, setter and deleter methods must all have the same name
Can you write Python code for it?

class Person:
    '''Simple Person Class

       Attributes:
           firstname: String representing first name of the person
           lastname: String representing last name of the person
    '''
    def __init__(self,firstname,lastname):
        '''Initialiser method for Person'''
        self.firstname = firstname
        self.lastname = lastname

    @property
    def fullname(self):
        '''Returns the full name of the person'''
        return self.firstname + ' ' + self.lastname

    @fullname.setter
    def fullname(self,value):
        firstname,lastname = value.split(" ")
        self.firstname = firstname
        self.lastname = lastname

    @fullname.deleter
    def fullname(self):
        del self.firstname
        del self.lastname

p1 = Person('John','Doe')
print(p1.fullname)

p1.fullname = 'Jack Daniels'
print(p1.fullname)


I am working on the file "Classes-en.ipynb".
The first task is:
All the atoms have neutrons in the nucleus, so this param must be also added to the class Atom that models what an Atom is. And also, a method to get the value. In this way, it is not needed to create a new class for each isotope because this information can be stored in the Atom class directly.
Using this approach the new class Atom is:
Can you write Python code for it?

class Atom():
    def __init__(self, electrons, protons, neutrons):
        self.electrons = electrons
        self.protons = protons
        self.neutrons = neutrons

    def get_electrons(self):
        return self.electrons

    def get_protons(self):
        return self.protons

    def get_neutrons(self):
        return self.neutrons

class Carbon(Atom):
    def __init__(self, neutrons=6):
        self.electrons = 6
        self.protons = 6
        self.neutrons = neutrons


I am working on the file "Classes-en.ipynb".
The first task is:
Class Static Methods and Variables
Analyzing the Carbon class, the number or electros and protons is always the same, no matter what is the carbon isotope object we are creating. So this data is not object specific but class specific. All the objects created from this Carbon class will have the same values for electros and protons.
To these variables that are related to the class, and that are not defined per object, we call the static variables. They are declared at class level in this way:
Can you write Python code for it?

class Atom():
    electrons = None
    protons = None

    def __init__(self, neutrons):
        self.neutrons = neutrons

    @classmethod
    def get_electrons(cls):
        print(cls)
        return cls.electrons

    @classmethod
    def get_protons(cls):
        return cls.protons

    def get_neutrons(self):
        return self.neutrons

class Carbon(Atom):
    electrons = 6
    protons = 6

    def __init__(self, neutrons=6):
        self.neutrons = neutrons

carbon = Carbon()
print(Carbon.get_electrons())
print(carbon.get_electrons())


I am working on the file "Classes-en.ipynb".
The first task is:
In this sample, the class does not receive any kind of configuration (params). Objects are created normally using params, which define the initial state of the object. In this example, a Carbon object is created with 6 electrons and 6 protons.
Can you write Python code for it?

class Atom():
    def __init__(self, electrons, protons):
        self.electrons = electrons
        self.protons = protons

carbon = Atom(electrons = 6, protons = 6)


I am working on the file "Classes-en.ipynb".
The first task is:
Now, Atoms by default are created with 6 neutrons, like Carbon12. So this is the default isotope generated when creating new Atoms. But this value can be chnaged just passing a different value in the constructor.
Can you write Python code for it?

carbon13 = Carbon(neutrons=7)
print(carbon13.get_neutrons())


I am working on the file "Classes-en.ipynb".
The first task is:
Private Methods
It is possible to declare as private methods from a class. These methods can not be used outside the definition of the class. And they are the key to enforce encapsulation of logic.
Can you write Python code for it?

class Atom():
    electrons = None
    protons = None

    def __init__(self, neutrons=None):
        self.neutrons = neutrons
        self.__atom_private()

    @classmethod
    def get_electrons(cls):
        print(cls)
        return cls.electrons

    @classmethod
    def get_protons(cls):
        return cls.protons

    def get_neutrons(self):
        return self.neutrons

    def __atom_private(self):
        print("Atom private method")

class Carbon(Atom):
    electrons = 6
    protons = 6

    def __init__(self, neutrons=6):
        self.neutrons = neutrons
        self.__atom_private()

atom = Atom()
carbon = Carbon()


I am working on the file "Classes.ipynb".
The first task is:
Next up we define a method to add two points. Addition is by elements - $(x_1, y_1) + (x_2, y_2) = (x_1+x_2, y_1+y_2)$.
We also allow to add an int, in which case we add the point to a another point with both coordinates equal to the argument value.
Can you write Python code for it?

class Point():
    """Holds on a point (x,y) in the plane"""
    def __init__(self, x=0, y=0):
        assert isinstance(x, (int, float)) and isinstance(y, (int, float))
        self.x = float(x)
        self.y = float(y)
    def __repr__(self):
        return "Point(" + str(self.x) + ", " + str(self.y) + ")"

    def add(self, other):
        assert isinstance(other, (int, Point))
        if isinstance(other, Point):
            return Point(self.x + other.x , self.y + other.y)
        else: # other is int, taken as (int, int)
            return Point(self.x + other , self.y + other)

Point(1,1).add(Point(2,2))

Point(1,1).add(2)


I am working on the file "Classes.ipynb".
The first task is:
Notice that when we send a Point to the console we get:
Can you write Python code for it?


I am working on the file "Classes.ipynb".
The first task is:
Second implementation - point and dimensions
The second implementation defines a rectangle by the lower left point, the height and the width.
We define the exact same methods as in Rectangle1, with the same input and output, but  different inner representation / implementation.
Can you write Python code for it?

class Rectangle2():
    """
    Holds a parallel-axes rectangle by storing lower left point, height and width
    """
    def __init__(self, point, height, width):
        assert isinstance(point, Point)
        assert isinstance(height, (int,float))
        assert isinstance(width, (int,float))
        assert height > 0
        assert width > 0
        self.point = point
        self.height = float(height)
        self.width = float(width)

    def __repr__(self):
        representation = "Rectangle with lower left {0} and upper right {1}"
        return representation.format(self.point, Point(self.point.x + self.width, self.point.y + self.height))

    def dimensions(self):
        return self.height, self.width

    def area(self):
        area = self.height * self.width
        return area

    def transpose(self):
        self.point = Point(self.point.x - self.height , self.point.y - self.width)
        self.height, self.width = self.width, self.height

rec = Rectangle2(Point(), 1, 2)
print(rec)
print("Area:", rec.area())
print("Dimensions:", rec.dimensions())
rec.transpose()
print("Transposed:", rec)


I am working on the file "Classes.ipynb".
The first task is:
So == checks by identity and > is not defined. Let us overload both these operators:
Can you write Python code for it?

class Point():
    """Holds on a point (x,y) in the plane"""
    def __init__(self, x=0, y=0):
        assert isinstance(x, (int, float)) and isinstance(y, (int, float))
        self.x = float(x)
        self.y = float(y)
    def __repr__(self):
        return "Point(" + str(self.x) + ", " + str(self.y) + ")"
    def __add__(self, other):
        assert isinstance(other, (int, Point))
        if isinstance(other, Point):
            return Point(self.x + other.x , self.y + other.y)
        else: # other is int, taken as (int, int)
            return Point(self.x + other , self.y + other)

    def __eq__(self, other):
        return (self.x, self.y) == (other.x, other.y)

    def __gt__(self, other):
        return (self.x > other.x and self.y > other.y)


I am working on the file "Classes.ipynb".
The first task is:
The addition operator + returns a new instance.
Next we will write a method that instead of returning a new instance, changes the current instance:
Can you write Python code for it?

class Point():
    """Holds on a point (x,y) in the plane"""
    def __init__(self, x=0, y=0):
        assert isinstance(x, (int, float)) and isinstance(y, (int, float))
        self.x = float(x)
        self.y = float(y)
    def __repr__(self):
        return "Point(" + str(self.x) + ", " + str(self.y) + ")"
    def __eq__(self, other):
        return (self.x, self.y) == (other.x, other.y)
    def __gt__(self, other):
        return (self.x > other.x and self.y > other.y)
    def __add__(self, other):
        assert isinstance(other, (int, Point))
        if isinstance(other, Point):
            return Point(self.x + other.x , self.y + other.y)
        else: # other is int, taken as (int, int)
            return Point(self.x + other , self.y + other)

    def increment(self, other):
        '''this method changes self (add "inplace")'''
        assert isinstance(other,Point)
        self.x += other.x
        self.y += other.y

p = Point(6.5, 7)
p + Point(1,2)
print(p)
p.increment(Point(1,2))
print(p)

Point(5,6) > Point(1,2)


I am working on the file "Classes.ipynb".
The first task is:
Then if one is strictly smaller than the other:
Can you write Python code for it?


I am working on the file "Classes.ipynb".
The first task is:
We can also use the method via the class instead of the instance, and give the instance of interest (the one that we want to know if it is the extreme) as the first argument self. Much like this, we can either do 'hi'.upper() or str.upper('hi').
Can you write Python code for it?

Point.is_extreme(Point(7,8), Point(1,1), Point(4,5), Point(2,3))


I am working on the file "Classes.ipynb".
The first task is:
We now write a method that given many points, checks if the current point is more extreme than the other points.
Note that the argument *points means that more than one argument may be given.
Can you write Python code for it?

class Point():
    """Holds on a point (x,y) in the plane"""
    def __init__(self, x=0, y=0):
        assert isinstance(x, (int, float)) and isinstance(y, (int, float))
        self.x = float(x)
        self.y = float(y)
    def __repr__(self):
        return "Point(" + str(self.x) + ", " + str(self.y) + ")"
    def __eq__(self, other):
        return (self.x, self.y) == (other.x, other.y)
    def __lt__(self, other):
        return (self.x < other.x and self.y < other.y)
    def __add__(self, other):
        assert isinstance(other, (int, Point))
        if isinstance(other, Point):
            return Point(self.x + other.x , self.y + other.y)
        else: # other is int, taken as (int, int)
            return Point(self.x + other , self.y + other)
    def increment(self, other):
        '''this method changes self (add "inplace")'''
        assert isinstance(other,Point)
        self.x += other.x
        self.y += other.y

    def is_extreme(self, *points):
        for point in points:
            if not self > point:
                return False
        return True

p = Point(5, 6)
p.is_extreme(Point(1,1))

p.is_extreme(Point(1,1), Point(2,5), Point(6,2))


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
At this point we can pick any classifier and train it. This is the iterative part of choosing and testing all the classifiers and tuning the hyper parameters

SGDClassifier
Training
Can you write Python code for it?

from sklearn.linear_model import SGDClassifier

clf = SGDClassifier(random_state = 0)
clf.fit(X_train, y_train_0)


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
Danger of Blindly Applying Evaluator As a Performance Measure
Let's check against a dumb classifier
Can you write Python code for it?


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
Each row: actual class
Each column: predicted class
First row: Non-zero images, the negative class:
* 53360 were correctly classified as non-zeros. True negatives.
* Remaining 717 were wrongly classified as 0s. False positive
Second row: The images of zeros, the positive class:
* 395 were incorrectly classified as 0s. False negatives
* 5528 were correctly classified as 0s. True positives

Precision
Precision measures the accuracy of positive predictions. Also called the precision of the classifier
$$\textrm{precision} = \frac{\textrm{True Positives}}{\textrm{True Positives} + \textrm{False Positives}}$$

Can you write Python code for it?

from sklearn.metrics import precision_score, recall_score

precision_score(y_train_0, y_train_pred) # 5528 / (717 + 5528)

5528 / (717+5528)


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
F1 Score
$F_1$ score is the harmonic mean of precision and recall. Regular mean gives equal weight to all values. Harmonic mean gives more weight to low values.
$$F_1=\frac{2}{\frac{1}{\textrm{precision}}+\frac{1}{\textrm{recall}}}=2\times \frac{\textrm{precision}\times \textrm{recall}}{\textrm{precision}+ \textrm{recall}}=\frac{TP}{TP+\frac{FN+FP}{2}}$$
The $F_1$ score favours classifiers that have similar precision and recall.
Can you write Python code for it?

from sklearn.metrics import f1_score

f1_score(y_train_0, y_train_pred)


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
Precision / Recall Tradeoff
Increasing precision reduced recall and vice versa
<img src="img\precision-recall.png">
Our classifier is designed to pick up zeros.
12 observations

Central Arrow
Suppose the decision threshold is positioned at the central arrow:
* We get 4 true positives (We have 4 zeros to the right of the central arrow)
* 1 false positive which is actually seven.
At this threshold, the precision accuracy is $\frac{4}{5}=80\%$
However, out of the 6 zeros, the classifier only picked up 4. The recall accuracy is $\frac{4}{6}=67\%$

Right Arrow

We get 3 true positives
0 false positive

At this threshold, the precision accuracy is $\frac{3}{3}=100\%$
However, out of the 6 zeros, the classifier only picked up 3. The recall accuracy is $\frac{3}{6}=50\%$

Left Arrow

We get 6 true positives
2 false positive

At this threshold, the precision accuracy is $\frac{6}{8}=75\%$
Out of the 6 zeros, the classifier picked up all 6. The recall accuracy is $\frac{6}{6}=100\%$
Can you write Python code for it?

clf = SGDClassifier(random_state=0)
clf.fit(X_train, y_train_0)

y[1000]

y_scores = clf.decision_function(X[1000].reshape(1, -1))
y_scores

threshold = 0

y_some_digits_pred = (y_scores > threshold)

y_some_digits_pred

threshold = 40000
y_some_digits_pred = (y_scores > threshold)
y_some_digits_pred

y_scores = cross_val_predict(clf, X_train, y_train_0, cv=3, method='decision_function')

plt.figure(figsize=(12,8)); plt.hist(y_scores, bins=100);


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
Prediction
Can you write Python code for it?


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
Setting High Precisions
Let's aim for 90% precisions.
Can you write Python code for it?

len(precisions)

len(thresholds)

plt.figure(figsize=(12,8));
plt.plot(thresholds, precisions[1:]);

idx = len(precisions[precisions < 0.9])

thresholds[idx]

y_train_pred_90 = (y_scores > 21454)

precision_score(y_train_0, y_train_pred_90)

recall_score(y_train_0, y_train_pred_90)


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
Setting High Precisions
Let's aim for 99% precisions.
Can you write Python code for it?

idx = len(precisions[precisions < 0.99])

thresholds[idx]

y_train_pred_90 = (y_scores > thresholds[idx])

precision_score(y_train_0, y_train_pred_90)

recall_score(y_train_0, y_train_pred_90)


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
Training a Binary Classifier
To simplify our problem, we will make this an exercise of "zero" or "non-zero", making it a two-class problem.
We need to first convert our target to 0 or non zero.
Can you write Python code for it?

y_train_0 = (y_train == 0)

y_train_0

y_test_0 = (y_test == 0)

y_test_0


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
Visualisation
Can you write Python code for it?

X, y = mnist['data'], mnist['target']

X

y

X[69999]

y[69999]

X.shape

y.shape

%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
import seaborn as sns

_ = X[1000]
_image = _.reshape(28, 28)
plt.imshow(_image);

y[1000]


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
With the decision scores, we can compute precision and recall for all possible thresholds using the precision_recall_curve() function:
Can you write Python code for it?

from sklearn.metrics import precision_recall_curve

precisions, recalls, thresholds = precision_recall_curve(y_train_0, y_scores)

def plot_precision_recall_vs_threshold(precisions, recalls, thresholds):
    plt.plot(thresholds, precisions[:-1], "b--", label="Precision")
    plt.plot(thresholds, recalls[:-1], "g--", label="Recall")
    plt.xlabel("Threshold")
    plt.legend(loc="upper left")
    plt.ylim([-0.5,1.5])

plt.figure(figsize=(12,8));
plot_precision_recall_vs_threshold(precisions, recalls, thresholds)
plt.show()


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
With this chart, you can select the threshold value that gives you the best precision/recall tradeoff for your task.
Some tasks may call for higher precision (accuracy of positive predictions). Like designing a classifier that picks up adult contents to protect kids. This will require the classifier to set a high bar to allow any contents to be consumed by children.
Some tasks may call for higher recall (ratio of positive instances that are correctly detected by the classifier). Such as detecting shoplifters/intruders on surveillance images - Anything that remotely resemble "positive" instances to be picked up.

One can also plot precisions against recalls to assist with the threshold selection
Can you write Python code for it?

plt.figure(figsize=(12,8));
plt.plot(precisions, recalls);
plt.xlabel('recalls');
plt.ylabel('precisions');
plt.title('PR Curve: precisions/recalls tradeoff');


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Classification Based Machine Learning Algorithm.ipynb".
The first task is:
cross_val_score using K-fold Cross-Validation
K-fold cross-validation splits the training set into K-folds and then make predictions and evaluate them on each fold using a model trained on the remaning folds.
Can you write Python code for it?

from sklearn.model_selection import cross_val_score

cross_val_score(clf, X_train, y_train_0, cv=3, scoring='accuracy')


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Global_Stock_Selection_14_Fundamental_factors+-+Logistic+git.ipynb".
The first task is:
Remove Ticker Name
remove all the ticker names. We do not need them to build our model
Can you write Python code for it?

res = df_combined.copy()

res = res.reset_index().set_index('date')
res.pop('security')
res


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Global_Stock_Selection_14_Fundamental_factors+-+Logistic+git.ipynb".
The first task is:
Training our Logistic ML
Can you write Python code for it?

start = time()
scaler = preprocessing.StandardScaler()
clf = LogisticRegression(random_state=0)
X_train_trans = scaler.fit_transform(X_train)
clf.fit(X_train_trans, Y_train)
end = time()

print "Time taken to train ML: %0.2f secs" % (end - start)


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Global_Stock_Selection_14_Fundamental_factors+-+Logistic+git.ipynb".
The first task is:
try replacing nan with 0. Alternative is to remove the data.
Prefer not to use mean as is the case for sklean imputer and it would distort the model
Can you write Python code for it?

for e in res.columns:
    res[e]  = res[e].replace(np.nan, 0)
    res[e]  = res[e].replace([np.inf, -np.inf], 0)

res


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Logistic Regression.ipynb".
The first task is:
Learning the Logistic Regression Model
The coefficients (Beta values b) of the logistic regression algorithm must be estimated from your training data. This is done using maximum-likelihood estimation.
Maximum-likelihood estimation is a common learning algorithm used by a variety of machine learning algorithms, although it does make assumptions about the distribution of your data (more on this when we talk about preparing your data).
The best coefficients would result in a model that would predict a value very close to 1 (e.g. male) for the default class and a value very close to 0 (e.g. female) for the other class. The intuition for maximum-likelihood for logistic regression is that a search procedure seeks values for the coefficients (Beta values) that minimize the error in the probabilities predicted by the model to those in the data (e.g. probability of 1 if the data is the primary class).
We are not going to go into the math of maximum likelihood. It is enough to say that a minimization algorithm is used to optimize the best values for the coefficients for your training data. This is often implemented in practice using efficient numerical optimization algorithm (like the Quasi-newton method).
When you are learning logistic, you can implement it yourself from scratch using the much simpler gradient descent algorithm.
Learning with Stochastic Gradient Descent
Logistic Regression uses gradient descent to update the coefficients.
Each gradient descent iteration, the coefficients are updated using the equation:
$$\beta=\beta+\textrm{learning rate}\times (y-\hat{y}) \times \hat{y} \times (1-\hat{y}) \times x $$

Using Scikit Learn to Estimate Coefficients
Can you write Python code for it?

from sklearn.linear_model import LogisticRegression

dataset

X = np.array(dataset)[:, 0:1]
y = np.array(dataset)[:, 1]

X

y

clf_LR = LogisticRegression(C=1.0, penalty='l2', tol=0.0001)

clf_LR.fit(X,y)

clf_LR.predict(X)

clf_LR.predict_proba(X)


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Logistic Regression.ipynb".
The first task is:
Let's say you have been provided with the coefficient
Can you write Python code for it?

coef = [-0.806605464, 0.2573316]

for row in dataset:
    yhat = 1.0 / (1.0 + np.exp(- coef[0] - coef[1] * row[0]))
    print("yhat {0:.4f}, yhat {1}".format(yhat, round(yhat)))


I am working on the file "Classification Based Machine Learning for Algorithmic Trading/Logistic Regression.ipynb".
The first task is:
Making Predictions with Logistic Regression
$$\hat{y}=\frac{1.0}{1.0+e^{-\beta_0-\beta_1x_i}}$$
$\beta_0$ is the intecept term
$\beta_1$ is the coefficient for $x_i$
$\hat{y}$ is the predicted output with real value between 0 and 1. To convert this to binary output of 0 or 1, this would either need to be rounded to an integer value or a cutoff point be provided to specify the class segregation point.
Can you write Python code for it?


I am working on the file "Classification of Organisms.ipynb".
The first task is:
Importing the Organism Characteristics/Conditions
Can you write Python code for it?

# Conditions/Questions for finding the correct animal
# Displays all column titles as an array
key.columns


I am working on the file "Classification of Organisms.ipynb".
The first task is:
Part 3: Scientific Classification of Organisms
Can you write Python code for it?

#sort your organisms by their taxonomical classification

# This conditional allows us to query a column and if the data within that cell matches,
# it will display the corresponding animal(s)
key2[(key2['kingdom'] == 'animalia')]

#Done?? Insert a image for one of the organisms you found using the dichotomous key.
from IPython.display import Image
from IPython.core.display import HTML
Image(url= 'https://lms.mrc.ac.uk/wp-content/uploads/insert-pretty-picture-here1.jpg')


I am working on the file "Classification of Organisms.ipynb".
The first task is:
Pre-Questions
A Dichotomous Key is....
a tool that allows scienctists to identify and classify organisms in the natural world.  Based on their characterists, scienctists can narrow down species into groups such as trees, flowers, mammals, reptiles, rocks, and fish. A Dichotomous Key can help to understand how scientists have classified organisms using Bionomial Nomenclature.
Dichotomous Key Video
Can you write Python code for it?

# Here is a helpful image of a sample Dichotomous Key!
from IPython.display import Image
from IPython.core.display import HTML
Image(url= 'http://biology-igcse.weebly.com/uploads/1/5/0/7/15070316/8196495_orig.gif')


I am working on the file "Classification.ipynb".
The first task is:
2. Pick Features
Feature selection is an important part of classification. Unfortunately there is no systematic way to select these features. To some extent this is where the science can come in. In my case I just guess. Really we just use what is available in the data.  Our features will be:

Magnitude (median)
Period
Amplitude
Skew
IQR (interquantile range)
Can you write Python code for it?

def setup_features():
    data = {
        'class': [],
        'predicted': [],
        'ra': [],
        'dec': [],
        'period': [],
        'mag': [],
        'mag_median': [],
        'amplitude': [],
        'skew': [],
        'iqr': [],
    }
    unique_id = catalog.index.unique()
    for uid in widgets.log_progress(unique_id):
            rows = catalog.loc[uid]

            data['class'].append(rows['Class'].iloc[0])
            data['predicted'].append(0)

            ra = rows['RA'].iloc[0]
            dec = rows['Dec'].iloc[0]
            coord = SkyCoord(ra=ra, dec=dec, unit=(u.hourangle, u.deg))
            data['ra'].append(coord.ra.value)
            data['dec'].append(coord.dec.value)

            data['period'].append(rows['Period'].iloc[0])
            data['mag'].append(rows['Mag'].iloc[0])
            data['mag_median'].append(np.median(rows['Mag']))
            data['amplitude'].append(rows['Amplitude'].iloc[0])
            data['skew'].append(stats.skew(rows['Mag']))
            data['iqr'].append(np.percentile(rows['Mag'], 75) - np.percentile(rows['Mag'], 25))

    features = pandas.DataFrame(data, index=unique_id)
    features.index.name = 'ID'

    # Write to CSV to save time
    features.to_csv('features.csv')

    return features

# Get our features from the file or build them
if not os.path.exists('features.csv'):
    features = setup_features()
else:
    features = pandas.read_csv('features.csv', index_col=0)

features_copy = features.copy()
# Remove meta information
try:
    del features['dec']
#     del features['predicted']
    del features['ra']
    del features['mag']
except:
    pass

features.head()


I am working on the file "Classification.ipynb".
The first task is:
3 Separate Data
We need three groups of data: training, testing, and unknown. In our case the unknown is really known but we are just pretending. The training and testing data is the labelled and known data. Ideally you want to train on about 2/3 of the known data and then test the remaining 1/3 to make sure your classifier is working ok. However, exactly how much you want to train on can affect the results.
3.1 Separate Labelled and Unlabelled Data
To start, we use a poor-man's version of the holdout method (e.g. hold some of them out) to split the initial set of data into labelled and unlablled. For simplicity we just use 75% of the data for training
Can you write Python code for it?

# Start to build the table we will use

# For Training data, we want to get a certain amount of our
# table, we slice off a percentage
row_start = 0
row_end = len(features) - int(.25 * len(features))

# Our collections of data to train and test
labeled_svm = features.iloc[row_start:row_end]
unlabeled_svm = features.iloc[row_end:]
test_all = features.iloc[row_end:]

# The correct classes corresponding to the training and test
classes = labeled_svm['class']
unlabeled_classes = unlabeled_svm['class']

# Remove the labels
del labeled_svm['class']
del unlabeled_svm['class']

labeled_svm.head()

types = { 1: 'EW', 2: 'EA', 3: 'beta Lyrae'}

# How many of each Variable Type do we have
print("Type\t\t\tType\t\tNumber".format())
print("{}".format('-'*45))
for x in range(1,4):
    print("Labeled Class size:\t {:14s} {}".format(types[x],len(classes[classes == x])))
    print("Unlabeled Class size:\t {:14s} {}\n".format(types[x],len(unlabeled_classes[unlabeled_classes == x])))

print("{}".format('-'*45))
print('Labeled Total: \t\t\t\t{}'.format(len(labeled_svm)))
print('Unlabeled Total: \t\t\t{}'.format(len(unlabeled_svm)))


I am working on the file "Classification.ipynb".
The first task is:
3.2 Determine Training Size
Now that we have some labelled data we need to split that data into training data and test data. While 2/3 to 1/3 for training to test is a good initial estimate, we can do some work to determine the best size by trying different classifiers
First we create a simple linear classifier and fit the labelled data to the known classes
Can you write Python code for it?

# Create our (linear) classifier
clf = svm.LinearSVC()
clf.fit(labeled_svm, classes)


I am working on the file "Classification.ipynb".
The first task is:
4. Pick your SVC (Support Vector Classification)
There are a number of possible different classifiers that are available but the classifiers du jour are all Support Vector Machines (SVM). However, even with an SVM there is the decision of which kernel to use. Luckily these can be tested.
We use the sklean.svm.svc module with a few different kernels to attempt some fits.
Determine Best Kernel Classifier
Run through the different kernels for an SVC to determine the best. Possible kernels are:

linear
poly
rbf
sigmoid
precomputed

We perform an exhaustive grid search across all the kernels with a range of different values. We compute these separately so that we can then compare each with an ROC. The grid search would handle this internally if we ran them all together but we want to generate an ROC plot so we do it separate.
Get the optimal SVC
Here we define the parameters for each of the kernel types and then run them through an exhaustive grid search. This is similar to the examples above where we ran through a list of $C$ and $\gamma$ values but here the GridSearchCV takes care of all the details. This gets us our optimal classifier based on this exhaustive search.
One area of uncertainty is the selection of our ranges below. We lifted the $C$ and $\gamma$ ranges from Guillermo's Hands-On (todo: get reference) and just copied that over for the coefficient range. We totall guessed at the degree range and would need to explore that more.
Can you write Python code for it?

def get_optimal_svc(param_grid=list(), plot=False):

    # Do the grid search
    grid = model_selection.GridSearchCV(svm.SVC(), param_grid=param_grid, cv=ss)
    grid.fit (labeled_svm, classes)

    if plot:
        score_dict = grid.cv_results_

        # We extract the scores
        scores = np.array([x[1] for x in score_dict])
        scores = scores.reshape(len(C_range), len(gamma_range))

        # Make a nice figure
        plt.figure(figsize=(8, 6))
        plt.subplots_adjust(left=0.15, right=0.95, bottom=0.15, top=0.95)
        plt.imshow(scores, interpolation='nearest', cmap=plt.cm.gist_heat)
        plt.xlabel('$\gamma$')
        plt.ylabel('$C$')
        plt.title('Heatmap for optimal: $C={}$ and $\gamma={}$\n'.format(grid.best_estimator_.C, grid.best_estimator_.gamma))
        plt.colorbar()
        plt.xticks(np.arange(len(gamma_range)), gamma_range, rotation=45)
        plt.yticks(np.arange(len(C_range)), C_range)
        plt.show()

    print("The best classifier for this run is: {}".format(grid.best_estimator_))
    return grid.best_estimator_


I am working on the file "Classification.ipynb".
The first task is:
Classification
Classification is about creating a way to classify or label unknown data based off of existing labeled data. The idea is that you should be able to classify a certain amount of your data (either via the hard work of dedicated graduates students or through naive and willing members of the public, e.g. Galaxy Zoo)
The Process:

Get some data
Pick your features!
Separate the data
Find a kernel (for your SVM, of course!)
Classify! For fun and profit!
Analyze (Did it work?)

1. Get Data
Ideally you would have some labelled and unlabelled data. Here we want to test classifiers so we grab a whole bunch of labelled data and then just pretend we don't know the labels for some of them.
The data we are working with are variable stars. There are many types of variable stars but we only try to classify between the periodic.
Data is obatained from the Catalina Sky Survey (http://catalinadata.org).
Can you write Python code for it?

# Load our matched data from the catalog
cat_data = 'catalog_data_final/matched_data.csv'

# Number of rows
!wc -l $cat_data

catalog = pandas.read_csv(cat_data, index_col=0, usecols=[1,2,3,7,10,11,12,13,14,15,16,17])

# Filter and scrub
catalog = catalog[catalog['Blend'] == 0]
catalog.rename(columns={
        'V_(mag)': 'V_mag',
        'Period_(days)': 'Period',
        'Var_Type': 'Class',
        'RA_(J2000)': 'RA',
}, inplace=True)

catalog.head()


I am working on the file "Classification.ipynb".
The first task is:
It may look a little slow, but note that time is spent on cropping, python interfacing, and running 10 images. For performance, if you really want to make prediction fast, you can optionally code in C++ and pipeline operations better. For experimenting and prototyping the current speed is fine.
Let's time classifying a single image with input preprocessed:
Can you write Python code for it?

# Resize the image to the standard (256, 256) and oversample net input sized crops.
input_oversampled = caffe.io.oversample([caffe.io.resize_image(input_image, net.image_dims)], net.crop_dims)
# 'data' is the input blob name in the model definition, so we preprocess for that input.
caffe_input = np.asarray([net.transformer.preprocess('data', in_) for in_ in input_oversampled])
# forward() takes keyword args for the input blobs with preprocessed input arrays.
%timeit net.forward(data=caffe_input)


I am working on the file "Classification.ipynb".
The first task is:
Let's take a look at our example image with Caffe's image loading helper.
Can you write Python code for it?

input_image = caffe.io.load_image(IMAGE_FILE)
plt.imshow(input_image)


I am working on the file "Classification.ipynb".
The first task is:
Loading a network is easy. caffe.Classifier takes care of everything. Note the arguments for configuring input preprocessing: mean subtraction switched on by giving a mean array, input channel swapping takes care of mapping RGB into the reference ImageNet model's BGR order, and raw scaling multiplies the feature scale from the input [0,1] to the ImageNet model's [0,255].
We will set the phase to test since we are doing testing, and will first use CPU for the computation.
Can you write Python code for it?

caffe.set_mode_cpu()
net = caffe.Classifier(MODEL_FILE, PRETRAINED,
                       mean=np.load(caffe_root + 'python/caffe/imagenet/ilsvrc_2012_mean.npy').mean(1).mean(1),
                       channel_swap=(2,1,0),
                       raw_scale=255,
                       image_dims=(256, 256))


I am working on the file "Classification.ipynb".
The first task is:
Note: This takes a long time
Can you write Python code for it?

# Setup our ranges
C_range = 10. ** np.arange(-4, 4)
gamma_range = 10. ** np.arange(-4, 4)
deg_range = np.arange(1, 3)
coef_range = 10. ** np.arange(-4, 4)

param_grids = [
#     { 'C': C_range, 'kernel': ['linear'] },
#     { 'C': C_range, 'gamma': gamma_range, 'degree': deg_range, 'coef0': coef_range, 'kernel': ['poly'] },
    { 'C': C_range, 'gamma': gamma_range, 'kernel': ['rbf'] },
#     { 'C': C_range, 'gamma': gamma_range, 'coef0': coef_range, 'kernel': ['sigmoid'] },

    { 'kernel': ['linear'] },
    { 'kernel': ['poly'] },
    { 'kernel': ['sigmoid'] },
]

best_svcs = list()

# Run the param_grid one at a time (to compare ROC below)
# Get the best clf for each kernel type
for param_grid in param_grids:
    print("{}...".format(param_grid.get('kernel')))
    clf = get_optimal_svc(param_grid=param_grid, plot=False)

    param_grid['clf'] = clf

    best_svcs.append(param_grid)


I am working on the file "Classification.ipynb".
The first task is:
Now, why don't we see how long it takes to perform the classification end to end? This result is run from an AMD A8-6600K APU, so you may observe some performance differences.
Can you write Python code for it?


I am working on the file "Classification.ipynb".
The first task is:
OK, so how about GPU? it is actually pretty easy:
Can you write Python code for it?


I am working on the file "Classification.ipynb".
The first task is:
PART 1: Sorting Organisms by One Characteristic

We will be looking at the characterists of 75 unique organisms in our Dichotomous Key.  The imput below will provide us with a some of the possible organisms you may discover and the different Organism Characteristics/Conditions in our data set.
Can you write Python code for it?

# Animal options in Dichotomous Key
# Displays all row titles as an array
key.organism

# Conditions/Questions for finding the correct animal
# Displays all column titles as an array
key.columns


I am working on the file "Classification.ipynb".
The first task is:
PART 2: Sorting Organisms by Many Characteristics

These are the conditions or the characteristics in which ceratin answers are categorized for certain organisms. Each characteristic/condition has a yes/no except for the Kingdoms. Change the conditionals in the code below to change what organism(s) are displayed. For most, the only change needs to be the 'yes' or 'no'.

Capitalization matters so be careful. You also must put in only allowed answers in every condition or the code will break!

Use and modify the section of code below to answer questions 6-8 in your coding booklet.
Can you write Python code for it?

# This conditional allows us to query a column and if the data within that cell matches it will display the animal(s).
#if you are unsure of what to put try making that column a comment by adding # in front of it.

key[
#physical characteristics
    (key['fur'] == 'yes') & \
    (key['feathers'] == 'no') & \
    (key['poisonous'] == 'no') & \
    (key['scales'] == 'no') & \
    (key['multicellular'] == 'yes') & \
    (key['fins'] == 'no') & \
    (key['wings'] == 'no') & \
    (key['vertebrate'] == 'yes') & \

#environmental characteristics
    (key['marine'] == 'no') & \
    (key['terrestrial'] == 'yes') & \

#feeding characteristics

    #decomposers get their food by breaking down decaying organisms
    (key['decomposer'] == 'no') & \

    #carnivores get their food by eating animals
    (key['carnivore'] == 'no') & \

    #herbivores get their food by eating plants
    (key['herbivore'] == 'yes') & \

    #omnivores get their food by eating both plants and animals
    (key['omnivore'] == 'no') & \

    #photosynthesis is the process of making food using energy from sunlight
    (key['photosynthesis'] == 'no') & \

    #autotrophs are organisms that generate their own food inside themselves
    (key['autotroph'] == 'no') & \

#possible kingdoms include: animalia, plantae, fungi
    (key['kingdom'] == 'animalia') & \

#cell type
    (key['eukaryotic'] == 'yes') & \
    (key['prokaryotic'] == 'no')
    ]


I am working on the file "Classification.ipynb".
The first task is:
Part 3 & 4: Scientific Classification of Organisms & Unstructured Coding

Use and modify the section of code below to answer questions 9-13 in your coding booklet.
Can you write Python code for it?


I am working on the file "Classification.ipynb".
The first task is:
Pre-Questions

Using the information and video below and what you've learned in this unit, answer questions 1 & 2 in your coding booklet.
A Dichotomous Key is....
a tool that allows scienctists to identify and classify organisms in the natural world.  Based on their characterists, scienctists can narrow down species into groups such as trees, flowers, mammals, reptiles, rocks, and fish. A Dichotomous Key can help to understand how scientists have classified organisms using Bionomial Nomenclature.
You can find out more about Dichotomous Keys by watching this video. Dichotomous Key Video
Can you write Python code for it?


I am working on the file "Classification.ipynb".
The first task is:
ROC Curve
Here we determine which classifier was the "best" via an ROC Curve.
NOTE: We could have just performed the exhaustive grid search above and it would have automatically compared all of these and output the absolute best, but because we wanted to generate an ROC we performed them separately.
Can you write Python code for it?

def plot_roc(classifier_list=None):
    # Plot the ROC for each kernel type
    plt.clf()
    fig = plt.figure()
    ax = fig.add_subplot(1,1,1)

    # Add a line for each kernel
    for classifier in classifier_list:
        fpr = classifier['fpr']
        tpr = classifier['tpr']
        auc = classifier['auc']
        ax.plot (fpr, tpr, label='{} (area = {:0.2f})'.format(classifier.get('kernel')[0], auc))

    plt.legend(loc=4)
    plt.xlabel('False Positive Rate')
    plt.ylabel('True Positive Rate')
    plt.title('Receiver Operating Characteristic Curve')
    plt.show()

plot_roc(classifier_list=best_svcs)


I am working on the file "Classification.ipynb".
The first task is:
Use and modify the section of code below to answer questions 3-5 in your coding booklet.
Can you write Python code for it?


I am working on the file "Classification.ipynb".
The first task is:
Voila! Now we are in GPU mode. Let's see if the code gives the same result:
Can you write Python code for it?

prediction = net.predict([input_image])
print 'prediction shape:', prediction[0].shape
plt.plot(prediction[0])


I am working on the file "Classification.ipynb".
The first task is:
We can also use a slightly more fancy version for (sometimes) better results:
Can you write Python code for it?

ss = model_selection.StratifiedShuffleSplit(test_size = 1./3.)
scores = model_selection.cross_val_score(clf, labeled_svm, classes, cv=ss)
print("Accuracy = {} +- {}".format(scores.mean(),scores.std()))


I am working on the file "Classification.ipynb".
The first task is:
You and your friends can test out all of the available splitter classes here: http://scikit-learn.org/stable/modules/classes.html#splitter-classes
We can also test a whole bunch of training sizes and plot them
Can you write Python code for it?

step = 100
stop = int(len(labeled_svm) * (2./3.))
Ns = np.arange(500, stop , step)

scores = np.zeros(len(Ns))
stds = np.zeros(len(Ns))
for i in range(len(Ns)):
    N = Ns[i]
    ss = model_selection.StratifiedShuffleSplit(test_size = 1./3., train_size = N)
    scores_i = model_selection.cross_val_score(clf, labeled_svm, classes, cv=ss)
    scores[i] = scores_i.mean()
    stds[i] = scores_i.std()

# Get our optimal n
optimal_n = Ns[np.argmax(scores)]

# Plot our results, including optimal n
plt.clf()
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.errorbar (Ns, scores, yerr = stds)
ax.set_xlabel("N")
ax.set_ylabel("Accuracy")
plt.title("Optimal: $n={}$".format(optimal_n))
plt.show()


I am working on the file "Classification.ipynb".
The first task is:
You can see that the prediction is 1000-dimensional, and is pretty sparse.
The predicted class 281 is "Tabby cat." Our pretrained model uses the synset ID ordering of the classes, as listed in ../data/ilsvrc12/synset_words.txt if you fetch the auxiliary imagenet data by ../data/ilsvrc12/get_ilsvrc_aux.sh. If you look at the top indices that maximize the prediction score, they are cats, foxes, and other cute mammals. Not unreasonable predictions, right?
Now let's classify by the center crop alone by turning off oversampling. Note that this makes a single input, although if you inspect the model definition prototxt you'll see the network has a batch size of 10. The python wrapper handles batching and padding for you!
Can you write Python code for it?

prediction = net.predict([input_image], oversample=False)
print 'prediction shape:', prediction[0].shape
plt.plot(prediction[0])
print 'predicted class:', prediction[0].argmax()


I am working on the file "Classification/Car Evaluation using Decision trees and Random Forests.ipynb".
The first task is:
After we have installed GraphViz on our computer, we can convert the tree.dot file into a PNG file by executing the following command from the command line in the location where we saved the tree.dot file:

dot -Tpng tree.dot -o tree.png

<img src="images/tree.png">
Combining weak to strong learners via random forests
Random forests have gained huge popularity in applications of machine learning during the last decade due to their good classification performance, scalability, and ease of use. Intuitively, a random forest can be considered as an ensemble of decision trees. The idea behind ensemble learning is to combine weak learners to build a more robust model, a strong learner, that has a better generalization error and is less susceptible to overfitting. The random forest algorithm can be summarized in four simple steps:

Draw a random bootstrap sample of size n (randomly choose n samples from the training set with replacement).

Grow a decision tree from the bootstrap sample. At each node:

Randomly select d features without replacement.
Split the node using the feature that provides the best split according to the objective function, for instance, by maximizing the information gain.

Repeat the steps 1 to 2 k times.

Aggregate the prediction by each tree to assign the class label by majority vote.

There is a slight modification in step 2 when we are training the individual decision trees: instead of evaluating all features to determine the best split at each node, we only consider a random subset of those.
Although random forests don't offer the same level of interpretability as decision trees, a big advantage of random forests is that we don't have to worry so much about choosing good hyperparameter values. We typically don't need to prune the random forest since the ensemble model is quite robust to noise from the individual decision trees. The only parameter that we really need to care about in practice is the number of trees k (step 3) that we choose for the random forest. Typically, the larger...
Can you write Python code for it?

t2=time()
print ("RandomForest")
rf = RandomForestClassifier(n_estimators=100,n_jobs=-1)
clf_rf = rf.fit(X_train,y_train)
print ("Acurracy: ", clf_rf.score(X_test,y_test))
t3=time()
print ("time elapsed: ", t3-t2)

tt2=time()
print ("cross result========")
scores = cross_validation.cross_val_score(rf, X, y, cv=3)
print (scores)
print (scores.mean())
tt3=time()
print ("time elapsed: ", tt3-tt2)


I am working on the file "Classification/Car Evaluation using Decision trees and Random Forests.ipynb".
The first task is:
As you can see, we achieved an accuracy of about 75% on the test set.
Cross validation for Decision Trees:
Can you write Python code for it?

tt0=time()
print ("cross result========")
scores = cross_validation.cross_val_score(dt, X, y, cv=3)
print (scores)
print (scores.mean())
tt1=time()
print ("time elapsed: ", tt1-tt0)


I am working on the file "Classification/Car Evaluation using Decision trees and Random Forests.ipynb".
The first task is:
Finding the Feature importances with forests of trees
This examples shows the use of forests of trees to evaluate the importance of features on an artificial classification task. The red bars are the feature importances of the forest, along with their inter-trees variability.
Can you write Python code for it?

%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

from sklearn.ensemble import ExtraTreesClassifier

# Build a classification task using 3 informative features

# Build a forest and compute the feature importances
forest = ExtraTreesClassifier(n_estimators=250,
                              random_state=0)

forest.fit(X, y)
importances = forest.feature_importances_
std = np.std([tree.feature_importances_ for tree in forest.estimators_],
             axis=0)
indices = np.argsort(importances)[::-1]

# Print the feature ranking
print("Feature ranking:")

for f in range(X.shape[1]):
    print("%d. feature %d - %s (%f) " % (f + 1, indices[f], features[indices[f]], importances[indices[f]]))

# Plot the feature importances of the forest
plt.figure(num=None, figsize=(14, 10), dpi=80, facecolor='w', edgecolor='k')
plt.title("Feature importances")
plt.bar(range(X.shape[1]), importances[indices],
       color="r", yerr=std[indices], align="center")
plt.xticks(range(X.shape[1]), indices)
plt.xlim([-1, X.shape[1]])
plt.show()

for f in range(5):
    print("%d. feature %d - %s (%f)" % (f + 1, indices[f], features[indices[f]] ,importances[indices[f]]))

best_features = []
for i in indices[:5]:
    best_features.append(features[i])


I am working on the file "Classification/Car Evaluation using Decision trees and Random Forests.ipynb".
The first task is:
Next we load our dataset into a pandas dataframe. We specify the column names before-hand.
Can you write Python code for it?

# read .csv from provided dataset
csv_filename="car.data"

# df=pd.read_csv(csv_filename,index_col=0)
df=pd.read_csv(csv_filename,
              names=["Buying", "Maintenance" , "Doors" , "Persons" , "Lug-Boot" , "Safety", "Class"])

df.head()


I am working on the file "Classification/Car Evaluation using Decision trees and Random Forests.ipynb".
The first task is:
Plot the top 5 feature importances of the forest
Can you write Python code for it?

plt.figure(num=None, figsize=(8, 6), dpi=80, facecolor='w', edgecolor='k')
plt.title("Feature importances")
plt.bar(range(5), importances[indices][:5],
       color="r",  yerr=std[indices][:5], align="center")
plt.xticks(range(5), best_features)
plt.xlim([-1, 5])
plt.show()


I am working on the file "Classification/Car Evaluation using Decision trees and Random Forests.ipynb".
The first task is:
Since the target class labels are strings, we'll have to convert in a format that our classifier would understand. For this we would use the LabelEncoder class of Scikit-Learn module 'preprocessing'.  This converts our class labels into [1,2,3,4] where the integers would correspond to the respective class.
Can you write Python code for it?

#Convert car-class labels to numbers
le = preprocessing.LabelEncoder()
df['Class'] = le.fit_transform(df.Class)

df['Class'].unique()

features = list(df.columns)
features.remove('Class')


I am working on the file "Classification/Car Evaluation using Decision trees and Random Forests.ipynb".
The first task is:
We split the dataset into 60% training and 40% testing sets.
Can you write Python code for it?

# split dataset to 60% training and 40% testing
X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, y, test_size=0.4, random_state=0)

from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(X_train)
X_train_std = sc.transform(X_train)
X_test_std = sc.transform(X_test)


I am working on the file "Classification/Classifiying Ionosphere structure using K nearest neigbours algorithm.ipynb".
The first task is:
After creating our estimator, we must then fit it on our training dataset. For the
nearest neighbor class, this records our dataset, allowing us to find the nearest
neighbor for a new data point, by comparing that point to the training dataset:
estimator.fit(X_train, y_train)
We then train the algorithm with our test set and evaluate with our testing set:
Can you write Python code for it?

estimator.fit(X_train, y_train)

y_predicted = estimator.predict(X_test)
accuracy = np.mean(y_test == y_predicted) * 100
print("The accuracy is {0:.1f}%".format(accuracy))


I am working on the file "Classification/Classifiying Ionosphere structure using K nearest neigbours algorithm.ipynb".
The first task is:
Next, we break the dataset by dividing every second feature by 10:
Can you write Python code for it?


I am working on the file "Classification/Classifiying Ionosphere structure using K nearest neigbours algorithm.ipynb".
The first task is:
Pipelines take a list of steps as input, representing the chain of the data mining
application. The last step needs to be an Estimator, while all previous steps are
Transformers. The input dataset is altered by each Transformer, with the output of
one step being the input of the next step. Finally, the samples are classified by the last
step's estimator. In our pipeline, we have two steps:
1. Use MinMaxScaler to scale the feature values from 0 to 1
2. Use KNeighborsClassifier as the classification algorithms
Each step is then represented by a tuple ('name', step). We can then create
our pipeline:
Can you write Python code for it?

scaling_pipeline = Pipeline([('scale', MinMaxScaler()),
                             ('predict', KNeighborsClassifier())])


I am working on the file "Classification/Classifiying Ionosphere structure using K nearest neigbours algorithm.ipynb".
The first task is:
Then, we import the nearest neighbor class and create an instance for it using the default parameters. By default, the algorithm will choose the five nearest neighbors to predict
the class of a testing sample:
Can you write Python code for it?

from sklearn.neighbors import KNeighborsClassifier

estimator = KNeighborsClassifier()


I am working on the file "Classification/Classifiying Ionosphere structure using K nearest neigbours algorithm.ipynb".
The first task is:
This gives us back our score of 82.3 percent accuracy. The MinMaxScaler resulted in
features of the same scale, meaning that no features overpowered others by simply
being bigger values. While the Nearest Neighbor algorithm can be confused with
larger features, some algorithms handle scale differences better. In contrast, some
are much worse!
Pipelines
As experiments grow, so does the complexity of the operations. We may split up
our dataset, binarize features, perform feature-based scaling, perform sample-based
scaling, and many more operations.
Keeping track of all of these operations can get quite confusing and can result in
being unable to replicate the result. Problems include forgetting a step, incorrectly
applying a transformation, or adding a transformation that wasn't needed.
Another issue is the order of the code. In the previous section, we created our
X_transformed dataset and then created a new estimator for the cross validation.
If we had multiple steps, we would need to track all of these changes to the dataset
in the code.
Pipelines are a construct that addresses these problems (and others, which we will
see in the next chapter). Pipelines store the steps in your data mining workflow. They
can take your raw data in, perform all the necessary transformations, and then create
a prediction. This allows us to use pipelines in functions such as cross_val_score,
where they expect an estimator. First, import the Pipeline object:
Can you write Python code for it?


I am working on the file "Classification/Classifiying Ionosphere structure using K nearest neigbours algorithm.ipynb".
The first task is:
This scores 86.4 percent accuracy, which is impressive for a default algorithm and
just a few lines of code! Most scikit-learn default parameters are chosen explicitly
to work well with a range of datasets. However, you should always aim to choose
parameters based on knowledge of the application experiment.
Can you write Python code for it?

from sklearn.cross_validation import cross_val_score

scores = cross_val_score(estimator, X, y, scoring='accuracy')
average_accuracy = np.mean(scores) * 100
print("The average accuracy is {0:.1f}%".format(average_accuracy))


I am working on the file "Classification/Classifiying Ionosphere structure using K nearest neigbours algorithm.ipynb".
The first task is:
We then import pyplot from the matplotlib library and plot the parameter values
alongside average scores:
Can you write Python code for it?

from matplotlib import pyplot as plt
plt.figure(figsize=(32,20))
plt.plot(parameter_values, avg_scores, '-o', linewidth=5, markersize=24)
#plt.axis([0, max(parameter_values), 0, 1.0])


I am working on the file "Classification/Logistic Regression for Banknote Authentication.ipynb".
The first task is:
Apparently, the first 5 instances of our datasets are all fake (Class is 0).
Can you write Python code for it?

print("No of Fake bank notes = " + str(len(df[df['Class'] == 0])))

print("No of Authentic bank notes = " + str(len(df[df['Class'] == 1])))


I am working on the file "Classification/Logistic Regression for Banknote Authentication.ipynb".
The first task is:
Here, y_test are the true class labels and y_pred are the class labels that we predicted previously.
Furthermore, we can predict the class-membership probability of the samples via
the predict_proba method. For example, we can predict the probabilities of the
first banknote sample:
Can you write Python code for it?


I am working on the file "Classification/Logistic Regression for Banknote Authentication.ipynb".
The first task is:
Many machine learning and optimization algorithms also require feature scaling
for optimal performance. Here, we will standardize the features using the StandardScaler class from scikit-learn's preprocessing module:
Standardizing the features:
Can you write Python code for it?

from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
sc.fit(X_train)
X_train_std = sc.transform(X_train)
X_test_std = sc.transform(X_test)


I am working on the file "Classification/Logistic Regression for Banknote Authentication.ipynb".
The first task is:
Our classifier's precision is 0.988; almost all of the notes that it predicted as
authentic were actually authentic. Its recall is also high, indicating that it correctly classified
approximately 98 percent of the authentic messages as authentic.
Calculating the F1 measure
The F1 measure is the harmonic mean, or weighted average, of the precision and
recall scores. Also called the f-measure or the f-score, the F1 score is calculated using
the following formula:
F1 = 2PR / (P + R)
The F1 measure penalizes classifiers with imbalanced precision and recall scores,
like the trivial classifier that always predicts the positive class. A model with perfect
precision and recall scores will achieve an F1 score of one. A model with a perfect
precision score and a recall score of zero will achieve an F1 score of zero. As for
precision and recall, scikit-learn provides a function to calculate the F1 score for
a set of predictions. Let's compute our classifier's F1 score.
Can you write Python code for it?

f1s = cross_val_score(lr, X_train_std, y_train, cv=5, scoring='f1')
print('F1', np.mean(f1s), f1s)


I am working on the file "Classification/Logistic Regression for Banknote Authentication.ipynb".
The first task is:
The arithmetic mean of our classifier's precision and recall scores is 0.98. As the
difference between the classifier's precision and recall is small, the F1 measure's
penalty is small. Models are sometimes evaluated using the F0.5 and F2 scores,
which favor precision over recall and recall over precision, respectively.
ROC AUC
A Receiver Operating Characteristic, or ROC curve, visualizes a classifier's performance. Unlike accuracy, the ROC curve is insensitive to data sets with unbalanced class proportions; unlike precision and recall, the ROC curve illustrates the classifier's performance for all values of the discrimination threshold. ROC curves plot the classifier's recall against its fall-out. Fall-out, or the false positive rate, is the number of false positives divided by the total number of negatives. It is
calculated using the following formula:
F = FP / (TN + FP)
Can you write Python code for it?

from sklearn.metrics import roc_auc_score, roc_curve, auc
roc_auc_score(y_test,lr.predict(X_test_std))


I am working on the file "Classification/Logistic Regression for Banknote Authentication.ipynb".
The first task is:
The confusion matrix indicates that there were 227 true negative predictions, 180
true positive predictions, 0 false negative predictions, and 5 false positive
prediction.
Scikit-learn also implements a large variety of different performance metrics that are available via the metrics module. For example, we can calculate the classification accuracy of the perceptron on the test set as follows:
Can you write Python code for it?

from sklearn.metrics import accuracy_score
print('Accuracy: %.2f' % accuracy_score(y_test, y_pred))


I am working on the file "Classification/Logistic Regression for Banknote Authentication.ipynb".
The first task is:
To evaluate how well a trained model performs on unseen data, we will further split the dataset into separate training and test datasets. Splitting data into 70% training and 30% test data:
Can you write Python code for it?

from sklearn.cross_validation import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
         X, y, test_size=0.3, random_state=0)


I am working on the file "Classification/Logistic Regression for Banknote Authentication.ipynb".
The first task is:
Training a logistic regression model with scikit-learn
Scikit-learn implements a highly optimized version of logistic regression that also supports multiclass settings off-the-shelf, we will skip the implementation and use the sklearn.linear_model.LogisticRegression class as well as the familiar fit method to train the model on the standardized flower training dataset:
Can you write Python code for it?

from sklearn.linear_model import LogisticRegression

lr = LogisticRegression(C=1000.0, random_state=0)

lr.fit(X_train_std, y_train)

y_test.shape


I am working on the file "Classification/Support Vector Machines for Energy-Efficiency Classification.ipynb".
The first task is:
As you can see via the df.describe() method, the mean for our target variables Y1 and Y2 is approx 22 and 24 respectively. Since the data for y1 and y2 is continuous, we will use the following piece of code to convert it into either high or low.
We do this by defining all those y1 and y2 values above or 20 are high and those below 20 are low. We then map 1 for high and 0 for low.
Can you write Python code for it?

# handle Y1 and Y2 attrubte to binary [High Hl and CL; and Low HL and Cl]
high = df.Y1 >= 20
low = df.Y1 < 20
df.loc[high,'Y1'] = 1
df.loc[low,'Y1'] = 0

high2 = df.Y2 >= 20
low2 = df.Y2 < 20
df.loc[high2,'Y2'] = 1
df.loc[low2,'Y2'] = 0

df.head()


I am working on the file "Classification/Support Vector Machines for Energy-Efficiency Classification.ipynb".
The first task is:
As you can see we achieved 88 percent accuracy for classifying both Y1 and Y2 using the LinearSVC variant of SVM implementation in Scikit-Learn.

Logistic regression versus SVM
In practical classification tasks, linear logistic regression and linear SVMs often yield very similar results. Logistic regression tries to maximize the conditional probabilities of the training data, which makes it more prone to misclassifying abnormal data points or outliers whereas this is not the case with SVMs. The SVMs try to find a the decision boundary or support vectors that are farthest from the closest points in average. Logistic regression has the advantage over SVMs due to it's simpler implementation. Furthermore, logistic regression models can be easily updated, which is useful when working with streaming data.

Solving non-linear problems using a kernel SVM
Another reason why SVMs are highly popular among machine learning practitioners is that they can be used to solve nonlinear classification problems by what are called as kernels. Before we discuss the main concept behind kernel SVM, let's first define and create a sample dataset to see how such a nonlinear classification problem may look.
Using the following code, we will create a simple dataset that has the form of
an XOR gate using the logical_xor function from NumPy, where 100 samples
will be assigned the class label 1 and 100 samples will be assigned the class
label -1, respectively:
Can you write Python code for it?

import matplotlib.pyplot as plt
import numpy as np

np.random.seed(0)
X_xor = np.random.randn(200, 2)
y_xor = np.logical_xor(X_xor[:, 0] > 0,
                       X_xor[:, 1] > 0)
y_xor = np.where(y_xor, 1, -1)

plt.scatter(X_xor[y_xor == 1, 0],
            X_xor[y_xor == 1, 1],
            c='b', marker='x',
            label='1')
plt.scatter(X_xor[y_xor == -1, 0],
            X_xor[y_xor == -1, 1],
            c='r',
            marker='s',
            label='-1')

plt.xlim([-3, 3])
plt.ylim([-3, 3])
plt.legend(loc='best')
plt.tight_layout()
# plt.savefig('./figures/xor.png', dpi=300)
plt.show()


I am working on the file "Classification/Support Vector Machines for Energy-Efficiency Classification.ipynb".
The first task is:
For Classification into Y2
Can you write Python code for it?

%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

from sklearn.ensemble import ExtraTreesClassifier

# Build a classification task using 3 informative features

# Build a forest and compute the feature importances
forest = ExtraTreesClassifier(n_estimators=250,
                              random_state=0)

forest.fit(X, y2)
importances = forest.feature_importances_
std = np.std([tree.feature_importances_ for tree in forest.estimators_],
             axis=0)
indices = np.argsort(importances)[::-1]

# Print the feature ranking
print("Feature ranking:")

for f in range(X.shape[1]):
    print("%d. feature %d - %s (%f) " % (f + 1, indices[f], features[indices[f]], importances[indices[f]]))

# Plot the feature importances of the forest
plt.figure(num=None, figsize=(14, 10), dpi=80, facecolor='w', edgecolor='k')
plt.title("Feature importances")
plt.bar(range(X.shape[1]), importances[indices],
       color="r", yerr=std[indices], align="center")
plt.xticks(range(X.shape[1]), indices)
plt.xlim([-1, X.shape[1]])
plt.show()

for f in range(5):
    print("%d. feature %d - %s (%f)" % (f + 1, indices[f], features[indices[f]] ,importances[indices[f]]))

best_features = []
for i in indices[:5]:
    best_features.append(features[i])
# Plot the top 5 feature importances of the forest
plt.figure(num=None, figsize=(8, 6), dpi=80, facecolor='w', edgecolor='k')
plt.title("Feature importances")
plt.bar(range(5), importances[indices][:5],
       color="r",  yerr=std[indices][:5], align="center")
plt.xticks(range(5), best_features)
plt.xlim([-1, 5])
plt.show()


I am working on the file "Classification/Support Vector Machines for Energy-Efficiency Classification.ipynb".
The first task is:
Now we have converted the dataset into a binary classification task. We can observe that the data for y1 and y2 is either 0 or 1, i.e the Energy Efficiency is low if its 0 or high if its 1.
Can you write Python code for it?


I am working on the file "Classification/Support Vector Machines for Energy-Efficiency Classification.ipynb".
The first task is:
So for both Y1 and Y2, X4 and X5 are the most important features.

Maximum margin classification with Support Vector Machines
Support Vector Machines(SVM) is a powerful and widely used learning algorithm. In other classification approaches we aim to minimize the classification error. Whereas in SVMs, our approach to acheive optimization is to maximize the margin. Here the margin is defined as the distance between the separating hyperplane also called as the decision boundary and the training samples that are closest to this hyperplane, which are the so-called support vectors. This can be understood from the following figure:
<img src="images/svm.png">
Dealing with the nonlinearly separable case using slack variables
Scikit-Learn implementation of SVMs provide us parameters to fine tune our algorithm implementions. These paramaters are explained according to Scikit-Learn documentation as follows:

The gamma parameter : Intuitively, the gamma parameter defines how far the influence of a single training example reaches, with low values meaning 'far' and high values meaning 'close'. The gamma parameters can be seen as the inverse of the radius of influence of samples selected by the model as support vectors.

The C parameter : The C parameter trades off misclassification of training examples against simplicity of the decision surface. A low C makes the decision surface smooth, while a high C aims at classifying all training examples correctly by giving the model freedom to select more samples as support vectors.

You can better understand the use of these parameters in plotting the SVM classification from this link.
Essentially using the variable C, we can control the penalty for misclassification error. Large values of C account for large error penalties whereas smaller values for C means we are less strict about misclassification. We can then we use the parameter C to control the 'width' of the margin and therefore tune the bias-variance trade-off as illustrated in the following figure:
Can you write Python code for it?


I am working on the file "Classification/Support Vector Machines for Energy-Efficiency Classification.ipynb".
The first task is:
Using the default implementation of SVM in Scikit-Learn, we achieved 98 percent accuracy for classifying Y1 i.e Heating load as high or low and 95 percent accuracy for classiying Y2 i.e Cooling load.
Can you write Python code for it?

from time import time
from sklearn.svm import LinearSVC

t7=time()
print ("SVM")

svc = LinearSVC(C=1.0, random_state=0)
svc2 = LinearSVC(C=1.0, random_state=0)

clf_svc=svc.fit(X_train, y_train)
clf_svc2=svc2.fit(X_train, y_train2)

print ("Acurracy for Y1: ", clf_svc.score(X_test,y_test))
print ("Acurracy for Y2: ", clf_svc2.score(X_test2,y_test2))

t8=time()
print ("time elapsed: ", t8-t7)


I am working on the file "Classification/Support Vector Machines for Energy-Efficiency Classification.ipynb".
The first task is:
We specify our features and target variables:
Can you write Python code for it?

features=list(df.columns[:-2])

X = df[features]
y1 = df['Y1']
y2 = df['Y2']

y2.unique()

X.shape, y1.shape, y2.shape


I am working on the file "Classification/Support Vector Machines for Energy-Efficiency Classification.ipynb".
The first task is:
cross validation for SVM
Can you write Python code for it?

tt7=time()
print ("cross result========")
scores = cross_validation.cross_val_score(svc, X,y1, cv=5)
scores2 = cross_validation.cross_val_score(svc2, X,y2, cv=5)
print ("For Y1")
print (scores )
print (scores.mean())
print ("For Y2")
print (scores2)
print (scores2.mean())
tt8=time()
print ("time elapsed: ", tt8-tt7)


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Answer: positive
Now compute the accuracy of the majority class classifier on test_data.
Quiz Question: Enter the accuracy of the majority class classifier model on the test_data. Round your answer to two decimal places (e.g. 0.76).
Can you write Python code for it?

num_positive  = (test_data['sentiment'] == +1).sum()
num_negative = (test_data['sentiment'] == -1).sum()
print num_positive
print num_negative

28095 / (28095 + 5241)


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Aside. You may get an warning to the effect of "Terminated due to numerical difficulties --- this model may not be ideal". It means that the quality metric (to be covered in Module 3) failed to improve in the last iteration of the run. The difficulty arises as the sentiment model puts too much weight on extremely rare words. A way to rectify this is to apply regularization, to be covered in Module 4. Regularization lessens the effect of extremely rare words. For the purpose of this assignment, however, please proceed with the model above.
Now that we have fitted the model, we can extract the weights (coefficients) as an SFrame as follows:
Can you write Python code for it?

weights = sentiment_model.coefficients
weights.column_names()


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Checkpoint: Make sure your class predictions match with the one obtained from GraphLab Create.
Probability predictions
Recall from the lectures that we can also calculate the probability predictions from the scores using:
$$
P(y_i = +1 | \mathbf{x}_i,\mathbf{w}) = \frac{1}{1 + \exp(-\mathbf{w}^T h(\mathbf{x}_i))}.
$$
Using the variable scores calculated previously, write code to calculate the probability that a sentiment is positive using the above formula. For each row, the probabilities should be a number in the range [0, 1].
Can you write Python code for it?

def calculateProbs(score):
    return 1 / (1 + math.exp(-score))

probabilities = scores.apply(calculateProbs)
print probabilities


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Checkpoint: Make sure your probability predictions match the ones obtained from GraphLab Create.
Can you write Python code for it?

print "Class predictions according to GraphLab Create:"
print sentiment_model.predict(sample_test_data, output_type='probability')


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Extract sentiments
We will ignore all reviews with rating = 3, since they tend to have a neutral sentiment.
Can you write Python code for it?

products = products[products['rating'] != 3]
len(products)


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
For each review, we will use the word_count column and trim out all words that are not in the significant_words list above. We will use the SArray dictionary trim by keys functionality. Note that we are performing this on both the training and test set.
Can you write Python code for it?

train_data['word_count_subset'] = train_data['word_count'].dict_trim_by_keys(significant_words, exclude=False)
test_data['word_count_subset'] = test_data['word_count'].dict_trim_by_keys(significant_words, exclude=False)


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Let's dig deeper into the first row of the sample_test_data. Here's the full review:
Can you write Python code for it?


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Let's see what the first example of the dataset looks like:
Can you write Python code for it?


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Let's sort the coefficients (in descending order) by the value to obtain the coefficients with the most positive effect on the sentiment.
Can you write Python code for it?


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Now, compute the classification accuracy of the simple_model on the train_data:
Can you write Python code for it?


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Now, let us explore what the sample example above looks like after these 2 transformations. Here, each entry in the word_count column is a dictionary where the key is the word and the value is a count of the number of times the word occurs.
Can you write Python code for it?


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Now, we will assign reviews with a rating of 4 or higher to be positive reviews, while the ones with rating of 2 or lower are negative. For the sentiment column, we use +1 for the positive class label and -1 for the negative class label.
Can you write Python code for it?

products['sentiment'] = products['rating'].apply(lambda rating : +1 if rating > 3 else -1)
products


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Now, we will perform 2 simple data transformations:

Remove punctuation using Python's built-in string functionality.
Transform the reviews into word-counts.

Aside. In this notebook, we remove all punctuations for the sake of simplicity. A smarter approach to punctuations would preserve phrases such as "I'd", "would've", "hadn't" and so forth. See this page for an example of smart handling of punctuations.
Can you write Python code for it?

def remove_punctuation(text):
    import string
    return text.translate(None, string.punctuation)

review_without_puctuation = products['review'].apply(remove_punctuation)
products['word_count'] = graphlab.text_analytics.count_words(review_without_puctuation)


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Predicting sentiment
These scores can be used to make class predictions as follows:
$$
\hat{y} =
\left{
\begin{array}{ll}
      +1 & \mathbf{w}^T h(\mathbf{x}_i) > 0 \
      -1 & \mathbf{w}^T h(\mathbf{x}_i) \leq 0 \
\end{array}
\right.
$$
Using scores, write code to calculate $\hat{y}$, the class predictions:
Can you write Python code for it?

predictions = scores.apply(lambda x: 1 if x > 0 else -1)
print predictions


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Quiz Question: Are the positive words in the simple_model (let us call them positive_significant_words) also positive words in the sentiment_model?
Can you write Python code for it?

positive_significant_words = simple_model.coefficients[simple_model.coefficients['value'] > 0]
sentiment_model.coefficients.filter_by(positive_significant_words['index'], 'index').print_rows(num_rows=21)


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Quiz Question: Which model (sentiment_model or simple_model) has higher accuracy on the TEST set?
Baseline: Majority class prediction
It is quite common to use the majority class classifier as the a baseline (or reference) model for comparison with your classifier model. The majority classifier model predicts the majority class for all data points. At the very least, you should healthily beat the majority class classifier, otherwise, the model is (usually) pointless.
What is the majority class in the train_data?
Can you write Python code for it?

num_positive  = (train_data['sentiment'] == +1).sum()
num_negative = (train_data['sentiment'] == -1).sum()
print num_positive
print num_negative


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Quiz Question: Which model (sentiment_model or simple_model) has higher accuracy on the TRAINING set?
Now, we will repeat this excercise on the test_data. Start by computing the classification accuracy of the sentiment_model on the test_data:
Can you write Python code for it?


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Quiz Question: Which of the following products are represented in the 20 most negative reviews?  [multiple choice]
Compute accuracy of the classifier
We will now evaluate the accuracy of the trained classifer. Recall that the accuracy is given by
$$
\mbox{accuracy} = \frac{\mbox{# correctly classified examples}}{\mbox{# total examples}}
$$
This can be computed as follows:

Step 1: Use the trained model to compute class predictions (Hint: Use the predict method)
Step 2: Count the number of data points when the predicted class labels match the ground truth labels (called true_labels below).
Step 3: Divide the total number of correct predictions by the total number of data points in the dataset.

Complete the function below to compute the classification accuracy:
Can you write Python code for it?

def get_classification_accuracy(model, data, true_labels):
    # First get the predictions
    predictions = model.predict(data)

    # Compute the number of correctly classified examples
    num_correct = 0
    for i in xrange(len(predictions)):
        if predictions[i] == true_labels[i]:
            num_correct = num_correct + 1

    # Then compute accuracy by dividing num_correct by total number of examples
    accuracy = num_correct / data.num_rows()

    return accuracy


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
That review seems pretty positive.
Now, let's see what the next row of the sample_test_data looks like. As we could guess from the sentiment (-1), the review is quite negative.
Can you write Python code for it?


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
Train a sentiment classifier with logistic regression
We will now use logistic regression to create a sentiment classifier on the training data. This model will use the column word_count as a feature and the column sentiment as the target. We will use validation_set=None to obtain same results as everyone else.
Note: This line may take 1-2 minutes.
Can you write Python code for it?

sentiment_model = graphlab.logistic_classifier.create(train_data,
                                                      target = 'sentiment',
                                                      features=['word_count'],
                                                      validation_set=None)

sentiment_model


I am working on the file "Classification/Week 1/module-2-linear-classifier-assignment-blank.ipynb".
The first task is:
We will now make a class prediction for the sample_test_data. The sentiment_model should predict +1 if the sentiment is positive and -1 if the sentiment is negative. Recall from the lecture that the score (sometimes called margin) for the logistic regression model  is defined as:
$$
\mbox{score}_i = \mathbf{w}^T h(\mathbf{x}_i)
$$
where $h(\mathbf{x}_i)$ represents the features for example $i$.  We will write some code to obtain the scores using GraphLab Create. For each row, the score (or margin) is a number in the range [-inf, inf].
Can you write Python code for it?

scores = sentiment_model.predict(sample_test_data, output_type='margin')
print scores


I am working on the file "Classification/Week 2/Assignment 1/module-3-linear-classifier-learning-assignment-blank.ipynb".
The first task is:
Are you running this notebook on an Amazon EC2 t2.micro instance? (If you are using your own machine, please skip this section)
It has been reported that t2.micro instances do not provide sufficient power to complete the conversion in acceptable amount of time. For interest of time, please refrain from running get_numpy_data function. Instead, download the binary file containing the four NumPy arrays you'll need for the assignment. To load the arrays, run the following commands:
arrays = np.load('module-3-assignment-numpy-arrays.npz')
feature_matrix, sentiment = arrays['feature_matrix'], arrays['sentiment']
Can you write Python code for it?