Remove matplotlib

@moorepants Check this one, if setup.py is correct. Then It is good to merge!