Created base system for spawning IRC bots

Refers to issue #16
Broken commit, attempt at making a UNIX socket file for cross-process communication
Skeleton code in place for possibly using a 'UNIX socket' as a standard HTTP web socket
Added modified version of websocket-client/websocket.py to support UNIX sockets
Added listener for 'UNIX websocket' events
Implemented a system for spawning IRC bots, with bidirectional
communication between processes

Refers to issue #16

Changes:
    modules/irc.py
    modules/server.py
    pyrt

New:
    modules/rpchandler.py
    proc/

modules/irc.py:
 - Added class RPCResponse
  * allows easy lookups of RPC responses
 - Added function _ModularBot.shutdown
  * enables clean shutdown on SIGTERM
 - Added function _ModularBot.RPCCommand
  * encapsulates a general method of JSON-encoding a command with an
    arbritrary number of arguments of keyword arguments
 - Added function _ModularBot._ssend
  * private function for use by RPCCommand
  * sends JSON-encoded message to the RPC socket, and waits for a
  * reply (blocks for 3 seconds before returning None)
 - Modifications to _ModularBot.__init__ function
  * signal handler for SIGTERM (calls shutdown function)
  * initalise RPC socket
  * write a PID file to proc/bots
 - Modifications to Irc class
  * enabled arbritrary config options to be sent as keyword
    arguments, these will be available to the IRC bot via
    self.config

modules/server.py:
 - Import new module modules.rpchandler
 - Pass filepath of RPC UNIX socket to Irc instance to allow future
   configuration of this
 - Initialise modules.rpchandler.RPCHandler for message parsing /
   handling
 - Pass all messages to RPCHandler for processing, and write any
   responses directly back to the socket

pyrt:
 - Added function _searchBotPID()
  * returns a list of PIDs as defined in the proc/bots directory
 - Modified path for the server PID to proc/pyrt.pid
 - Modified the stop function to kill any running IRC bots before
   shutting down the server
 - Modified the stop function to remove the RPC socket before exiting

modules/rpchandler.py:
 - Provides the modules.rpchandler.RPCHandler class
 - Defines allowed RPC methods and performs them

proc/:
 - directory for holding all running process PIDs for clean shutdowns
General changes for testing purposes
Secured RPC interface from outside interference
Reworked handle function to correctly pass arguments\n\nConsequently, this enables auto-start for drag 'n' drop uploads
Reworked handle function to correctly pass arguments

This has the direct consequence of enabling auto-start for drag and drop
uploads as originally intended
Merge branch 'testing' of github.com:mountainpenguin/pyrt into testing
Basic implementation of 'remote fetch'

Concerns issue #12 (adding torrents) and issue #16 (IRC auto)

Changed files:
    modules/ajaxPage.py
    modules/server.py

New files:
    modules/remotes.py
    modules/sites/__init__.py (PACKAGE)
    modules/sites/ptp.py

Created a basic remote fetch implementation.
Intended to allow the provision of a consistent, and simple framework
for downloading torrents from any source, making additional sources
simple to implement.

modules/remotes.py:
> Contains three major classes:
>   Base, Settings, RemoteStorage
>
> Base:
>   Provides a base class for setting up remote source 'templates'
>   Contains two functions which must be overriden; initialise and fetch
>
>    `initalise` allows the remote source name, and base_url to be set
>    for sources that require persistent cookies, site logins and
>    persistent cookie sessions could be acquired here
>
>    `fetch` defines the meat of the remote operation, it should be
>    overriden to return a tuple of (filename, torrentcontents)
>
>   It provides several other functions which are not intended to be
>   overriden; GET, POST, getFilename, and process
>
>   `process` is intended to be used after fetch to save the torrent
>   file and tell rtorrent to load it
>
>   See docstrings for more info
>
> Settings:
>   Provides a semi-implemented recursive dictionary object that acts
>   both as a normal python dictionary, and as a class
>   Nested structures such as '{"a" : {"b" : {"c" : "d"} } }' are
>   accessible both by the simple 'a.b.c' (which returns 'd') or by standard
>   a["b"]["c"]
>
> RemoteStorage (not yet implemented):
>   Intended to provide a persistent (and secure!) record of pertinent
>   information for remote sources (e.g. authkeys and passwords),
>   theoretically these might be shareable across the app, allowing
>   source classes to request them when needed

modules/sites/ptp.py:
> Source class for passthepopcorn
> Test case for running a remote fetch

modules/ajaxPage.py:
> Added new function 'load_from_remote' which simply runs rtorrent
> methods start_from_file or load_from_file with some log messages
>
> Also added a possible bug fix for the root refresh when drop_downs are
> open

modules/server.py:
> Added error handling (and suppression) for all ajax calls over
> ajaxSocket, allowing sockets to remain open if AJAX calls fail
>
> Added the request handler 'testing' that tests a remote fetch from PTP
> using arguments definedas GET parameters
Filled out RemoteStorage class
Changed import / as statements to be more verbose
Remote fetch via IRC first implementation

Closes issue #12 and progresses issue #16

New Files:
> modules/autohandler.py
> modules/sites/test.py
> static/images/RSS.png
> static/images/irc.png
> static/images/submit.png

Modified Files:
> .gitignore
> modules/irc.py
> modules/login.py
> modules/remotes.py
> modules/rpchandler.py
> modules/server.py
> modules/sites/ptp.py
> modules/weblog.py
> htdocs/autoHTML.tmpl
> static/javascript/auto.js
> static/scss/auto.scss (==> /css/auto.css)
> static/scss/main.scss (==> /css/main.css)

modules/autohandler.py
> Provides responses for web socket requests via /auto
>
> AutoHandler main class is initialised on every websocket request in
> modules/server.py
>
> Provides (currently) three methods: get_sources, set_source, and
> start_bot
> These are accessed via the handle_messages method
>
> get_sources:
>  calls modules/remotes.py searchSites method (see below)
>  formats and returns HTML to be inserted into /auto
>  called on every page load of /auto
>
> set_source:
>  calls modules/remotes.py RemoteStorage addRemote method (see commit
>  f17d4d and below)
>  in summary: adds appropriate settings required for remote fetches to
>  persistent storage
>
> handle_messages:
>  parses, verifies, and executes autoSocket commands

modules/sites/test.py
> Example 'source' handler

.gitignore
> Ensured that the .remotes.pickle file isn't committed

modules/irc.py
> Changed the _Config class to be a remotes.Settings object (subclass of
>  dict)
> Added a very basic fetch mechanism:
>  if an integer is announced in the channel, it will send this as a
>  torrentid to the ptp fetch mechanism

modules/login.py
> Added the new function getPermSalt:
>  returns the salt from the user password hash

modules/remotes.py
> Added two new functions: searchSites, and getSiteMod
> These are both static methods
>
> searchSites:
>  searches the modules/sites/ directory for any python files
>  attempts to load them, and obtains the attributes DESCRIPTION and
>   REQUIRED_KEYS
>  it then returns these as a list
>  this function is currently accessed by the modules/autohandler.py
>   AutoHandler get_source method
>
> getSiteMod:
>  this returns the named 'source' module, loaded and ready to go
>  (note: the module Main class still has to be instanced)
>
> Base class modifications:
>  Modified the Base class to also require shared object storage upon
>  initalisation, this allows direct access to required authentication
>  strings during remote operations
>
>  Added the private _fetch_keywords function for obtaining settings
>  stored in RemoteStorage, this function is run after initalise
>
>  Added the post_init function designed for overriding which is executed
>  after _fetch_keywords, this is intended to be used for 'source's that
>  require fetching of cookies for downloading, the settings that are
>  stored are loaded into the _required_keys attribute of settings
>
> Modified methods for RemoteStorage class
>  Removed the requirement for specifying a base_url in addRemote, this
>  should be defined within the 'source' handler module directly
>
>  Removed the concept of 'remoteid's, now refer to everything by 'name'
>  (also indirectly removes the possibility that two settings could be
>  stored for the same handler)

modules/rpchandler.py
> Added the reqirement for passing the shared log, ajax, and storage
> instances, this is so that they can be passed onto 'source' handler
> objects when they are initialised
>
> Added the 'fetchTorrent' method:
>  this loads the site module referred to by name using the
>  modules/remotes.py getSiteMod method
>  it then runs fetch, and process
>  thus this method allows an IRC bot to trigger the application to
>  download a torrent remotely from an arbritray source

modules/server.py
> Removed the testing IRC bot spawning for autoSocket to use the
> autohandler handle_message method

modules/sites/ptp.py
> Renamed the class to Main to allow a consistent interface
> All subclasses of remotes.Base should be named 'Main'
> Removed the necessity of specifying authkey and torrent_pass during
> initalisation, they are now directly fetched from RemoteStorage

modules/weblog.py
> HTML escape all log message (encodes &, < and > as HTML entities)

htdocs/autoHTML.tmpl
> Added jquery UI to scripts (for modal dialogs)
> Added crypting scripts for hashing
> Removed the RSS, and Web tabs (will be implemented later)
> Made the Auto tab refresh the page
> Changed IRC id to #tab_irc (Note this tab now spawns an IRC bot, see
>  auto.js)
> Removed IRC, RSS and web wrappers
> Added HTML for an interface to available 'source' handlers as defined
>  in modules/sites/ (see modules/remotes.py above)
> Added a modal dialog for submitting a password field

static/javascript/auto.js
> Allows the setting of anything that might be required by 'source'
> handler modules by using the modules/remote.py searchSites method (via
> autosocket and autohandler)
>
> Placed in structural code ready for reversible encryption of these
> settings, but haven't implemented it
>
> Added various UI elements to the main /auto page, allowing clicking of
> table rows to reveal settings "drop-down" menus
>
> Added very basic validation of setting inputs (currently only checks
> if they are non-empty)
>
> Added simple initial request for 'source' handler module information
> (get_sources method)
>
> Added basic error handling for ERROR responses (ignore them)
> Modified the navigate_tab function to simply refresh the page

static/scss/auto.scss (==> static/css/auto.css)
> Added a whole bunch of styling

static/scss/main.scss (==> static/css/main.css)
> Changed selectors that interfere with the /auto page styling
Changed IRC tab to be a 'suboption' rather than a full option
Implemented managing bots, and a system for regex filters

Refers to issue #16

Modified files:
> htdocs/autoHTML.tmpl
> modules/autohandler.py
> modules/irc.py
> modules/remotes.py
> modules/rpchandler.py
> modules/sites/ptp.py
> modules/sites/test.py
> static/scss/auto.scss (==> static/css/auto.css)
> static/javascript/auto.js

htdocs/autoHTML.tmpl
> Removed form from modal dialog to prevent accidental submission

modules/autohandler.py
> Added methods: get_source_single, stop_bot,
> get_filters, add_filter, and remove_filter
>
> get_source_single
>  Performs the same role as get_sources, but for a single name (should
>  try to unify both of these under a single function in the future)
>
> stop_bot
>  Sends a SIGTERM to a running bot instance
>
> get_filters
>  Returns stored filters for given handler name, or an empty list
>
> add_filter
>  Validates and adds a regex filter for given handler name
>
> remove_filter
>  Removes a regex filter from given handler name (NOTE: not persistent,
>  will be fixed in next commit)
>
> Modified 'remote_row' status <td> to be easily referrable by handler
> name
>
> Added two items to drop downs, filters and botcontrols
> These allow viewing, and editing of regex filters and starting and
> stopping IRC bots
>
> Added the requirement to specify key descriptions for 'required_keys'
> which are displayed as placeholders (see modules/sites/ptp.py below)
>
> Modified start_bot function to check for errors when starting IRC
> instances

modules/irc.py
> Added an update function to fetch any changes in defined handler
> filters, this is called on every pubmsg in defined channels
>
> Fixed shutdown sequence to really remove PID files, and to report this
> back to the main process
>
> Added handling for nicknameinuse and erroneousnickname server
> responses (response is to die)
>
> Added registration and deregistration with the main process, allowing
> other components of the app to hook into running instances
>
> Added sending of arbritrary startup commands as defined in the handler
> module file
>
> Implented regex matching of PRIVMSGs to the configured channel, and
> parsing of a torrent id from this based on the handler module file
> defined regex
> This torrent id is then passed back to the main process for
> downloading
>
> Made the Irc class fully configurable via the attributes of the
> handler module file

modules/remotes.py
> Added new functions: addFilter, removeFilter, registerBot,
> deregisterBot, isBotActive
>
> addFilter
>  Adds a regex filter to storage, which has been compiled into a re
>  object already
>
> removeFilter
>  Removes a filter from storage (NOTE: this will be updated in the next
>  commit to actually save changes to disk)
>
> registerBot
>  Stores the PID of a running IRC bot, this is not persistent storage,
>  all bots should be dead between restarts
>
> deregisterBot
>  Removes the PID of a dying IRC bot
>
> isBotActive
>  Returns the PID of a running bot instance based on the given handler
>  name
>
> Forced all names to be accessed, and stored, in uppercase

modules/rpchandler.py
> Added new methods: register, deregister, and get_filters
>
> register
>  Calls remotes.registerBot
>
> deregister
>  Calls remotes.deregisterBot
>
> get_filters
>  Returns a list of filters for a given handler name

modules/sites/ptp.py
> Added significant changes to approach
>
> REQUIRED_KEYS atribute of the module now must be a list of tuples, the
> first element being the required key, and the second being a
> description of that key
>
> Added the METHODS attribute of the module
> This defines which auto method(s) should be used for auto-downloading,
> this currently can only be IRC, but RSS and WEB downloads are planned
>
> If the IRC method is specified, four more attributes must be defined:
> IRC_NETWORK
>  the network address that the bot should connect to
> IRC_PORT
>  the port that the bot should connect to
> IRC_CHANNEL
>  the announce channel that the bot should join
> IRC_MATCH
>  a compiled re object that will match an announce message, and extract
>  the relevant information as the first matched group, any post
>  processing can be defined in the fetch function of the Main class
>
> A fifth attribute can be optionally defined
> IRC_COMMANDS
>  This is a list of RAW IRC messages that can be sent directly after
>  the bot has connected, for example, this might include identifying to
>  Services

static/javascript/auto.js
> Significant changes
> Added communication of multiple commands / actions with autohandler
>
> Clicking a bot_button will start or stop a bot instance
>
> Clicking an add_filter button will add that filter
>
> Clicking on an existing filter will confirm, and then remove that
> filter
>
> Added auto refreshing of the drop_down on certain command responses,
> via two new functions: refresh_drop_down, and
> refresh_drop_down_respond
>
> refresh_drop_down
>  Sends a request for a new drop_down using the autohandler
>  get_source_single method
>
> refresh_drop_down_respond
>  Async callback for the refresh_drop_down, actually changes the DOM

static/scss/auto.scss (==> static/css/auto.css)
> A whole load of styling changes for all the new features on /auto
Removed testing code and fixed remotes bug

modules/remotes.py
> Fixed bug where removed filters were not persistently removed (were
> not being written to disk)
Added new handler 'what.py' and fixed bugs

Refers to issue #16

Fixed errors caused when no filters are defined

Added handler 'what.py' based on 'ptp.py'
Fixed drop down refresh error

Closes #26