These commits represent a near full rewrite of blipp, as well as adding AA support, generalized scheduling, unit tests, a new configuration scheme that ties into UNIS, and other small things. add test package and cpu probe test https://uisapp2.iu.edu/jira-prd/browse/GEMINI-91

cleaning up unused imports, commented code, etc

modifying tox to run tests, ethtool dependency was causing issues, so I removed it for the time being https://uisapp2.iu.edu/jira-prd/browse/GEMINI-91

minor cleanup

start ping probe, update blippnotes

work on ping probe, add settings file

more work on ping and testing ping

adding more tests to ping... still not fully functional

comment the cod ethat dynamically reloads settings files... referencing __file__ was causing issues, and settings needs to change anyway

filling out ping settings and refactoring cpu settings slightly

minor reformatting in sched_obj

add new style blipp config file - the .annotated one has js comments explaining parts of it, but python's json module did not parse the comments properly

updated my notes

new config stuff more or less working - not tied into actual blipp yet, changes to unis_client may have broken the rest of blipp

updating gitignore files to clean up git status

add some conf tests, and a bunch of refactoring

add files starting with # or .# to .gitignore

conf and test_conf pass pep8 (or close enough)

cleaning up whitespace, adding some logging

add some integration tests for unis_client, and some more general methods for get/post/delete

add blipp specific configuration class, with test for expanding blipp's probe configuration - also fix _reconcile_config to be non-destructive

new scheduling is defined by generator methods whose names and arguments are passed in through config. A simple schedule method that just increments the next time to run by 'every' seconds is included

arbiter is the new sched, probeRunner is the new sched_obj - conf through arbiter is working and tested

minor enhancements to arbiter and runner

start turning notes into legitimate documentation

completely revamp collector and shrink scope of its responsibilities also add tests for it

Tons of changes/fixes

Completely refactored conf.py to hopefully make more sense, and handle all cases properly
Added new http.py which all network activity will eventually go through
Started trying live tests, and fixed a slew of bugs in a variety of files - not done yet
Added mytestconfig.json, which is the config file I'm using for testing right now.

more changes and bugfixes for conf - seems to work well now

notes updates - closer to being in line with reality

few more changes to conf, and getting testing in order for conf and probe_runner

oops - here are the probe runner tests

clearing out old source

collector should report when num_collected is equal to the reporting param... not greater

empirically, it just seems right for http response of 4XX to return None to the application - makes the code look clean. also requests likes its kwargs labelled

this is the direction conf handling is going - merge a default dict with all the other sources

streamlining the *_client(s) since config is already in the service format, we can just use 'post'

a simple config file I've been testing with - I'll post a full annotated example soon

Streamlining unis client as per https://uisapp2.iu.edu/jira-prd/browse/GEMINI-97
add tests for unis client and clean up other stuff I found along the way, including a workaround for https://uisapp2.iu.edu/jira-prd/browse/GEMINI-98 in conf.py

slight notes updates

updated, annotated sample configuration

couple conf bugs I discovered while writing up the config file

some notes updates for the demo on 2/5

updated 2/5 demo notes

example helper script to clear out old service definitions so they don't muck up testing

streamline ms client and add unit tests https://uisapp2.iu.edu/jira-prd/browse/GEMINI-97

remove create_collections from collector and corresponding tests... functionality is now in ms_client

notes updates

adding AA was more complicated than initially expected

Since auth stuff had to be added to every request, i created GeminiClient which processes requests from
unis and ms clients. It adds headers, auth, schema info, and encodes/decodes json. Now there is almost no code duplication between the ms and unis client. I also needed to change UNISInstance to take in a config, which should have been done before - that caused a ripple of changes throughout anything that instantiated a UNISInstance.

add a todo in notes

some fixes for auth and a test

expanded probe configs now have all fields under properties.configurations, and have a properties attribute with all other fields (aside from configurations) that are under properties

was removing stuff from probe_settings... I'm just going to commit the changes and remove them entirely in the next commit

remove settings files - all config goes through the config dict now

quick fix for http request logging

formatted printing in start_blipp logging

fix bug in ms_client

fix eventType reporting https://uisapp2.iu.edu/jira-prd/browse/GEMINI-109

cleaning up conf to be more straightforward (I hope) https://uisapp2.iu.edu/jira-prd/browse/GEMINI-108

updated notes

add merge_dicts and delete_nones util methods to replace reconcile_config

remove reconcile_config and use utils.merge_dicts everywhere, add profiling example to start_blipp
update entry point in setup.py to new place
fix a couple blipp bugs

fix bug in gemini_client - posting to metadata needs to add auth_uuid to a different location and this was not being detected correctly

fix problem in conf - GETting a particular service id can still return a list of services - handle this
switch from requests to httplib
reverting httplib changes... going to try to get requests to work with verify=False
fix so that requests doesn't try to verify server cert if no cafile is passed in
default ssl_cafile to empty string which should then pass verify=False
update .gitignore
change use_ssl to use json boolean instead of string
organize sample and test configs a bit more
when pushing data, metadata url is unis url, not ms url
add utility method for constructing a unis query from a dict
fix bugs in query_string_from_dict
change unis_client to query for existing metadata before posting new metadata
try to make refresh_config behave a bit more sanely
rewriting net's get_data method - still need to rewrite getting subjects and hook that in
add ability to pass full url, or just path to gemini_client
net probe looks up ports in UNIS and matches with what it finds on the host instead of always creating new ports
join dead probes when they are noticed
bug in mem where short strings weren't always switched for full eventTypes before returning data