Started the process of rewiring Sepp to use the new abstract base class and data containers.
Added to data classes, which now inherit from np.ndarray in the case of cartesian point data.
TODO: rename to cartesian instead of euclidean?
TODO: need to supply a new interface to KDE class that will work with the new data types.
TODO: need to refactor plotting calls because can now ONLY evaluate a flat array so will need to reshape meshgrid output etc.
Further rewiring progress
Fixed KDE tests, started implementing in SEPP.  TODO: need a decoupled KDE class now unfortunately.
Tried unsuccessfully to setup Django testing WITHOUT db creation for,e.g., data tests which require no DB.
Added to KDE code integration with data models.  Improved data models.
TODO: 3 KDE tests failing, fix these.
TODO: check that flatten() calls won't have a bad effect on runtime.
Partway through refactoring (AGAIN) the KDE code.
Added a 'separate' method to the DataArray class to spew out ndarrays with original shape
current issue: pickle LOSES the additional attribute that stores this, so there is no 'original_shape' attribute to load again.
TODO: investiagte how pickle serializes an np.ndarray and overload, if possible.  If not then may need a hack to the parallel code that transfers original_shape back again.  Yuck.
Fixed a lot of tests, fixed pickling of data class.  Finished separable KDE class with tests.
BUT have broken the SEPP code!  Some kind of norming issue - have left printing statements in to attempt to fix this up.
Fixed the refactoring for now?
Simulation validation now runs fine.  Problem was a bug in the actual kernel.pdf!!
Added tests for kde models and kernels.
Added to cad SEPP code.
Started looking at network code - fixed a few errors I had introduced by removing the tag 'linestring'
Added several new KDE models - one-sided in time normal and exponential in time, normal in space.
These *don't* seem to fix the issue when validating with simulated data - was expecting a good match close to t=0 but still a fair way out.  Maybe the best possible solution is a normal kernel that REFLECTS at t=0?  Will be slower!  TODO: give it a go.
TODO: having issues getting this working properly on the Cad data again - why is that?  Seeing very very tight spatial extents.
Fixed broken code in one sided kernel.  TODO: add tests for it.
Started working on a REFLECTIVE kernel - TODO: finish.
Added models-level KDE code for reflective kernel, but think that the kernel-level implementation will be nicer.  So finish kernel code then plug in to model.
Tried switching CAD SEPP application over to different KDE model.  Haven't got far enogh to show results yet.
Finished new kernels code for one sided and reflection in time.  Started writing a new generic test suite with tests that every kernel should pass - norming, expected values, correct calls.
Modified simulation / cad / chicago runners to use new SEPP models.  All seem to work OK on simulated data (reflection is best? but very similar).
Fixed data ordering in chicago data getter.
Added implementation code to chicago to systematically investigate parameter effects.
Started shifting validation code to use new SEPP classes (TODO: finish)
Plotting command that centres y axis on zero
Added dill to requirements
Increased npts in plotting routines.  Added controlling kwarg.  Added func to make a data array from separate components.  TODO: consider adding to data class itself.
Modified prediction code - hadn't used it up until now.  Turns out that there was a bug in the data model involving original_shape.  The 'fix' is a bit hacky and horrible - added a 'getrows' fn to dDataArray that strips the original shape.  Otherwise reshape eventually gets called with nonsense shape params.
Modified surface plotting code to take this into account.
Added TEMPORARY testing file to check that code output is scale-invariant as expected.  TODO: add this to tests and delete this file.
FIXME: chicago run seems to be failing in implement_delta_effect - very narrow x triggering and all triggering seems to be in y-direction... Can't think why this is happening, but doesn't *appear* to be a property of the data.  It switches dimension if the input data are swapped, so not a simle bug.
Think it might be a norming issue, but can't spot anything out of place.  Try Dividing y data by 10 to get on same scale, then see what happens?
Changed data model implementation for cartesian etc...
Now array is a member of the class rather than the class itself.
Advantage: can explicitly deal with original shape issues
Disadvantage: need to add all operations manually.
Think this will overall be better, especially given the limited number of operations required??
Updated tests accordingly TODO: need to add more for more advanced classes.
Updated kernels code to use these new classes.  Created generic norming tests.  Failing on reflective case - why??
Merge branch 'rewire_Sepp' of https://github.com/gaberosser/crime-fighter into rewire_Sepp
Added more tests for data model and kde kernels.  Nice new suite for kernels in particular.  All tests seem OK, but need to check that.
Updated linkage function in SEPP utils module.  Function now uses getrows() method of data object, since direct indexing is not reliable.
TODO: see kde kernels normcdf function and call - this is currently broken as scipy.special.erf doesn't work on new data object.
Added to kernel tests.  Though I'm now thinking that it might be nicer if these worked with an np.ndarray rather than the data object, since we have to prep_input EVERY time, rather than just once... Sigh.
The runner is working again, and Chicago is producing vaguely sensible looking results - TODO: check the x/y discrepancy again.
Populating Chicago data was broken due to missing SRID - fixed by switching to a different SRID that is identical.
Added KDE model for deterministic nn.  Can't make this work though.
Added weighted_kde properties to SeppStochastic for both trigger and bg.  These return the FULL weighted KDE, as in the deterministic model.
TODO: can we actually just point the set_kdes command to this in the deterministic model??
Started adding commands to test convergence... Currently they are to be found in SEPP models, but may want to move to implementation module or similar?  Also started adding plotting.
Added similar to runner, am halfway through shifting them TODO: finish moving them.
TODO: finish copying to plotting as well.
Added support for start and end date filters in aggregation class