#18 Scatter plots

This pull request is for adding a feature to generate scatter plots. It includes the following things.

1) A `ScatterPlotSysTest` class in `sys_tests.py`, which is a base class inherited by specific scatter plot classes for three quantitates for test (g1, g2, or sigma) times two quantities for y-axis (a star quantity or a residual quantity), in total 6 classes whose names are `ScatterPlot[Star, Residual]VsPSF[G1, G2, Sigma]SysTest`.

2) An interface to the HSC pipeline is added in `hsc/sys_test_adapters.py`. Each test can be run by adding `ScatterPlotStarVsPSFG1`, `ScatterPlotStarVsPSFG2`, `ScatterPlotStarVsPSFSigma`, `ScatterPlotResidualVsPSFG1`, `ScatterPlotResidualVsPSFG2`, or `ScatterPlotResidualVsPSFSigma` to `sys_test` in the config. Examples of a command line are as follows
- single CCD
```
StileCCD.py $SUPRIME_DATA_DIR --id visit=1228 ccd=49 --rerun [rerun_name]:[stile_rerun_name] --clobber-config -c "sys_tests.names=['ScatterPlotStarVsPSFG1', 'ScatterPlotResidualVsPSFG1']"
```
- multiple CCDs
```
StileVisit.py $SUPRIME_DATA_DIR --id visit=1228 ccd=49^50 --rerun [rerun_name]:[stile_rerun_name] --clobber-config -c "sys_tests.names=['ScatterPlotStarVsPSFG1', 'ScatterPlotResidualVsPSFG1']"
```
.  For multiple CCDs, mean and std is calculated for each CCD, and linear regression is performed on these quantities. One can disable this feature by adding an option `scatterplot_per_ccd=False`. I found that sometimes linear regression on the mean and std for each CCD and that on all stars (without using the mean and std) is significantly different. Which should we believe? I guess the latter, but I would like to hear what you think.
I looked into this, and found that the reason why y-range looks okay is there is error bars. Thus I changed the code such that the limit is set when yerr is None. Does this fix look okay?


On 2014/08/27, at 21:50, msimet <notifications@github.com> wrote:

> In stile/sys_tests.py:
> 
> > +                raise TypeError('lim should be ((xmin, xmax), (ymin, ymax)).')
> > +            elif z is not None and len(lim) !=3:
> > +                raise TypeError('lim should be ((xmin, xmax), (ymin, ymax), (zmin, zmax)).')
> > +            xlim = lim[0]
> > +            ylim = lim[1]
> > +            if z is not None:
> > +                zlim = lim[2]
> > +        # calculate n-sigma limits around mean if lim is float
> > +        elif isinstance(lim, float):
> > +            p = lim
> > +            xlim = (numpy.percentile(x, 50.-0.5*p), numpy.percentile(x, 50.+0.5*p))
> > +            ylim = (numpy.percentile(y, 50.-0.5*p), numpy.percentile(y, 50.+0.5*p))
> > +            if z is not None:
> > +                zlim = (numpy.percentile(z, 50.-0.5*p), numpy.percentile(z, 50.+0.5*p))
> > +        # in other cases (except for the default value None), raise an exception
> > +        elif lim is not None:
> It will be a little bit of work, but I wonder if we should explicitly set limits even when lim is None. I noticed that a lot of the plots this function generates look uneven on the x-axis: matplotlib will pick round numbers for the endpoints (eg -0.2 and 0.2) but that doesn't always look good.
> 
> I think the easiest thing to do is to pick some number p, say 0.05, and set xlim to be (xmin-p*(xmax-xmin),xmax+p*(xmax-xmin)) -- so the x-axis spans the x-range, with some extra fraction of the x-range in padding. The y-range didn't look uneven in the same way to me, so you could leave that, I think.
> 
> I don't think this is necessary, but it might make the plots look a little neater. Anyone else have any thoughts on this?
> 
> $B!=(B
> Reply to this email directly or view it on GitHub.
>