Project: datamanagement
Issue DM-3777: server side preparation for  histogram plot (2)
1. On the initialization, server needs to return the following summary table for all numeric columns: a. column name / description / unit b. min and max values c. number of points 2. For a given column and binning options, return the table of bins: a. first column - numInBin - number of points in a bin b. second column - binMin - bin's lower bound c. third column - binMax - bin's upper bound [10-13-15] Added by LZ Here is the detailed requirement from Tatiana: The implementation of this ticket are two search processors (similar to IpacTableFromSource search processor). Both search processors should be located in edu.caltech.ipac.firefly.server.query package and produce an IPAC Table. 1. Getting table statistics INPUT: The input to the first search processor should be TableServerRequest (treq) with "searchRequest" parameter set. The searchRequest parameter will be a serialized JSON string, which can represent a search request to any other search processor. This parameter determines how the input IPAC table is obtained. (The input IPAC table is produced similarly to how IpacTableFromSource produces the result with "processor" parameter set.) To parse SearchRequest parameter into key-value pairs - the parameters for the search request - you can use json.simple library: JSONObject searchRequestJSON = (JSONObject)JSONValue.parse(treq.get("searchRequest")); if (searchRequestJSON != null) { for (Object param : searchRequestJSON.keySet()) { String name = (String)param; String value = (String)searchRequestJSON.get(param); } } One of the parameters should be the "id" (ServerReuqest.ID_KEY) - which tells which search processor to use to obtain input IPAC table. OUTPUT: The output IPAC table produced by the first search processor should contain 5 columns: columnName, description, unit (unit or empty string), min, max, numPoints (the number of non-null values). 2. Getting histogram data INPUT: The input to the second processor should be TableServerRequest (treq) with "searchRequest" parameter set (exactly the same as "searchRequest" parameter in item 1) plus binningOptions. The binningOptions parameter will be serialized JSON object with the following keys (object properties): { "algorithm" : "fixedSizeBins", "binSize" : 20, "column" : "wavelength", "scale" : "linear", "min" : 0, "max" :100 } Notes: -the algorithm is simple fixed size binning for now, in future we'll need to add variable size bins algorithm described here: http://www.astroml.org/examples/algorithms/plot_bayesian_blocks.html - at the beginning, you can assume that column is column name, but it can be an expression based on several columns. - scale can be "linear" or "log" - min and max define filter on the points included into histogram calculation, any of them or both can be absent. If both are absent, no filter needs to be applied. OUTPUT: The output IPAC table produced by the second search processor should contain 3 columns: numInBin, binMin, binMax in that order. The table should be sorted by binMin. The binMax of row i should be always less or equal to binMin of row i+1. If number of points in the bin is 0, it's OK to skip this bin.
Story points: 12