Package: icecube-skywriter
skywriterUpstream Tools for SkyDriver & the Skymap Scanner.I3 to JSON converterThe main tool provided byskywriteris thei3_to_jsonconverter function and script. The function reads a (list of) I3 file(s), processes them using the the realtime alert followup code (AlertEventFollowupmodule) and writes the events out as JSON files.As SkyMap Scanner is designed to process theSplitUncleanedInIcePulsespulse series, the converter generates it by running theI3TriggerSplittermodule on the original DAQ (Q) frame, hence creating one or more new Physics (P) frames (one for each "subevent" in the data). If already present, theSplitUncleanedInIcePulsesobject is deleted beforehand. In case of multiple subevents, only the subevent-id corresponding matching the original P-frame is processed.TheAlertEventFollowupmodule expects the Physics I3Frame to have a given set of keys. Such set of keys is currently hardcoded in the converter code. If the keys are present in the original P-frame, the corresponding objects are copied over to the output JSON. If not, they are filled with dummy values. If you plan to use these keys in your own Skymap Scanner reconstruction module make sure they have meaningful values.ExtraI3Particleobjects, typically corresponding to reconstruction outputs, can be "extracted" through the--extraoption. This means that the equatorial coordinates are calculated and written out in a JSON branch.GCDIn order to produce lightweight payloads, theAlertEventFollowupcode uses the so called "GCD diff" functionality (sometimes called "GCD compress"). The input GCD is compared against a given "base GCD" and only the difference is stored in the JSON-encoded GCD, along with the filename of the base GCD.Skymap Scanner will look up a matching filename of the base GCD file in a pre-defined set of locations. In principle, it should not matter much which base GCD is used as long as Skymap Scanner can retrieve it. If you use a custom base GCD you should make sure it is made available to Skymap Scanner.Example usageAssuming you have set up your environment and loaded an IceTray shell, you can run the converter as (for example):python skywriter/i3_to_json.py --extra "OnlineL2_SplineMPE" /data/ana/realtime/alert_catalog_v2/input_files/Level2pass2_IC86.2011_data_Run00118435_Subrun00000000_00000144_event58198553.i3this will use the default base GCD.Known limitationsThe converter always writes out JSON files and does not allow to simply obtain JSON-encoded string corresponding to the input event. Yet you may treat output files as ephemeral, and delete them after the scan.The converter supports the reading of a sequence of I3 files, but it is mostly tested with a single input file. If you encounter problems with converting multiple files, please file an issue. More in general, rather than producing sets of JSON files for archival, it could be better to do any conversion on-the-fly and track the conversion options in the user code.It is not possible to copy an arbitrary key/object from the input P-frame to the output P-frame. There is also no warranty that a particle extracted with the--extraoption will have a corresponding object in the output P-frame. If you need such a feature, or you want to improve this behavior, feel free to file a pull request!Events in the input frame(s) are currently identified byrun_id,event_id,subevent_id, which in turn are used to name the output JSON files. There is no warranty that these are unique in simulation events. Note that while Skymap Scanner does not care about the event being scanned having a unique id, SkyDriver currently identifies events byrun_idandevent_idonly (although in principle, you could still scan different events with identicalrun_idandevent_id, it is just a matter of bookkeeping and information retrieval). The best approach would be to ensure that sets of simulated events processed with SkyDriver are assigned unique(run_id, event_id).