## 3 ZTF Data Processing Overview
### IRSA Archive  
IRSA/IPAC curates and distributes the images and catalogs. Presently, there are around 6.9 million single-exposure images, 135,000 co-added images, 106 billion source catalog files, and 2 billion light curves generated from single-exposure extractions with already released catalogs [35]. The Image data are available in the Flexible Image Transport System (FITS) format as linked open data, which includes science images, calibration images, as well as the metadata of the calibration images, raw images, and reference images, which reference the actual images [23; 36]. The ZTF archive web directory structure is shown in Figure 2. The IRSA preserves the ZTF FITS file in the B-tree data structure defined in prior work in their web directory, as in Table 2. The root node of the IRSA web directory is called the top node, and the index is stored in the leaf node. The leaf nodes store the ZTF FITS image data where all insertions and updates occur.  
The \(i\)th record contains an absolute identity as \(x_{i}\) to identify the base record. The components with \(x_{i}\) are year, two-digit month, and two-digit day, fractional day, field, filtercode, CCDID, image type code, quadrant ID, as shown in Figure 2. There are approximately 106 billion records in the ZTF archive. Each record has up to 1 billion data unit fields, which are identified as \(y_{i,j}(y_{i,1}.\ldots.\ y_{i,1 billion})\) where \((i,j)\) indicate the record identity \((x_{i})\) and position of the field \(((y_{ij})\) in the individual record.  
In the Figure 2,  
* /raw = raw image data file;
* /cal = calibration product file;
* /sci = epochal science product file and difference images;
* /ref = reference image (co-adds) and catalog files;
* \(<\)fff\(>\) = first three (leftmost) digits of \(<\)fieldID\(>\);
* \(<\)caltype\(>\) = calibration type, e.g., \"bias\", \"hifreflat\";
* \(<\)imgtype\(>\) = single character label in raw camera image file;
* \(<\)ptype\(>\) = product type suffix string from pipeline;
* YYYY = year;
* MMDD = month and day (all UT-based);
* dddddd = fractional time of day of exposure (all UT-based);
* fieldID = 6-digit survey field ID if targeted science (on-sky) exposure, otherwise \"00000\" for calibrations;
* filterID = 2-letter filter code:zg, zr, zi for exposure acquired in g, R, or i respectively (for on-sky & flat-fields) bi, dk for bias and dark images, respectively (filter neutral);
* ccdID = 2-digit detector chip ID: 01, 02,... 16;
* quadID = 1-digit quadrant ID in ccd: 1, 2, 3, or 4 imgtype o: on-sky object observation or science exposure, b: bias calibration image, d: dark calibration image, f: dome/screen flatfield calibration image, c: focus image, g: guider image;
* ptype = fits, txt format image product type.  
\begin{table}
\begin{tabular}{c c} \hline
**Header** & **Data Unit** \\ \hline \hline \multirow{2}{*}{\(x_{i},y_{ij}\), Size and index for the data (1,... 50 million)} & \multicolumn{1}{c}{\(y_{ij}\)} \\ \cline{2-3}  & Name, size of Data & Data type \\ \cline{2-3}  & Night, field, etc. & FITS, Log... \\ \hline \hline \end{tabular}
\end{table}
Table 2: ZTF FITS file index in web archive.  
Figure 2: ZTF Archive.  
The API uses HTTP URLs to search and retrieve ZTF Image Metadata, Access Control, and ZTF Lightcurve Data. Users can construct the URL to obtain the image data product using the file path patterns described on the ZTF Metadata Page. Access Control is only for nonpublic data, which require a password. Access Control pages have general instructions for using the IRSA APIs with a password [37]. ZTF Lightcurve Data can be queried or retrieved using the IRSA ZTF-LC-API form of HTTP URLs via Specify ZTF objects by their identifiers (ID), by position (POS), by a collection of ZTF files (COLLECTION), by the format of output table (FORMAT) parameter [38]. The IRSA/IPAC directory has image URLs link indexed to support ZTF FITS files with API supports for visualization of images. This makes it easy to embed the archive directly in the user's software.