total_unmasked_sparse_pixels = unmasked_sparse_grid_pixel_centres.shape[<NUM_LIT:0>]<EOL>unmasked_sparse_to_sparse = np.zeros(total_unmasked_sparse_pixels)<EOL>pixel_index = <NUM_LIT:0><EOL>for unmasked_sparse_pixel_index in range(total_unmasked_sparse_pixels):<EOL><INDENT>y = unmasked_sparse_grid_pixel_centres[unmasked_sparse_pixel_index, <NUM_LIT:0>]<EOL>x = unmasked_sparse_grid_pixel_centres[unmasked_sparse_pixel_index, <NUM_LIT:1>]<EOL>unmasked_sparse_to_sparse[unmasked_sparse_pixel_index] = pixel_index<EOL>if not mask[y, x]:<EOL><INDENT>if pixel_index < total_sparse_pixels - <NUM_LIT:1>:<EOL><INDENT>pixel_index += <NUM_LIT:1><EOL><DEDENT><DEDENT><DEDENT>return unmasked_sparse_to_sparse<EOL>

Determine the mapping between every pixelization-grid pixel and masked pixelization-grid pixel. This is
    performed by checking whether each pixelization-grid pixel is within the regular-masks, and mapping the indexes.

    Pixelization pixels are paired with the next masked pixel index. This may mean that a pixel is not paired with a
    pixel near it, if the next pixel is on the next row of the grid. This is not a problem, as it is only
    unmasked pixels that are referened when computing image_to_pix, which is what this array is used for.

    Parameters
    -----------
    total_sparse_pixels : int
        The total number of pixels in the pixelization grid which fall within the regular-masks.
    mask : ccd.masks.Mask
        The regular-masks within which pixelization pixels must be inside
    unmasked_sparse_grid_pixel_centres : ndarray
        The centres of the unmasked pixelization grid pixels.


total_regular_pixels = regular_to_unmasked_sparse.shape[<NUM_LIT:0>]<EOL>regular_to_sparse = np.zeros(total_regular_pixels)<EOL>for regular_index in range(total_regular_pixels):<EOL><INDENT>regular_to_sparse[regular_index] = unmasked_sparse_to_sparse[regular_to_unmasked_sparse[regular_index]]<EOL><DEDENT>return regular_to_sparse<EOL>

Using the mapping between the regular-grid and unmasked pixelization grid, compute the mapping between each regular
    pixel and the masked pixelization grid.

    Parameters
    -----------
    regular_to_unmasked_sparse : ndarray
        The index mapping between every regular-pixel and masked pixelization pixel.
    unmasked_sparse_to_sparse : ndarray
        The index mapping between every masked pixelization pixel and unmasked pixelization pixel.


total_pix_pixels = sparse_to_unmasked_sparse.shape[<NUM_LIT:0>]<EOL>pix_grid = np.zeros((total_pix_pixels, <NUM_LIT:2>))<EOL>for pixel_index in range(total_pix_pixels):<EOL><INDENT>pix_grid[pixel_index, <NUM_LIT:0>] = unmasked_sparse_grid[sparse_to_unmasked_sparse[pixel_index], <NUM_LIT:0>]<EOL>pix_grid[pixel_index, <NUM_LIT:1>] = unmasked_sparse_grid[sparse_to_unmasked_sparse[pixel_index], <NUM_LIT:1>]<EOL><DEDENT>return pix_grid<EOL>

Use the central arc-second coordinate of every unmasked pixelization grid's pixels and mapping between each
    pixelization pixel and unmasked pixelization pixel to compute the central arc-second coordinate of every masked
    pixelization grid pixel.

    Parameters
    -----------
    unmasked_sparse_grid : ndarray
        The (y,x) arc-second centre of every unmasked pixelization grid pixel.
    sparse_to_unmasked_sparse : ndarray
        The index mapping between every pixelization pixel and masked pixelization pixel.


return np.linspace(self.arc_second_minima[<NUM_LIT:0>], self.arc_second_maxima[<NUM_LIT:0>], <NUM_LIT:4>)<EOL>

Compute the yticks labels of this grid, used for plotting the y-axis ticks when visualizing an image-grid


return np.linspace(self.arc_second_minima[<NUM_LIT:1>], self.arc_second_maxima[<NUM_LIT:1>], <NUM_LIT:4>)<EOL>

Compute the xticks labels of this grid, used for plotting the x-axis ticks when visualizing an image-grid


return grid_util.grid_arcsec_1d_to_grid_pixels_1d(grid_arcsec_1d=grid_arcsec, shape=self.shape,<EOL>pixel_scales=self.pixel_scales, origin=self.origin)<EOL>

Convert a grid of (y,x) arc second coordinates to a grid of (y,x) pixel values. Pixel coordinates are \
        returned as floats such that they include the decimal offset from each pixel's top-left corner.

        The pixel coordinate origin is at the top left corner of the grid, such that the pixel [0,0] corresponds to \
        highest y arc-second coordinate value and lowest x arc-second coordinate.

        The arc-second coordinate origin is defined by the class attribute origin, and coordinates are shifted to this \
        origin before computing their 1D grid pixel indexes.

        Parameters
        ----------
        grid_arcsec: ndarray
            A grid of (y,x) coordinates in arc seconds.


return grid_util.grid_arcsec_1d_to_grid_pixel_centres_1d(grid_arcsec_1d=grid_arcsec,<EOL>shape=self.shape,<EOL>pixel_scales=self.pixel_scales,<EOL>origin=self.origin).astype('<STR_LIT:int>')<EOL>

Convert a grid of (y,x) arc second coordinates to a grid of (y,x) pixel values. Pixel coordinates are \
        returned as integers such that they map directly to the pixel they are contained within.

        The pixel coordinate origin is at the top left corner of the grid, such that the pixel [0,0] corresponds to \
        higher y arc-second coordinate value and lowest x arc-second coordinate.

        The arc-second coordinate origin is defined by the class attribute origin, and coordinates are shifted to this \
        origin before computing their 1D grid pixel indexes.

        Parameters
        ----------
        grid_arcsec: ndarray
            The grid of (y,x) coordinates in arc seconds.


return grid_util.grid_arcsec_1d_to_grid_pixel_indexes_1d(grid_arcsec_1d=grid_arcsec,<EOL>shape=self.shape,<EOL>pixel_scales=self.pixel_scales,<EOL>origin=self.origin).astype('<STR_LIT:int>')<EOL>

Convert a grid of (y,x) arc second coordinates to a grid of (y,x) pixel 1D indexes. Pixel coordinates are \
        returned as integers such that they are the pixel from the top-left of the 2D grid going rights and then \
        downwards.

        For example:

        The pixel at the top-left, whose 2D index is [0,0], corresponds to 1D index 0.
        The fifth pixel on the top row, whose 2D index is [0,5], corresponds to 1D index 4.
        The first pixel on the second row, whose 2D index is [0,1], has 1D index 10 if a row has 10 pixels.

        The arc-second coordinate origin is defined by the class attribute origin, and coordinates are shifted to this \
        origin before computing their 1D grid pixel indexes.

        Parameters
        ----------
        grid_arcsec: ndarray
            The grid of (y,x) coordinates in arc seconds.


return grid_util.grid_pixels_1d_to_grid_arcsec_1d(grid_pixels_1d=grid_pixels, shape=self.shape,<EOL>pixel_scales=self.pixel_scales, origin=self.origin)<EOL>

Convert a grid of (y,x) pixel coordinates to a grid of (y,x) arc second values.

        The pixel coordinate origin is at the top left corner of the grid, such that the pixel [0,0] corresponds to \
        higher y arc-second coordinate value and lowest x arc-second coordinate.

        The arc-second coordinate origin is defined by the class attribute origin, and coordinates are shifted to this \
        origin before computing their 1D grid pixel indexes.

        Parameters
        ----------
        grid_pixels : ndarray
            The grid of (y,x) coordinates in pixels.


return grid_util.regular_grid_1d_from_shape_pixel_scales_and_origin(shape=self.shape,<EOL>pixel_scales=self.pixel_scales,<EOL>origin=self.origin)<EOL>

The arc second-grid of (y,x) coordinates of every pixel.

        This is defined from the top-left corner, such that the first pixel at location [0, 0] will have a negative x \
        value y value in arc seconds.


return grid_util.regular_grid_2d_from_shape_pixel_scales_and_origin(shape=self.shape,<EOL>pixel_scales=self.pixel_scales,<EOL>origin=self.origin)<EOL>


arguments = vars(self)<EOL>arguments.update({"<STR_LIT>": array})<EOL>if '<STR_LIT>' in arguments:<EOL><INDENT>arguments.pop("<STR_LIT>")<EOL><DEDENT>return self.__class__(**arguments)<EOL>

Parameters
----------
array: ndarray
    An ndarray

Returns
-------
new_array: ScaledSquarePixelArray
    A new instance of this class that shares all of this instances attributes with a new ndarray.


@classmethod<EOL><INDENT>def from_fits(cls, file_path, hdu):<DEDENT>

return cls(array_util.numpy_array_2d_from_fits(file_path, hdu))<EOL>

Loads the image from a .fits file.

Parameters
----------
file_path : str
    The full path of the fits file.
hdu : int
    The HDU number in the fits file containing the image image.


<EOL>super(ScaledArray, self).__init__()<EOL>

A scaled-array is a 2D array with a pixel-scale, such that all pixels on the array has associated with \
        them arc-second coordinates.

        Parameters
        ----------
        array: ndarray
            An array representing image (e.g. an image, noise-map, etc.)
        origin : (float, float)
            The arc-second origin of the scaled array's coordinate system.


def __init__(self, array, pixel_scale, origin=(<NUM_LIT:0.0>, <NUM_LIT:0.0>)):

pixel_scale_sanity_checks(pixel_scales=(pixel_scale, pixel_scale))<EOL>self.pixel_scale = pixel_scale<EOL>self.origin = origin<EOL>super(ScaledSquarePixelArray, self).__init__(array=array)<EOL>

A scaled array with square-pixels.

        Parameters
        ----------
        array: ndarray
            An array representing image (e.g. an image, noise-map, etc.)
        pixel_scale: float
            The arc-second to pixel conversion factor of each pixel.
        origin : (float, float)
            The arc-second origin of the scaled array's coordinate system.


@classmethod<EOL><INDENT>def from_fits_with_pixel_scale(cls, file_path, hdu, pixel_scale, origin=(<NUM_LIT:0.0>, <NUM_LIT:0.0>)):<DEDENT>

return cls(array_util.numpy_array_2d_from_fits(file_path, hdu), pixel_scale, origin)<EOL>

Loads the image from a .fits file.

Parameters
----------
file_path : str
    The full path of the fits file.
hdu : int
    The HDU number in the fits file containing the image image.
pixel_scale: float
    The arc-second to pixel conversion factor of each pixel.


@classmethod<EOL><INDENT>def single_value(cls, value, shape, pixel_scale, origin=(<NUM_LIT:0.0>, <NUM_LIT:0.0>)):<DEDENT>

array = np.ones(shape) * value<EOL>return cls(array, pixel_scale, origin)<EOL>

Creates an instance of Array and fills it with a single value

Parameters
----------
value: float
    The value with which the array should be filled
shape: (int, int)
    The shape of the array
pixel_scale: float
    The scale of a pixel in arc seconds

Returns
-------
array: ScaledSquarePixelArray
    An array filled with a single value


return self.new_with_array(super(ScaledSquarePixelArray, self).flatten(order))<EOL>

Returns
-------
flat_scaled_array: ScaledSquarePixelArray
    A copy of this array flattened to 1D


def zoomed_scaled_array_around_mask(self, mask, buffer=<NUM_LIT:1>):

return self.new_with_array(array=array_util.extracted_array_2d_from_array_2d_and_coordinates(<EOL>array_2d=self,  y0=mask.zoom_region[<NUM_LIT:0>]-buffer, y1=mask.zoom_region[<NUM_LIT:1>]+buffer,<EOL>x0=mask.zoom_region[<NUM_LIT:2>]-buffer, x1=mask.zoom_region[<NUM_LIT:3>]+buffer))<EOL>

Extract the 2D region of an array corresponding to the rectangle encompassing all unmasked values.

        This is used to extract and visualize only the region of an image that is used in an analysis.

        Parameters
        ----------
        mask : mask.Mask
            The mask around which the scaled array is extracted.
        buffer : int
            The buffer of pixels around the extraction.


def resized_scaled_array_from_array(self, new_shape, new_centre_pixels=None, new_centre_arcsec=None):

if new_centre_pixels is None and new_centre_arcsec is None:<EOL><INDENT>new_centre = (-<NUM_LIT:1>, -<NUM_LIT:1>)  <EOL><DEDENT>elif new_centre_pixels is not None and new_centre_arcsec is None:<EOL><INDENT>new_centre = new_centre_pixels<EOL><DEDENT>elif new_centre_pixels is None and new_centre_arcsec is not None:<EOL><INDENT>new_centre = self.arc_second_coordinates_to_pixel_coordinates(arc_second_coordinates=new_centre_arcsec)<EOL><DEDENT>else:<EOL><INDENT>raise exc.DataException('<STR_LIT>'<EOL>'<STR_LIT>')<EOL><DEDENT>return self.new_with_array(array=array_util.resized_array_2d_from_array_2d_and_resized_shape(<EOL>array_2d=self, resized_shape=new_shape, origin=new_centre))<EOL>

resized the array to a new shape and at a new origin.

        Parameters
        -----------
        new_shape : (int, int)
            The new two-dimensional shape of the array.


def __init__(self, array, pixel_scales, origin=(<NUM_LIT:0.0>, <NUM_LIT:0.0>)):

pixel_scale_sanity_checks(pixel_scales=pixel_scales)<EOL>self.pixel_scales = pixel_scales<EOL>self.origin = origin<EOL>super(ScaledRectangularPixelArray, self).__init__(array=array)<EOL>

Parameters
----------
array: ndarray
    An array representing image (e.g. an image, noise-map, etc.)
pixel_scales : (float, float)
    The arc-second to pixel conversion factor of each pixel.
origin : (float, float)
    The arc-second origin of the scaled array's coordinate system.


@classmethod<EOL><INDENT>def single_value(cls, value, shape, pixel_scales, origin=(<NUM_LIT:0.0>, <NUM_LIT:0.0>)):<DEDENT>

array = np.ones(shape) * value<EOL>return cls(array, pixel_scales, origin=origin)<EOL>

Creates an instance of Array and fills it with a single value

Parameters
----------
value: float
    The value with which the array should be filled
shape: (int, int)
    The shape of the array
pixel_scales: (float, float)
    The scale of a pixel in arc seconds

Returns
-------
array: ScaledSquarePixelArray
    An array filled with a single value


@classmethod<EOL><INDENT>def from_fits_with_pixel_scale(cls, file_path, hdu, pixel_scales, origin=(<NUM_LIT:0.0>, <NUM_LIT:0.0>)):<DEDENT>

return cls(array_util.numpy_array_2d_from_fits(file_path, hdu), pixel_scales, origin)<EOL>

Loads the image from a .fits file.

Parameters
----------
file_path : str
    The full path of the fits file.
hdu : int
    The HDU number in the fits file containing the hyper.
pixel_scales: (float, float)
    The arc-second to pixel conversion factor of each pixel.


return self.new_with_array(super().flatten(order))<EOL>


if psf.shape[<NUM_LIT:0>] % <NUM_LIT:2> == <NUM_LIT:0> or psf.shape[<NUM_LIT:1>] % <NUM_LIT:2> == <NUM_LIT:0>:<EOL><INDENT>raise exc.KernelException("<STR_LIT>")<EOL><DEDENT>self.mask_index_array = np.full(mask.shape, -<NUM_LIT:1>)<EOL>self.pixels_in_mask = int(np.size(mask) - np.sum(mask))<EOL>count = <NUM_LIT:0><EOL>for x in range(mask.shape[<NUM_LIT:0>]):<EOL><INDENT>for y in range(mask.shape[<NUM_LIT:1>]):<EOL><INDENT>if not mask[x, y]:<EOL><INDENT>self.mask_index_array[x, y] = count<EOL>count += <NUM_LIT:1><EOL><DEDENT><DEDENT><DEDENT>self.psf = psf<EOL>self.psf_shape = psf.shape<EOL>self.psf_max_size = self.psf_shape[<NUM_LIT:0>] * self.psf_shape[<NUM_LIT:1>]<EOL>image_index = <NUM_LIT:0><EOL>self.image_frame_indexes = np.zeros((self.pixels_in_mask, self.psf_max_size), dtype='<STR_LIT:int>')<EOL>self.image_frame_psfs = np.zeros((self.pixels_in_mask, self.psf_max_size))<EOL>self.image_frame_lengths = np.zeros((self.pixels_in_mask), dtype='<STR_LIT:int>')<EOL>for x in range(self.mask_index_array.shape[<NUM_LIT:0>]):<EOL><INDENT>for y in range(self.mask_index_array.shape[<NUM_LIT:1>]):<EOL><INDENT>if not mask[x][y]:<EOL><INDENT>image_frame_indexes, image_frame_psfs = self.frame_at_coordinates_jit((x, y), mask,<EOL>self.mask_index_array,<EOL>self.psf[:, :])<EOL>self.image_frame_indexes[image_index, :] = image_frame_indexes<EOL>self.image_frame_psfs[image_index, :] = image_frame_psfs<EOL>self.image_frame_lengths[image_index] = image_frame_indexes[image_frame_indexes >= <NUM_LIT:0>].shape[<NUM_LIT:0>]<EOL>image_index += <NUM_LIT:1><EOL><DEDENT><DEDENT><DEDENT>

Class to create regular frames and blurring frames used to convolve a psf with a 1D regular of non-masked \
values.

Parameters
----------
mask : Mask
    A masks where True eliminates datas.
psf : regular.PSF or ndarray
    An array representing a PSF.


@staticmethod<EOL><INDENT>@decorator_util.jit()<EOL>def frame_at_coordinates_jit(coordinates, mask, mask_index_array, psf):<DEDENT>

psf_shape = psf.shape<EOL>psf_max_size = psf_shape[<NUM_LIT:0>] * psf_shape[<NUM_LIT:1>]<EOL>half_x = int(psf_shape[<NUM_LIT:0>] / <NUM_LIT:2>)<EOL>half_y = int(psf_shape[<NUM_LIT:1>] / <NUM_LIT:2>)<EOL>frame = -<NUM_LIT:1> * np.ones((psf_max_size))<EOL>psf_frame = -<NUM_LIT:1.0> * np.ones((psf_max_size))<EOL>count = <NUM_LIT:0><EOL>for i in range(psf_shape[<NUM_LIT:0>]):<EOL><INDENT>for j in range(psf_shape[<NUM_LIT:1>]):<EOL><INDENT>x = coordinates[<NUM_LIT:0>] - half_x + i<EOL>y = coordinates[<NUM_LIT:1>] - half_y + j<EOL>if <NUM_LIT:0> <= x < mask_index_array.shape[<NUM_LIT:0>] and <NUM_LIT:0> <= y < mask_index_array.shape[<NUM_LIT:1>]:<EOL><INDENT>value = mask_index_array[x, y]<EOL>if value >= <NUM_LIT:0> and not mask[x, y]:<EOL><INDENT>frame[count] = value<EOL>psf_frame[count] = psf[i, j]<EOL>count += <NUM_LIT:1><EOL><DEDENT><DEDENT><DEDENT><DEDENT>return frame, psf_frame<EOL>

Compute the frame (indexes of pixels light is blurred into) and psf_frame (psf kernel values of those \
        pixels) for a given coordinate in a masks and its PSF.

        Parameters
        ----------
        coordinates: (int, int)
            The coordinates of mask_index_array on which the frame should be centred
        psf_shape: (int, int)
            The shape of the psf for which this frame will be used


def __init__(self, mask, blurring_mask, psf):

if mask.shape != blurring_mask.shape:<EOL><INDENT>raise exc.KernelException("<STR_LIT>")<EOL><DEDENT>super(ConvolverImage, self).__init__(mask, psf)<EOL>blurring_mask = blurring_mask<EOL>self.pixels_in_blurring_mask = int(np.size(blurring_mask) - np.sum(blurring_mask))<EOL>image_index = <NUM_LIT:0><EOL>self.blurring_frame_indexes = np.zeros((self.pixels_in_blurring_mask, self.psf_max_size), dtype='<STR_LIT:int>')<EOL>self.blurring_frame_psfs = np.zeros((self.pixels_in_blurring_mask, self.psf_max_size))<EOL>self.blurring_frame_lengths = np.zeros((self.pixels_in_blurring_mask), dtype='<STR_LIT:int>')<EOL>for x in range(mask.shape[<NUM_LIT:0>]):<EOL><INDENT>for y in range(mask.shape[<NUM_LIT:1>]):<EOL><INDENT>if mask[x][y] and not blurring_mask[x, y]:<EOL><INDENT>image_frame_indexes, image_frame_psfs = self.frame_at_coordinates_jit((x, y), mask,<EOL>self.mask_index_array, self.psf)<EOL>self.blurring_frame_indexes[image_index, :] = image_frame_indexes<EOL>self.blurring_frame_psfs[image_index, :] = image_frame_psfs<EOL>self.blurring_frame_lengths[image_index] = image_frame_indexes[image_frame_indexes >= <NUM_LIT:0>].shape[<NUM_LIT:0>]<EOL>image_index += <NUM_LIT:1><EOL><DEDENT><DEDENT><DEDENT>

Class to create regular frames and blurring frames used to convolve a psf with a 1D regular of non-masked \
        values.

        Parameters
        ----------
        mask : Mask
            The regular masks, where True eliminates datas.
        blurring_mask : Mask
            A masks of pixels outside the masks but whose light blurs into it after PSF convolution.
        psf : regular.PSF or ndarray
            An array representing a PSF.


def convolve_image(self, image_array, blurring_array):

return self.convolve_jit(image_array, self.image_frame_indexes, self.image_frame_psfs, self.image_frame_lengths,<EOL>blurring_array, self.blurring_frame_indexes, self.blurring_frame_psfs,<EOL>self.blurring_frame_lengths)<EOL>

For a given 1D regular array and blurring array, convolve the two using this convolver.

        Parameters
        -----------
        image_array : ndarray
            1D array of the regular values which are to be blurred with the convolver's PSF.
        blurring_array : ndarray
            1D array of the blurring regular values which blur into the regular-array after PSF convolution.


def plot_image(<EOL>image, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False,<EOL>should_plot_border=False, positions=None, as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>mask_pointsize=<NUM_LIT:10>, position_pointsize=<NUM_LIT:30>, grid_pointsize=<NUM_LIT:1>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT:image>'):

origin = get_origin(array=image, plot_origin=plot_origin)<EOL>array_plotters.plot_array(<EOL>array=image, origin=origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask,<EOL>should_plot_border=should_plot_border, positions=positions, as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize, position_pointsize=position_pointsize, grid_pointsize=grid_pointsize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>

Plot the observed image of the ccd data.

    Set *autolens.data.array.plotters.array_plotters* for a description of all input parameters not described below.

    Parameters
    -----------
    image : ScaledSquarePixelArray
        The image of the data.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.
    image_plane_pix_grid : ndarray or data.array.grid_stacks.PixGrid
        If an adaptive pixelization whose pixels are formed by tracing pixels from the data, this plots those pixels \
        over the immage.


def plot_noise_map(<EOL>noise_map, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False, as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>mask_pointsize=<NUM_LIT:10>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

origin = get_origin(array=noise_map, plot_origin=plot_origin)<EOL>array_plotters.plot_array(<EOL>array=noise_map, origin=origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask, as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>

Plot the noise_map of the ccd data.

    Set *autolens.data.array.plotters.array_plotters* for a description of all input parameters not described below.

    Parameters
    -----------
    noise_map : ScaledSquarePixelArray
        The noise map of the data.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.


def plot_psf(<EOL>psf, plot_origin=True, as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

origin = get_origin(array=psf, plot_origin=plot_origin)<EOL>array_plotters.plot_array(<EOL>array=psf, origin=origin, as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>

Plot the PSF of the ccd data.

    Set *autolens.data.array.plotters.array_plotters* for a description of all input parameters not described below.

    Parameters
    -----------
    signal_to_noise_map : ScaledSquarePixelArray
        The psf of the data.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.


def plot_signal_to_noise_map(<EOL>signal_to_noise_map, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False,<EOL>as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>mask_pointsize=<NUM_LIT:10>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

origin = get_origin(array=signal_to_noise_map, plot_origin=plot_origin)<EOL>array_plotters.plot_array(<EOL>array=signal_to_noise_map, origin=origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask,<EOL>as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>

Plot the signal-to-noise_map of the ccd data.

    Set *autolens.data.array.plotters.array_plotters* for a description of all input parameters not described below.

    Parameters
    -----------
    signal_to_noise_map : ScaledSquarePixelArray
        The signal-to-noise map of the data.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.


def plot_absolute_signal_to_noise_map(<EOL>absolute_signal_to_noise_map, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False,<EOL>as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>mask_pointsize=<NUM_LIT:10>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

origin = get_origin(array=absolute_signal_to_noise_map, plot_origin=plot_origin)<EOL>array_plotters.plot_array(<EOL>array=absolute_signal_to_noise_map, origin=origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask,<EOL>as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>

Plot the absolute signal-to-noise map of the ccd data.

    Set *autolens.data.array.plotters.array_plotters* for a description of all input parameters not described below.

    Parameters
    -----------
    absolute_signal_to_noise_map : ScaledSquarePixelArray
        The absolute signal-to-noise map of the data.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.


def plot_potential_chi_squared_map(<EOL>potential_chi_squared_map, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False,<EOL>as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>mask_pointsize=<NUM_LIT:10>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

origin = get_origin(array=potential_chi_squared_map, plot_origin=plot_origin)<EOL>array_plotters.plot_array(<EOL>array=potential_chi_squared_map, origin=origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask,<EOL>as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>

Plot the signal-to-noise_map of the ccd data.

    Set *autolens.data.array.plotters.array_plotters* for a description of all input parameters not described below.

    Parameters
    -----------
    potential_chi_squared_map : ScaledSquarePixelArray
        The signal-to-noise map of the data.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.


if plot_origin:<EOL><INDENT>return array.origin<EOL><DEDENT>else:<EOL><INDENT>return None<EOL><DEDENT>

Get the (y,x) origin of the ccd data if it going to be plotted.

    Parameters
    -----------
    array : data.array.scaled_array.ScaledArray
        The array from which the origin is extracted.
    plot_origin : True
        If true, the origin of the data's coordinate system is returned.


def plot_ccd_subplot(<EOL>ccd_data, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False,<EOL>should_plot_border=False, positions=None,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=None, aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>titlesize=<NUM_LIT:10>, xlabelsize=<NUM_LIT:10>, ylabelsize=<NUM_LIT:10>, xyticksize=<NUM_LIT:10>,<EOL>mask_pointsize=<NUM_LIT:10>, position_pointsize=<NUM_LIT:30>, grid_pointsize=<NUM_LIT:1>,<EOL>output_path=None, output_filename='<STR_LIT>', output_format='<STR_LIT>'):

rows, columns, figsize_tool = plotter_util.get_subplot_rows_columns_figsize(number_subplots=<NUM_LIT:6>)<EOL>if figsize is None:<EOL><INDENT>figsize = figsize_tool<EOL><DEDENT>plt.figure(figsize=figsize)<EOL>plt.subplot(rows, columns, <NUM_LIT:1>)<EOL>plot_image(<EOL>ccd_data=ccd_data, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask, should_plot_border=should_plot_border, positions=positions, as_subplot=True,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize, position_pointsize=position_pointsize, grid_pointsize=grid_pointsize,<EOL>output_path=output_path, output_format=output_format)<EOL>plt.subplot(rows, columns, <NUM_LIT:2>)<EOL>plot_noise_map(<EOL>ccd_data=ccd_data, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask, as_subplot=True,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format)<EOL>plt.subplot(rows, columns, <NUM_LIT:3>)<EOL>plot_psf(<EOL>ccd_data=ccd_data, as_subplot=True,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>output_path=output_path, output_format=output_format)<EOL>plt.subplot(rows, columns, <NUM_LIT:4>)<EOL>plot_signal_to_noise_map(<EOL>ccd_data=ccd_data, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask, as_subplot=True,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format)<EOL>plt.subplot(rows, columns, <NUM_LIT:5>)<EOL>plot_absolute_signal_to_noise_map(<EOL>ccd_data=ccd_data, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask, as_subplot=True,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad,<EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format)<EOL>plt.subplot(rows, columns, <NUM_LIT:6>)<EOL>plot_potential_chi_squared_map(<EOL>ccd_data=ccd_data, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask, as_subplot=True,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad,<EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format)<EOL>plotter_util.output_subplot_array(output_path=output_path, output_filename=output_filename,<EOL>output_format=output_format)<EOL>plt.close()<EOL>

Plot the ccd data as a sub-plot of all its quantites (e.g. the data, noise_map-map, PSF, Signal-to_noise-map, \
     etc).

    Set *autolens.data.array.plotters.array_plotters* for a description of all innput parameters not described below.

    Parameters
    -----------
    ccd_data : data.CCDData
        The ccd data, which includes the observed data, noise_map-map, PSF, signal-to-noise_map-map, etc.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.
    image_plane_pix_grid : ndarray or data.array.grid_stacks.PixGrid
        If an adaptive pixelization whose pixels are formed by tracing pixels from the data, this plots those pixels \
        over the immage.
    ignore_config : bool
        If *False*, the config file general.ini is used to determine whether the subpot is plotted. If *True*, the \
        config file is ignored.


if should_plot_image:<EOL><INDENT>plot_image(<EOL>ccd_data=ccd_data, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask, positions=positions,<EOL>units=units,<EOL>output_path=output_path, output_format=output_format)<EOL><DEDENT>if should_plot_noise_map:<EOL><INDENT>plot_noise_map(<EOL>ccd_data=ccd_data, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask,<EOL>units=units,<EOL>output_path=output_path, output_format=output_format)<EOL><DEDENT>if should_plot_psf:<EOL><INDENT>plot_psf(<EOL>ccd_data=ccd_data, plot_origin=plot_origin,<EOL>output_path=output_path, output_format=output_format)<EOL><DEDENT>if should_plot_signal_to_noise_map:<EOL><INDENT>plot_signal_to_noise_map(<EOL>ccd_data=ccd_data, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask,<EOL>units=units,<EOL>output_path=output_path, output_format=output_format)<EOL><DEDENT>if should_plot_absolute_signal_to_noise_map:<EOL><INDENT>plot_absolute_signal_to_noise_map(<EOL>ccd_data=ccd_data, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask,<EOL>units=units,<EOL>output_path=output_path, output_format=output_format)<EOL><DEDENT>if should_plot_potential_chi_squared_map:<EOL><INDENT>plot_potential_chi_squared_map(<EOL>ccd_data=ccd_data, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask,<EOL>units=units,<EOL>output_path=output_path, output_format=output_format)<EOL><DEDENT>

Plot each attribute of the ccd data as individual figures one by one (e.g. the data, noise_map-map, PSF, \
     Signal-to_noise-map, etc).

    Set *autolens.data.array.plotters.array_plotters* for a description of all innput parameters not described below.

    Parameters
    -----------
    ccd_data : data.CCDData
        The ccd data, which includes the observed data, noise_map-map, PSF, signal-to-noise_map-map, etc.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.


def plot_image(<EOL>ccd_data, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False,<EOL>should_plot_border=False, positions=None,<EOL>as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>mask_pointsize=<NUM_LIT:10>, position_pointsize=<NUM_LIT:30>, grid_pointsize=<NUM_LIT:1>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

data_plotters.plot_image(<EOL>image=ccd_data.image, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask, should_plot_border=should_plot_border, positions=positions,<EOL>as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize, position_pointsize=position_pointsize, grid_pointsize=grid_pointsize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>

Plot the observed data of the ccd data.

    Set *autolens.data.array.plotters.array_plotters* for a description of all innput parameters not described below.

    Parameters
    -----------
    image : data.CCDData
        The ccd data, which includes the observed data, noise_map-map, PSF, signal-to-noise_map-map, etc.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.
    image_plane_pix_grid : ndarray or data.array.grid_stacks.PixGrid
        If an adaptive pixelization whose pixels are formed by tracing pixels from the data, this plots those pixels \
        over the immage.


def plot_noise_map(<EOL>ccd_data, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False, as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>, mask_pointsize=<NUM_LIT:10>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

data_plotters.plot_noise_map(<EOL>noise_map=ccd_data.noise_map, plot_origin=plot_origin, mask=mask, extract_array_from_mask=extract_array_from_mask,<EOL>zoom_around_mask=zoom_around_mask, as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>

Plot the noise_map-map of the ccd data.

    Set *autolens.data.array.plotters.array_plotters* for a description of all innput parameters not described below.

    Parameters
    -----------
    image : data.CCDData
        The ccd data, which includes the observed data, noise_map-map, PSF, signal-to-noise_map-map, etc.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.


def plot_psf(<EOL>ccd_data, plot_origin=True, as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

data_plotters.plot_psf(<EOL>psf=ccd_data.psf, plot_origin=plot_origin, as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>

Plot the PSF of the ccd data.

    Set *autolens.data.array.plotters.array_plotters* for a description of all innput parameters not described below.

    Parameters
    -----------
    image : data.CCDData
        The ccd data, which includes the observed data, noise_map-map, PSF, signal-to-noise_map-map, etc.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.


def plot_signal_to_noise_map(<EOL>ccd_data, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False, as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>mask_pointsize=<NUM_LIT:10>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

data_plotters.plot_signal_to_noise_map(<EOL>signal_to_noise_map=ccd_data.signal_to_noise_map, plot_origin=plot_origin, mask=mask,<EOL>extract_array_from_mask=extract_array_from_mask, zoom_around_mask=zoom_around_mask, as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>

Plot the signal-to-noise_map-map of the ccd data.

    Set *autolens.data.array.plotters.array_plotters* for a description of all innput parameters not described below.

    Parameters
    -----------
    image : data.CCDData
        The ccd data, which includes the observed image, noise_map-map, PSF, signal-to-noise_map-map, etc.
    plot_origin : True
        If true, the origin of the data's coordinate system is plotted as a 'x'.


def plot_absolute_signal_to_noise_map(<EOL>ccd_data, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False, as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>mask_pointsize=<NUM_LIT:10>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

data_plotters.plot_absolute_signal_to_noise_map(<EOL>absolute_signal_to_noise_map=ccd_data.absolute_signal_to_noise_map, plot_origin=plot_origin, mask=mask,<EOL>extract_array_from_mask=extract_array_from_mask, zoom_around_mask=zoom_around_mask, as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>


def plot_potential_chi_squared_map(<EOL>ccd_data, plot_origin=True, mask=None, extract_array_from_mask=False, zoom_around_mask=False, as_subplot=False,<EOL>units='<STR_LIT>', kpc_per_arcsec=None, figsize=(<NUM_LIT:7>, <NUM_LIT:7>), aspect='<STR_LIT>',<EOL>cmap='<STR_LIT>', norm='<STR_LIT>', norm_min=None, norm_max=None, linthresh=<NUM_LIT>, linscale=<NUM_LIT>,<EOL>cb_ticksize=<NUM_LIT:10>, cb_fraction=<NUM_LIT>, cb_pad=<NUM_LIT>, cb_tick_values=None, cb_tick_labels=None,<EOL>title='<STR_LIT>', titlesize=<NUM_LIT:16>, xlabelsize=<NUM_LIT:16>, ylabelsize=<NUM_LIT:16>, xyticksize=<NUM_LIT:16>,<EOL>mask_pointsize=<NUM_LIT:10>,<EOL>output_path=None, output_format='<STR_LIT>', output_filename='<STR_LIT>'):

data_plotters.plot_potential_chi_squared_map(<EOL>potential_chi_squared_map=ccd_data.potential_chi_squared_map, plot_origin=plot_origin, mask=mask,<EOL>extract_array_from_mask=extract_array_from_mask, zoom_around_mask=zoom_around_mask, as_subplot=as_subplot,<EOL>units=units, kpc_per_arcsec=kpc_per_arcsec, figsize=figsize, aspect=aspect,<EOL>cmap=cmap, norm=norm, norm_min=norm_min, norm_max=norm_max, linthresh=linthresh, linscale=linscale,<EOL>cb_ticksize=cb_ticksize, cb_fraction=cb_fraction, cb_pad=cb_pad, <EOL>cb_tick_values=cb_tick_values, cb_tick_labels=cb_tick_labels,<EOL>title=title, titlesize=titlesize, xlabelsize=xlabelsize, ylabelsize=ylabelsize, xyticksize=xyticksize,<EOL>mask_pointsize=mask_pointsize,<EOL>output_path=output_path, output_format=output_format, output_filename=output_filename)<EOL>


if data_resolution == '<STR_LIT>':<EOL><INDENT>return <NUM_LIT><EOL><DEDENT>elif data_resolution == '<STR_LIT>':<EOL><INDENT>return <NUM_LIT:0.1><EOL><DEDENT>elif data_resolution == '<STR_LIT>':<EOL><INDENT>return <NUM_LIT><EOL><DEDENT>elif data_resolution == '<STR_LIT>':<EOL><INDENT>return <NUM_LIT><EOL><DEDENT>elif data_resolution == '<STR_LIT>':<EOL><INDENT>return <NUM_LIT><EOL><DEDENT>else:<EOL><INDENT>raise ValueError('<STR_LIT>', data_resolution)<EOL><DEDENT>

Determine the pixel scale from a data resolution type based on real observations.

    These options are representative of LSST, Euclid, HST, over-sampled HST and Adaptive Optics image.

    Parameters
    ----------
    data_resolution : str
        A string giving the resolution of the desired data type (LSST | Euclid | HST | HST_Up | AO).


if data_resolution == '<STR_LIT>':<EOL><INDENT>return (<NUM_LIT:100>, <NUM_LIT:100>)<EOL><DEDENT>elif data_resolution == '<STR_LIT>':<EOL><INDENT>return (<NUM_LIT>, <NUM_LIT>)<EOL><DEDENT>elif data_resolution == '<STR_LIT>':<EOL><INDENT>return (<NUM_LIT>, <NUM_LIT>)<EOL><DEDENT>elif data_resolution == '<STR_LIT>':<EOL><INDENT>return (<NUM_LIT>, <NUM_LIT>)<EOL><DEDENT>elif data_resolution == '<STR_LIT>':<EOL><INDENT>return (<NUM_LIT>, <NUM_LIT>)<EOL><DEDENT>else:<EOL><INDENT>raise ValueError('<STR_LIT>', data_resolution)<EOL><DEDENT>

Determine the shape of an image from a data resolution type based on real observations.

    These options are representative of LSST, Euclid, HST, over-sampled HST and Adaptive Optics image.

    Parameters
    ----------
    data_resolution : str
        A string giving the resolution of the desired data type (LSST | Euclid | HST | HST_Up | AO).


errno = call(['<STR_LIT>', '<STR_LIT>', '<STR_LIT>'])<EOL>raise SystemExit(errno)<EOL>


data = json.dumps({<EOL>'<STR_LIT>': self.version,<EOL>'<STR_LIT>': self.service_name,<EOL>'<STR_LIT>': params,<EOL>'<STR_LIT:id>': text_type(uuid.uuid4())<EOL>})<EOL>data_binary = data.encode('<STR_LIT:utf-8>')<EOL>url_request = Request(self.service_url, data_binary, headers=self.headers)<EOL>return urlopen(url_request).read()<EOL>

Performs the actual sending action and returns the result


status_or_headers = headers = None<EOL>if isinstance(rv, tuple):<EOL><INDENT>rv, status_or_headers, headers = rv + (None,) * (<NUM_LIT:3> - len(rv))<EOL><DEDENT>if rv is None:<EOL><INDENT>raise ValueError('<STR_LIT>')<EOL><DEDENT>if isinstance(status_or_headers, (dict, list)):<EOL><INDENT>headers, status_or_headers = status_or_headers, None<EOL><DEDENT>D = json.loads(extract_raw_data_request(request))<EOL>if type(D) is list:<EOL><INDENT>raise InvalidRequestError('<STR_LIT>')<EOL><DEDENT>else:<EOL><INDENT>response_obj = self.empty_response(version=D['<STR_LIT>'])<EOL>response_obj['<STR_LIT:id>'] = D['<STR_LIT:id>']<EOL>response_obj['<STR_LIT:result>'] = rv<EOL>response_obj.pop('<STR_LIT:error>', None)<EOL>rv = jsonify(response_obj)<EOL><DEDENT>if status_or_headers is not None:<EOL><INDENT>if isinstance(status_or_headers, string_types):<EOL><INDENT>rv.status = status_or_headers<EOL><DEDENT>else:<EOL><INDENT>rv.status_code = status_or_headers<EOL><DEDENT><DEDENT>if headers:<EOL><INDENT>rv.headers.extend(headers)<EOL><DEDENT>return rv<EOL>

Converts the return value from a view function to a real
        response object that is an instance of :attr:`response_class`.


out = None<EOL>if isinstance(string, builtin_str):<EOL><INDENT>out = string<EOL><DEDENT>else:<EOL><INDENT>if PY2:<EOL><INDENT>out = string.encode(encoding)<EOL><DEDENT>else:<EOL><INDENT>out = string.decode(encoding)<EOL><DEDENT><DEDENT>return out<EOL>

Given a string object, regardless of type, returns a representation of that
string in the native string type, encoding and decoding where necessary.
This assumes ASCII unless told otherwise.


if type(value) == expected:<EOL><INDENT>return True<EOL><DEDENT>return False<EOL>

Returns whether or not ``value`` is the ``expected`` type.


def _eval_arg_type(arg_type, T=Any, arg=None, sig=None):

try:<EOL><INDENT>T = eval(arg_type)<EOL><DEDENT>except Exception as e:<EOL><INDENT>raise ValueError('<STR_LIT>'.format(arg_type, arg, sig, text_type(e)))<EOL><DEDENT>else:<EOL><INDENT>if type(T) not in (type, Type):<EOL><INDENT>raise TypeError('<STR_LIT>'.format(repr(T), arg, sig))<EOL><DEDENT>return T<EOL><DEDENT>

Returns a type from a snippit of python source. Should normally be
    something just like 'str' or 'Object'.

        arg_type            the source to be evaluated
        T                         the default type
        arg                     context of where this type was extracted
        sig                     context from where the arg was extracted

    Returns a type or a Type


def _parse_sig(sig, arg_names, validate=False):

d = SIG_RE.match(sig)<EOL>if not d:<EOL><INDENT>raise ValueError('<STR_LIT>'.format(sig))<EOL><DEDENT>d = d.groupdict()<EOL>ret = [(n, Any) for n in arg_names]<EOL>if text_type('<STR_LIT>') in d and type(d['<STR_LIT>']) in string_types and d['<STR_LIT>'].strip():<EOL><INDENT>for i, arg in enumerate(d['<STR_LIT>'].strip().split('<STR_LIT:U+002C>')):<EOL><INDENT>_type_checking_available(sig, validate)<EOL>if text_type('<STR_LIT:=>') in arg:<EOL><INDENT>if not type(ret) is OrderedDict:<EOL><INDENT>ret = OrderedDict(ret)<EOL><DEDENT>dk = KWARG_RE.match(arg)<EOL>if not dk:<EOL><INDENT>raise ValueError('<STR_LIT>'.format(arg, sig))<EOL><DEDENT>dk = dk.groupdict()<EOL>if not sum([(k in dk and type(dk[k]) in string_types and bool(dk[k].strip()))<EOL>for k in ('<STR_LIT>', '<STR_LIT>')]):<EOL><INDENT>raise ValueError('<STR_LIT>'.format(arg, sig))<EOL><DEDENT>ret[dk['<STR_LIT>']] = _eval_arg_type(dk['<STR_LIT>'], None, arg, sig)<EOL><DEDENT>else:<EOL><INDENT>if type(ret) is OrderedDict:<EOL><INDENT>raise ValueError('<STR_LIT>'<EOL>'<STR_LIT>'.format(sig))<EOL><DEDENT>if len(ret) < i + <NUM_LIT:1>:<EOL><INDENT>ret.append((text_type(i), _eval_arg_type(arg, None, arg, sig)))<EOL><DEDENT>else:<EOL><INDENT>ret[i] = (ret[i][<NUM_LIT:0>], _eval_arg_type(arg, None, arg, sig))<EOL><DEDENT><DEDENT><DEDENT><DEDENT>if not type(ret) is OrderedDict:<EOL><INDENT>ret = OrderedDict(ret)<EOL><DEDENT>return (d['<STR_LIT>'],<EOL>ret,<EOL>(_eval_arg_type(d['<STR_LIT>'], Any, '<STR_LIT>', sig)<EOL>if d['<STR_LIT>'] else Any))<EOL>

Parses signatures into a ``OrderedDict`` of paramName => type.
    Numerically-indexed arguments that do not correspond to an argument
    name in python (ie: it takes a variable number of arguments) will be
    keyed as the stringified version of it's index.

        sig           the signature to be parsed
        arg_names     a list of argument names extracted from python source

    Returns a tuple of (method name, types dict, return type)


if '<STR_LIT:(>' in sig:<EOL><INDENT>parts = sig.split('<STR_LIT:(>')<EOL>sig = '<STR_LIT>'.format(<EOL>parts[<NUM_LIT:0>], '<STR_LIT:U+002CU+0020>'.join(types),<EOL>('<STR_LIT:U+002CU+0020>' if parts[<NUM_LIT:1>].index('<STR_LIT:)>') > <NUM_LIT:0> else '<STR_LIT>'), parts[<NUM_LIT:1>]<EOL>)<EOL><DEDENT>else:<EOL><INDENT>sig = '<STR_LIT>'.format(sig, '<STR_LIT:U+002CU+0020>'.join(types))<EOL><DEDENT>return sig<EOL>

A function to inject arguments manually into a method signature before
    it's been parsed. If using keyword arguments use 'kw=type' instead in
    the types array.

        sig         the string signature
        types       a list of types to be inserted

    Returns the altered signature.


from flask_jsonrpc.site import jsonrpc_site<EOL>return jsonrpc_site.make_response(args)<EOL>

Sometimes it is necessary to set additional headers in a view.  Because
    views do not have to return response objects but can return a value that
    is converted into a response object by Flask itself, it becomes tricky to
    add headers to it.  This function can be called instead of using a return
    and you will get a response object which you can use to attach headers.


def jsonify_status_code(status_code, *args, **kw):

is_batch = kw.pop('<STR_LIT>', False)<EOL>if is_batch:<EOL><INDENT>response = flask_make_response(json.dumps(*args, **kw))<EOL>response.mimetype = '<STR_LIT:application/json>'<EOL>response.status_code = status_code<EOL>return response<EOL><DEDENT>response = jsonify(*args, **kw)<EOL>response.status_code = status_code<EOL>return response<EOL>

Returns a jsonified response with the specified HTTP status code.

    The positional and keyword arguments are passed directly to the
    :func:`flask.jsonify` function which creates the response.


def __init__(self, message=None, code=None, data=None):

super(Error, self).__init__()<EOL>if message is not None:<EOL><INDENT>self.message = message<EOL><DEDENT>if code is not None:<EOL><INDENT>self.code = code<EOL><DEDENT>if data is not None:<EOL><INDENT>self.data = data<EOL><DEDENT>

Setup the Exception and overwrite the default message


error = {<EOL>'<STR_LIT:name>': text_type(self.__class__.__name__),<EOL>'<STR_LIT:code>': self.code,<EOL>'<STR_LIT:message>': '<STR_LIT>'.format(text_type(self.message)),<EOL>'<STR_LIT:data>': self.data<EOL>}<EOL>if current_app.config['<STR_LIT>']:<EOL><INDENT>import sys, traceback<EOL>error['<STR_LIT>'] = traceback.format_exc()<EOL>error['<STR_LIT>'] = sys.executable<EOL><DEDENT>return error<EOL>

Return the Exception data in a format for JSON-RPC


def register(self, app, options, first_registration=False):

self.jsonrpc_site = options.get('<STR_LIT>')<EOL>self._got_registered_once = True<EOL>state = self.make_setup_state(app, options, first_registration)<EOL>if self.has_static_folder andnot self.name + '<STR_LIT>' in state.app.view_functions.keys():<EOL><INDENT>state.add_url_rule(self.static_url_path + '<STR_LIT>',<EOL>view_func=self.send_static_file,<EOL>endpoint='<STR_LIT>')<EOL><DEDENT>for deferred in self.deferred_functions:<EOL><INDENT>deferred(state)<EOL><DEDENT>

Called by :meth:`Flask.register_blueprint` to register a blueprint
        on the application. This can be overridden to customize the register
        behavior. Keyword arguments from
        :func:`~flask.Flask.register_blueprint` are directly forwarded to this
        method in the `options` dictionary.


Parses command line arguments.

    Returns:
      A Namespace of parsed arguments.


def _emit_search_criteria(user_ids, job_ids, task_ids, labels):

print('<STR_LIT>')<EOL>print('<STR_LIT>')<EOL>print('<STR_LIT>' % user_ids)<EOL>print('<STR_LIT>')<EOL>print('<STR_LIT>' % job_ids)<EOL>if task_ids:<EOL><INDENT>print('<STR_LIT>')<EOL>print('<STR_LIT>' % task_ids)<EOL><DEDENT>if labels:<EOL><INDENT>print('<STR_LIT>')<EOL>print('<STR_LIT>' % repr(labels))<EOL><DEDENT>

Print the filters used to delete tasks. Use raw flags as arguments.


<EOL>deleted_tasks, error_messages = provider.delete_jobs(<EOL>user_ids, job_ids, task_ids, labels, create_time_min, create_time_max)<EOL>for msg in error_messages:<EOL><INDENT>print(msg)<EOL><DEDENT>return deleted_tasks<EOL>

Kill jobs or job tasks.

    This function separates ddel logic from flag parsing and user output. Users
    of ddel who intend to access the data programmatically should use this.

    Args:
      provider: an instantiated dsub provider.
      user_ids: a set of user ids who "own" the job(s) to delete.
      job_ids: a set of job ids to delete.
      task_ids: a set of task ids to delete.
      labels: a set of LabelParam, each must match the job(s) to be cancelled.
      create_time_min: a timezone-aware datetime value for the earliest create
                       time of a task, inclusive.
      create_time_max: a timezone-aware datetime value for the most recent create
                       time of a task, inclusive.

    Returns:
      list of job ids which were deleted.


if args.machine_type:<EOL><INDENT>raise ValueError('<STR_LIT>'<EOL>'<STR_LIT>'<EOL>'<STR_LIT>')<EOL><DEDENT>if args.mount:<EOL><INDENT>raise ValueError('<STR_LIT>')<EOL><DEDENT>


if (args.zones and args.regions) or (not args.zones and not args.regions):<EOL><INDENT>raise ValueError('<STR_LIT>')<EOL><DEDENT>if args.machine_type and (args.min_cores or args.min_ram):<EOL><INDENT>raise ValueError(<EOL>'<STR_LIT>')<EOL><DEDENT>


Parses command line arguments.

    Args:
      prog: The path of the program (dsub.py) or an alternate program name to
      display in usage.
      argv: The list of program arguments to parse.

    Returns:
      A Namespace of parsed arguments.


Extract job-global resources requirements from input args.

    Args:
      args: parsed command-line arguments

    Returns:
      Resources object containing the requested resources for the job


def _get_job_metadata(provider, user_id, job_name, script, task_ids,<EOL>user_project, unique_job_id):

create_time = dsub_util.replace_timezone(datetime.datetime.now(), tzlocal())<EOL>user_id = user_id or dsub_util.get_os_user()<EOL>job_metadata = provider.prepare_job_metadata(script.name, job_name, user_id,<EOL>create_time)<EOL>if unique_job_id:<EOL><INDENT>job_metadata['<STR_LIT>'] = uuid.uuid4().hex<EOL><DEDENT>job_metadata['<STR_LIT>'] = create_time<EOL>job_metadata['<STR_LIT>'] = script<EOL>job_metadata['<STR_LIT>'] = user_project<EOL>if task_ids:<EOL><INDENT>job_metadata['<STR_LIT>'] = dsub_util.compact_interval_string(list(task_ids))<EOL><DEDENT>return job_metadata<EOL>

Allow provider to extract job-specific metadata from command-line args.

    Args:
      provider: job service provider
      user_id: user submitting the job
      job_name: name for the job
      script: the script to run
      task_ids: a set of the task-ids for all tasks in the job
      user_project: name of the project to be billed for the request
      unique_job_id: generate a unique job id

    Returns:
      A dictionary of job-specific metadata (such as job id, name, etc.)


def _resolve_task_logging(job_metadata, job_resources, task_descriptors):

if not job_resources.logging:<EOL><INDENT>return<EOL><DEDENT>for task_descriptor in task_descriptors:<EOL><INDENT>logging_uri = provider_base.format_logging_uri(<EOL>job_resources.logging.uri, job_metadata, task_descriptor.task_metadata)<EOL>logging_path = job_model.LoggingParam(logging_uri,<EOL>job_resources.logging.file_provider)<EOL>if task_descriptor.task_resources:<EOL><INDENT>task_descriptor.task_resources = task_descriptor.task_resources._replace(<EOL>logging_path=logging_path)<EOL><DEDENT>else:<EOL><INDENT>task_descriptor.task_resources = job_model.Resources(<EOL>logging_path=logging_path)<EOL><DEDENT><DEDENT>

Resolve the logging path from job and task properties.

    Args:
      job_metadata: Job metadata, such as job-id, job-name, and user-id.
      job_resources: Resources specified such as ram, cpu, and logging path.
      task_descriptors: Task metadata, parameters, and resources.

    Resolve the logging path, which may have substitution parameters such as
    job-id, task-id, user-id, and job-name.


def _resolve_task_resources(job_metadata, job_resources, task_descriptors):

_resolve_task_logging(job_metadata, job_resources, task_descriptors)<EOL>

Resolve task properties (such as the logging path) from job properties.

    Args:
      job_metadata: Job metadata, such as job-id, job-name, and user-id.
      job_resources: Resources specified such as ram, cpu, and logging path.
      task_descriptors: Task metadata, parameters, and resources.

    This function exists to be called at the point that all job properties have
    been validated and resolved. It is also called prior to re-trying a task.

    The only property to be resolved right now is the logging path,
    which may have substitution parameters such as
    job-id, task-id, task-attempt, user-id, and job-name.


def _wait_after(provider, job_ids, poll_interval, stop_on_failure):

<EOL>job_ids_to_check = {j for j in job_ids if j != dsub_util.NO_JOB}<EOL>error_messages = []<EOL>while job_ids_to_check and (not error_messages or not stop_on_failure):<EOL><INDENT>print('<STR_LIT>' % ('<STR_LIT:U+002CU+0020>'.join(job_ids_to_check)))<EOL>jobs_left = _wait_for_any_job(provider, job_ids_to_check, poll_interval)<EOL>jobs_completed = job_ids_to_check.difference(jobs_left)<EOL>tasks_completed = provider.lookup_job_tasks({'<STR_LIT:*>'}, job_ids=jobs_completed)<EOL>dominant_job_tasks = _dominant_task_for_jobs(tasks_completed)<EOL>if len(dominant_job_tasks) != len(jobs_completed):<EOL><INDENT>jobs_found = dsub_util.tasks_to_job_ids(dominant_job_tasks)<EOL>jobs_not_found = jobs_completed.difference(jobs_found)<EOL>for j in jobs_not_found:<EOL><INDENT>error = '<STR_LIT>' % j<EOL>print_error('<STR_LIT>' % error)<EOL>error_messages += [error]<EOL><DEDENT><DEDENT>for t in dominant_job_tasks:<EOL><INDENT>job_id = t.get_field('<STR_LIT>')<EOL>status = t.get_field('<STR_LIT>')<EOL>print('<STR_LIT>' % (str(job_id), str(status)))<EOL>if status in ['<STR_LIT>', '<STR_LIT>']:<EOL><INDENT>error_messages += [provider.get_tasks_completion_messages([t])]<EOL><DEDENT><DEDENT>job_ids_to_check = jobs_left<EOL><DEDENT>return error_messages<EOL>

Print status info as we wait for those jobs.

    Blocks until either all of the listed jobs succeed,
    or one of them fails.

    Args:
      provider: job service provider
      job_ids: a set of job IDs (string) to wait for
      poll_interval: integer seconds to wait between iterations
      stop_on_failure: whether to stop waiting if one of the tasks fails.

    Returns:
      Empty list if there was no error,
      a list of error messages from the failed tasks otherwise.


def _wait_and_retry(provider, job_id, poll_interval, retries, job_descriptor):

while True:<EOL><INDENT>tasks = provider.lookup_job_tasks({'<STR_LIT:*>'}, job_ids=[job_id])<EOL>running_tasks = set()<EOL>completed_tasks = set()<EOL>canceled_tasks = set()<EOL>fully_failed_tasks = set()<EOL>task_fail_count = dict()<EOL>message_task = None<EOL>task_dict = dict()<EOL>for t in tasks:<EOL><INDENT>task_id = job_model.numeric_task_id(t.get_field('<STR_LIT>'))<EOL>task_dict[task_id] = t<EOL>status = t.get_field('<STR_LIT>')<EOL>if status == '<STR_LIT>':<EOL><INDENT>task_fail_count[task_id] = task_fail_count.get(task_id, <NUM_LIT:0>) + <NUM_LIT:1><EOL>if task_fail_count[task_id] > retries:<EOL><INDENT>fully_failed_tasks.add(task_id)<EOL>message_task = t<EOL><DEDENT><DEDENT>elif status == '<STR_LIT>':<EOL><INDENT>canceled_tasks.add(task_id)<EOL>if not message_task:<EOL><INDENT>message_task = t<EOL><DEDENT><DEDENT>elif status == '<STR_LIT>':<EOL><INDENT>completed_tasks.add(task_id)<EOL><DEDENT>elif status == '<STR_LIT>':<EOL><INDENT>running_tasks.add(task_id)<EOL><DEDENT><DEDENT>retry_tasks = (<EOL>set(task_fail_count).difference(fully_failed_tasks)<EOL>.difference(running_tasks).difference(completed_tasks)<EOL>.difference(canceled_tasks))<EOL>if not retry_tasks and not running_tasks:<EOL><INDENT>if message_task:<EOL><INDENT>return [provider.get_tasks_completion_messages([message_task])]<EOL><DEDENT>return []<EOL><DEDENT>for task_id in retry_tasks:<EOL><INDENT>identifier = '<STR_LIT>'.format(job_id, task_id) if task_id else job_id<EOL>print('<STR_LIT>'.format(<EOL>identifier, task_fail_count[task_id]))<EOL>msg = task_dict[task_id].get_field('<STR_LIT>')<EOL>print('<STR_LIT>'.format(msg))<EOL>_retry_task(provider, job_descriptor, task_id,<EOL>task_fail_count[task_id] + <NUM_LIT:1>)<EOL><DEDENT>SLEEP_FUNCTION(poll_interval)<EOL><DEDENT>

Wait for job and retry any tasks that fail.

    Stops retrying an individual task when: it succeeds, is canceled, or has been
    retried "retries" times.

    This function exits when there are no tasks running and there are no tasks
    eligible to be retried.

    Args:
      provider: job service provider
      job_id: a single job ID (string) to wait for
      poll_interval: integer seconds to wait between iterations
      retries: number of retries
      job_descriptor: job descriptor used to originally submit job

    Returns:
      Empty list if there was no error,
      a list containing an error message from a failed task otherwise.


def _retry_task(provider, job_descriptor, task_id, task_attempt):

td_orig = job_descriptor.find_task_descriptor(task_id)<EOL>new_task_descriptors = [<EOL>job_model.TaskDescriptor({<EOL>'<STR_LIT>': task_id,<EOL>'<STR_LIT>': task_attempt<EOL>}, td_orig.task_params, td_orig.task_resources)<EOL>]<EOL>_resolve_task_resources(job_descriptor.job_metadata,<EOL>job_descriptor.job_resources, new_task_descriptors)<EOL>provider.submit_job(<EOL>job_model.JobDescriptor(<EOL>job_descriptor.job_metadata, job_descriptor.job_params,<EOL>job_descriptor.job_resources, new_task_descriptors), False)<EOL>

Retry task_id (numeric id) assigning it task_attempt.


per_job = _group_tasks_by_jobid(tasks)<EOL>ret = []<EOL>for job_id in per_job.keys():<EOL><INDENT>tasks_in_salience_order = sorted(per_job[job_id], key=_importance_of_task)<EOL>ret.append(tasks_in_salience_order[<NUM_LIT:0>])<EOL><DEDENT>return ret<EOL>

A list with, for each job, its dominant task.

    The dominant task is the one that exemplifies its job's
    status. It is either:
    - the first (FAILURE or CANCELED) task, or if none
    - the first RUNNING task, or if none
    - the first SUCCESS task.

    Args:
      tasks: a list of tasks to consider

    Returns:
      A list with, for each job, its dominant task.


ret = collections.defaultdict(list)<EOL>for t in tasks:<EOL><INDENT>ret[t.get_field('<STR_LIT>')].append(t)<EOL><DEDENT>return ret<EOL>

A defaultdict with, for each job, a list of its tasks.


<EOL>importance = {'<STR_LIT>': <NUM_LIT:0>, '<STR_LIT>': <NUM_LIT:0>, '<STR_LIT>': <NUM_LIT:1>, '<STR_LIT>': <NUM_LIT:2>}<EOL>return (importance[task.get_field('<STR_LIT>')], task.get_field(<EOL>'<STR_LIT>', datetime.datetime.max))<EOL>

Tuple (importance, end-time). Smaller values are more important.


def _wait_for_any_job(provider, job_ids, poll_interval):

if not job_ids:<EOL><INDENT>return<EOL><DEDENT>while True:<EOL><INDENT>tasks = provider.lookup_job_tasks({'<STR_LIT:*>'}, job_ids=job_ids)<EOL>running_jobs = set()<EOL>failed_jobs = set()<EOL>for t in tasks:<EOL><INDENT>status = t.get_field('<STR_LIT>')<EOL>job_id = t.get_field('<STR_LIT>')<EOL>if status in ['<STR_LIT>', '<STR_LIT>']:<EOL><INDENT>failed_jobs.add(job_id)<EOL><DEDENT>if status == '<STR_LIT>':<EOL><INDENT>running_jobs.add(job_id)<EOL><DEDENT><DEDENT>remaining_jobs = running_jobs.difference(failed_jobs)<EOL>if failed_jobs or len(remaining_jobs) != len(job_ids):<EOL><INDENT>return remaining_jobs<EOL><DEDENT>SLEEP_FUNCTION(poll_interval)<EOL><DEDENT>

Waits until any of the listed jobs is not running.

    In particular, if any of the jobs sees one of its tasks fail,
    we count the whole job as failing (but do not terminate the remaining
    tasks ourselves).

    Args:
      provider: job service provider
      job_ids: a list of job IDs (string) to wait for
      poll_interval: integer seconds to wait between iterations

    Returns:
      A set of the jobIDs with still at least one running task.


if not task_descriptors:<EOL><INDENT>return<EOL><DEDENT>task_params = task_descriptors[<NUM_LIT:0>].task_params<EOL>from_jobs = {label.name for label in job_params['<STR_LIT>']}<EOL>from_tasks = {label.name for label in task_params['<STR_LIT>']}<EOL>intersect = from_jobs & from_tasks<EOL>if intersect:<EOL><INDENT>raise ValueError(<EOL>'<STR_LIT>'<EOL>'<STR_LIT>'.format('<STR_LIT:U+002C>'.join(intersect)))<EOL><DEDENT>from_jobs = {<EOL>item.name<EOL>for item in job_params['<STR_LIT>'] | job_params['<STR_LIT>']<EOL>| job_params['<STR_LIT>']<EOL>}<EOL>from_tasks = {<EOL>item.name<EOL>for item in task_params['<STR_LIT>'] | task_params['<STR_LIT>']<EOL>| task_params['<STR_LIT>']<EOL>}<EOL>intersect = from_jobs & from_tasks<EOL>if intersect:<EOL><INDENT>raise ValueError(<EOL>'<STR_LIT>'<EOL>'<STR_LIT>'.format('<STR_LIT:U+002C>'.join(intersect)))<EOL><DEDENT>

Validates that job and task argument names do not overlap.


if args.command and args.script:<EOL><INDENT>raise ValueError('<STR_LIT>')<EOL><DEDENT>provider_base.check_for_unsupported_flag(args)<EOL>input_file_param_util = param_util.InputFileParamUtil(<EOL>DEFAULT_INPUT_LOCAL_PATH)<EOL>output_file_param_util = param_util.OutputFileParamUtil(<EOL>DEFAULT_OUTPUT_LOCAL_PATH)<EOL>mount_param_util = param_util.MountParamUtil(DEFAULT_MOUNT_LOCAL_PATH)<EOL>job_params = param_util.args_to_job_params(<EOL>args.env, args.label, args.input, args.input_recursive, args.output,<EOL>args.output_recursive, args.mount, input_file_param_util,<EOL>output_file_param_util, mount_param_util)<EOL>if args.tasks:<EOL><INDENT>task_descriptors = param_util.tasks_file_to_task_descriptors(<EOL>args.tasks, args.retries, input_file_param_util, output_file_param_util)<EOL>_validate_job_and_task_arguments(job_params, task_descriptors)<EOL><DEDENT>else:<EOL><INDENT>task_metadata = {'<STR_LIT>': None}<EOL>if args.retries:<EOL><INDENT>task_metadata['<STR_LIT>'] = <NUM_LIT:1><EOL><DEDENT>task_descriptors = [<EOL>job_model.TaskDescriptor(task_metadata, {<EOL>'<STR_LIT>': set(),<EOL>'<STR_LIT>': set(),<EOL>'<STR_LIT>': set(),<EOL>'<STR_LIT>': set()<EOL>}, job_model.Resources())<EOL>]<EOL><DEDENT>return run(<EOL>provider_base.get_provider(args, resources),<EOL>_get_job_resources(args),<EOL>job_params,<EOL>task_descriptors,<EOL>name=args.name,<EOL>dry_run=args.dry_run,<EOL>command=args.command,<EOL>script=args.script,<EOL>user=args.user,<EOL>user_project=args.user_project,<EOL>wait=args.wait,<EOL>retries=args.retries,<EOL>poll_interval=args.poll_interval,<EOL>after=args.after,<EOL>skip=args.skip,<EOL>project=args.project,<EOL>disable_warning=True,<EOL>unique_job_id=args.unique_job_id)<EOL>

Execute job/task submission from command-line arguments.


if not dry_run:<EOL><INDENT>provider_base.emit_provider_message(provider)<EOL><DEDENT>if not disable_warning:<EOL><INDENT>raise ValueError('<STR_LIT>')<EOL><DEDENT>if command and script:<EOL><INDENT>raise ValueError('<STR_LIT>')<EOL><DEDENT>if command:<EOL><INDENT>if name:<EOL><INDENT>command_name = name<EOL><DEDENT>else:<EOL><INDENT>command_name = _name_for_command(command)<EOL><DEDENT>script = job_model.Script(command_name, '<STR_LIT>' + command)<EOL><DEDENT>elif script:<EOL><INDENT>script_file = dsub_util.load_file(script)<EOL>script = job_model.Script(os.path.basename(script), script_file.read())<EOL><DEDENT>else:<EOL><INDENT>raise ValueError('<STR_LIT>')<EOL><DEDENT>if retries and not wait:<EOL><INDENT>raise ValueError('<STR_LIT>')<EOL><DEDENT>job_model.ensure_job_params_are_complete(job_params)<EOL>job_model.ensure_task_params_are_complete(task_descriptors)<EOL>task_ids = {<EOL>task_descriptor.task_metadata.get('<STR_LIT>')<EOL>for task_descriptor in task_descriptors<EOL>if task_descriptor.task_metadata.get('<STR_LIT>') is not None<EOL>}<EOL>job_metadata = _get_job_metadata(provider, user, name, script, task_ids,<EOL>user_project, unique_job_id)<EOL>_resolve_task_resources(job_metadata, job_resources, task_descriptors)<EOL>if not dry_run:<EOL><INDENT>print('<STR_LIT>' % job_metadata['<STR_LIT>'])<EOL><DEDENT>if after:<EOL><INDENT>if dry_run:<EOL><INDENT>print('<STR_LIT>' % after)<EOL><DEDENT>else:<EOL><INDENT>print('<STR_LIT>')<EOL>error_messages = _wait_after(provider, after, poll_interval, True)<EOL>if error_messages:<EOL><INDENT>for msg in error_messages:<EOL><INDENT>print_error(msg)<EOL><DEDENT>raise dsub_errors.PredecessorJobFailureError(<EOL>'<STR_LIT>',<EOL>error_messages, None)<EOL><DEDENT><DEDENT><DEDENT>job_descriptor = job_model.JobDescriptor(job_metadata, job_params,<EOL>job_resources, task_descriptors)<EOL>launched_job = provider.submit_job(job_descriptor, skip)<EOL>if not dry_run:<EOL><INDENT>if launched_job['<STR_LIT>'] == dsub_util.NO_JOB:<EOL><INDENT>print('<STR_LIT>')<EOL>return {'<STR_LIT>': dsub_util.NO_JOB}<EOL><DEDENT>print('<STR_LIT>' % launched_job['<STR_LIT>'])<EOL>if launched_job.get('<STR_LIT>'):<EOL><INDENT>print('<STR_LIT>' % len(launched_job['<STR_LIT>']))<EOL><DEDENT>print('<STR_LIT>')<EOL>print("<STR_LIT>" %<EOL>(provider_base.get_dstat_provider_args(provider, project),<EOL>launched_job['<STR_LIT>'], launched_job['<STR_LIT>']))<EOL>print('<STR_LIT>')<EOL>print("<STR_LIT>" %<EOL>(provider_base.get_ddel_provider_args(provider, project),<EOL>launched_job['<STR_LIT>'], launched_job['<STR_LIT>']))<EOL><DEDENT>if wait:<EOL><INDENT>print('<STR_LIT>')<EOL>if retries:<EOL><INDENT>error_messages = _wait_and_retry(provider, job_metadata['<STR_LIT>'],<EOL>poll_interval, retries, job_descriptor)<EOL><DEDENT>else:<EOL><INDENT>error_messages = _wait_after(provider, [job_metadata['<STR_LIT>']],<EOL>poll_interval, False)<EOL><DEDENT>if error_messages:<EOL><INDENT>for msg in error_messages:<EOL><INDENT>print_error(msg)<EOL><DEDENT>raise dsub_errors.JobExecutionError(<EOL>'<STR_LIT>'<EOL>'<STR_LIT>', error_messages, launched_job)<EOL><DEDENT><DEDENT>return launched_job<EOL>

Actual dsub body, post-stdout-redirection.


lines = command.splitlines()<EOL>for line in lines:<EOL><INDENT>line = line.strip()<EOL>if line and not line.startswith('<STR_LIT:#>') and line != '<STR_LIT:\\>':<EOL><INDENT>return os.path.basename(re.split(r'<STR_LIT>', line)[<NUM_LIT:0>])<EOL><DEDENT><DEDENT>return '<STR_LIT>'<EOL>

r"""Craft a simple command name from the command.

    The best command strings for this are going to be those where a simple
    command was given; we will use the command to derive the name.

    We won't always be able to figure something out and the caller should just
    specify a "--name" on the command-line.

    For example, commands like "export VAR=val\necho ${VAR}", this function would
    return "export".

    If the command starts space or a comment, then we'll skip to the first code
    we can find.

    If we find nothing, just return "command".

    >>> _name_for_command('samtools index "${BAM}"')
    'samtools'
    >>> _name_for_command('/usr/bin/sort "${INFILE}" > "${OUTFILE}"')
    'sort'
    >>> _name_for_command('# This should be ignored')
    'command'
    >>> _name_for_command('\\\n\\\n# Bad continuations, but ignore.\necho hello.')
    'echo'

    Arguments:
      command: the user-provided command
    Returns:
      a proposed name for the task.


if not rows:<EOL><INDENT>return []<EOL><DEDENT>key_field = '<STR_LIT>'<EOL>if key_field not in rows[<NUM_LIT:0>]:<EOL><INDENT>key_field = '<STR_LIT>'<EOL><DEDENT>grouped = collections.defaultdict(lambda: collections.defaultdict(lambda: []))<EOL>for row in rows:<EOL><INDENT>grouped[row.get(key_field, '<STR_LIT>')][row.get('<STR_LIT:status>', '<STR_LIT>')] += [row]<EOL><DEDENT>new_rows = []<EOL>for job_key in sorted(grouped.keys()):<EOL><INDENT>group = grouped.get(job_key, None)<EOL>canonical_status = ['<STR_LIT>', '<STR_LIT>', '<STR_LIT>', '<STR_LIT>']<EOL>for status in canonical_status + sorted(group.keys()):<EOL><INDENT>if status not in group:<EOL><INDENT>continue<EOL><DEDENT>task_count = len(group[status])<EOL>del group[status]<EOL>if task_count:<EOL><INDENT>summary_row = collections.OrderedDict()<EOL>summary_row[key_field] = job_key<EOL>summary_row['<STR_LIT:status>'] = status<EOL>summary_row['<STR_LIT>'] = task_count<EOL>new_rows.append(summary_row)<EOL><DEDENT><DEDENT><DEDENT>return new_rows<EOL>

Create a new table that is a summary of the input rows.

    All with the same (job-name or job-id, status) go together.

    Args:
      rows: the input rows, a list of dictionaries.
    Returns:
      A new row set of summary information.


<EOL>row_spec = collections.namedtuple('<STR_LIT>',<EOL>['<STR_LIT:key>', '<STR_LIT>', '<STR_LIT>'])<EOL>default_columns = [<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', False, None),<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', True, None)<EOL>]<EOL>full_columns = default_columns + [<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT:status>', True, None),<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', False, None),<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', True, {}),<EOL>row_spec('<STR_LIT>', True, {}),<EOL>row_spec('<STR_LIT>', True, {}),<EOL>row_spec('<STR_LIT>', False, {}),<EOL>row_spec('<STR_LIT>', True, {}),<EOL>row_spec('<STR_LIT>', False, {}),<EOL>row_spec('<STR_LIT>', True, {}),<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', True, {}),<EOL>row_spec('<STR_LIT>', True, []),<EOL>row_spec('<STR_LIT>', False, None),<EOL>row_spec('<STR_LIT>', False, None),<EOL>row_spec('<STR_LIT>', False, None),<EOL>row_spec('<STR_LIT>', False, None),<EOL>]<EOL>summary_columns = default_columns + [<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT>', True, None),<EOL>row_spec('<STR_LIT:status>', True, None),<EOL>]<EOL>assert not (full and summary), '<STR_LIT>'<EOL>if full:<EOL><INDENT>columns = full_columns<EOL><DEDENT>elif summary:<EOL><INDENT>columns = summary_columns<EOL><DEDENT>else:<EOL><INDENT>columns = default_columns<EOL><DEDENT>row = {}<EOL>for col in columns:<EOL><INDENT>key, required, default = col<EOL>value = task.get_field(key, default)<EOL>if required or value is not None:<EOL><INDENT>row[key] = value<EOL><DEDENT><DEDENT>return row<EOL>

return a dict with the task's info (more if "full" is set).


Parses command line arguments.

    Returns:
      A Namespace of parsed arguments.


some_job_running = True<EOL>while some_job_running:<EOL><INDENT>tasks = provider.lookup_job_tasks(<EOL>statuses,<EOL>user_ids=user_ids,<EOL>job_ids=job_ids,<EOL>job_names=job_names,<EOL>task_ids=task_ids,<EOL>task_attempts=task_attempts,<EOL>labels=labels,<EOL>create_time_min=create_time_min,<EOL>create_time_max=create_time_max,<EOL>max_tasks=max_tasks,<EOL>page_size=max_tasks)<EOL>some_job_running = False<EOL>formatted_tasks = []<EOL>for task in tasks:<EOL><INDENT>if <NUM_LIT:0> < max_tasks <= len(formatted_tasks):<EOL><INDENT>break<EOL><DEDENT>if raw_format:<EOL><INDENT>formatted_tasks.append(task.raw_task_data())<EOL><DEDENT>else:<EOL><INDENT>formatted_tasks.append(_prepare_row(task, full_output, summary_output))<EOL><DEDENT>if task.get_field('<STR_LIT>') == '<STR_LIT>':<EOL><INDENT>some_job_running = True<EOL><DEDENT><DEDENT>yield formatted_tasks<EOL>if poll_interval and some_job_running:<EOL><INDENT>time.sleep(poll_interval)<EOL><DEDENT>else:<EOL><INDENT>break<EOL><DEDENT><DEDENT>

Generate jobs as lists of task dicts ready for formatting/output.

    Args:
      provider: an instantiated dsub provider.
      statuses: a set of status strings that eligible jobs may match.
      user_ids: a set of user strings that eligible jobs may match.
      job_ids: a set of job-id strings eligible jobs may match.
      job_names: a set of job-name strings eligible jobs may match.
      task_ids: a set of task-id strings eligible tasks may match.
      task_attempts: a set of task-attempt strings eligible tasks may match.
      labels: set of LabelParam that all tasks must match.
      create_time_min: a timezone-aware datetime value for the earliest create
                       time of a task, inclusive.
      create_time_max: a timezone-aware datetime value for the most recent create
                       time of a task, inclusive.
      max_tasks: (int) maximum number of tasks to return per dstat job lookup.
      full_output: (bool) return all dsub fields.
      summary_output: (bool) return a summary of the job list.
      poll_interval: (int) wait time between poll events, dstat will poll jobs
                     until all jobs succeed or fail. Set to zero to disable
                     polling and return after the first lookup.
      raw_format: (bool) set True to prevent dsub from normalizing the task dict,
                  this defaults to False and should only be set True if a
                  provider-specific view of tasks is absolutely required.
                  (NB: provider interfaces change over time, no transition path
                  will be provided for users depending on this flag).

    Yields:
      lists of task dictionaries - each list representing a dstat poll event.


tasks_generator = provider.lookup_job_tasks(<EOL>statuses,<EOL>user_ids=user_ids,<EOL>job_ids=job_ids,<EOL>job_names=job_names,<EOL>task_ids=task_ids,<EOL>task_attempts=task_attempts,<EOL>labels=labels,<EOL>create_time_min=create_time_min,<EOL>create_time_max=create_time_max,<EOL>max_tasks=max_tasks,<EOL>page_size=page_size)<EOL>for task in tasks_generator:<EOL><INDENT>yield _prepare_row(task, True, summary_output)<EOL><DEDENT>

Generate formatted jobs individually, in order of create-time.

    Args:
      provider: an instantiated dsub provider.
      statuses: a set of status strings that eligible jobs may match.
      user_ids: a set of user strings that eligible jobs may match.
      job_ids: a set of job-id strings eligible jobs may match.
      job_names: a set of job-name strings eligible jobs may match.
      task_ids: a set of task-id strings eligible tasks may match.
      task_attempts: a set of task-attempt strings eligible tasks may match.
      labels: set of LabelParam that all tasks must match.
      create_time_min: a timezone-aware datetime value for the earliest create
                       time of a task, inclusive.
      create_time_max: a timezone-aware datetime value for the most recent create
                       time of a task, inclusive.
      max_tasks: (int) maximum number of tasks to return per dstat job lookup.
      page_size: the page size to use for each query to the backend. May be
                 ignored by some provider implementations.
      summary_output: (bool) summarize the job list.

    Yields:
      Individual task dictionaries with associated metadata


date_fields = ['<STR_LIT>', '<STR_LIT>', '<STR_LIT>', '<STR_LIT>']<EOL>int_fields = ['<STR_LIT>']<EOL>for col in date_fields:<EOL><INDENT>if col in row:<EOL><INDENT>row[col] = self.default_format_date(row[col])<EOL><DEDENT><DEDENT>for col in int_fields:<EOL><INDENT>if col in row and row[col] is not None:<EOL><INDENT>row[col] = int(row[col])<EOL><DEDENT><DEDENT>return row<EOL>


raise NotImplementedError('<STR_LIT>')<EOL>

Function to be defined by the derived class to print output.


def trim_display_field(self, value, max_length):

if not value:<EOL><INDENT>return '<STR_LIT>'<EOL><DEDENT>if len(value) > max_length:<EOL><INDENT>return value[:max_length - <NUM_LIT:3>] + '<STR_LIT>'<EOL><DEDENT>return value<EOL>

Return a value for display; if longer than max length, use ellipsis.


return '<STR_LIT:U+002CU+0020>'.join(<EOL>'<STR_LIT>' % (key, value) for key, value in sorted(values.items()))<EOL>

Returns a string of comma-delimited key=value pairs.


def string_presenter(self, dumper, data):

if '<STR_LIT:\n>' in data:<EOL><INDENT>return dumper.represent_scalar('<STR_LIT>', data, style='<STR_LIT:|>')<EOL><DEDENT>else:<EOL><INDENT>return dumper.represent_scalar('<STR_LIT>', data)<EOL><DEDENT>

Presenter to force yaml.dump to use multi-line string style.


docker_path = raw_uri.replace('<STR_LIT>', '<STR_LIT>', <NUM_LIT:1>)<EOL>return raw_uri, docker_path<EOL>

Rewrite GCS file paths as required by the rewrite_uris method.

    The GCS rewriter performs no operations on the raw_path and simply returns
    it as the normalized URI. The docker path has the gs:// prefix replaced
    with gs/ so that it can be mounted inside a docker image.

    Args:
      raw_uri: (str) the raw GCS URI, prefix, or pattern.

    Returns:
      normalized: a cleaned version of the uri provided by command line.
      docker_path: the uri rewritten in the format required for mounting inside
                   a docker worker.


<EOL>raw_path, filename = os.path.split(raw_uri)<EOL>prefix_replacements = [('<STR_LIT>', '<STR_LIT:/>'), ('<STR_LIT>', os.getenv('<STR_LIT>')), ('<STR_LIT>',<EOL>'<STR_LIT>'),<EOL>('<STR_LIT>', '<STR_LIT:/>')]<EOL>normed_path = raw_path<EOL>for prefix, replacement in prefix_replacements:<EOL><INDENT>if normed_path.startswith(prefix):<EOL><INDENT>normed_path = os.path.join(replacement, normed_path[len(prefix):])<EOL><DEDENT><DEDENT>normed_uri = directory_fmt(os.path.abspath(normed_path))<EOL>normed_uri = os.path.join(normed_uri, filename)<EOL>docker_rewrites = [(r'<STR_LIT>', '<STR_LIT>'), (r'<STR_LIT>', '<STR_LIT>'),<EOL>(r'<STR_LIT>', '<STR_LIT>'), (r'<STR_LIT>', '<STR_LIT>')]<EOL>docker_path = os.path.normpath(raw_path)<EOL>for pattern, replacement in docker_rewrites:<EOL><INDENT>docker_path = re.sub(pattern, replacement, docker_path)<EOL><DEDENT>docker_path = docker_path.lstrip('<STR_LIT>')  <EOL>docker_path = directory_fmt('<STR_LIT>' + docker_path) + filename<EOL>return normed_uri, docker_path<EOL>

Rewrite local file URIs as required by the rewrite_uris method.

    Local file paths, unlike GCS paths, may have their raw URI simplified by
    os.path.normpath which collapses extraneous indirect characters.

    >>> _local_uri_rewriter('/tmp/a_path/../B_PATH/file.txt')
    ('/tmp/B_PATH/file.txt', 'file/tmp/B_PATH/file.txt')
    >>> _local_uri_rewriter('/myhome/./mydir/')
    ('/myhome/mydir/', 'file/myhome/mydir/')

    The local path rewriter will also work to preserve relative paths even
    when creating the docker path. This prevents leaking of information on the
    invoker's system to the remote system. Doing this requires a number of path
    substitutions denoted with the _<rewrite>_ convention.

    >>> _local_uri_rewriter('./../upper_dir/')[1]
    'file/_dotdot_/upper_dir/'
    >>> _local_uri_rewriter('~/localdata/*.bam')[1]
    'file/_home_/localdata/*.bam'

    Args:
      raw_uri: (str) the raw file or directory path.

    Returns:
      normalized: a simplified and/or expanded version of the uri.
      docker_path: the uri rewritten in the format required for mounting inside
                   a docker worker.


return _get_filtered_mounts(mounts, job_model.GCSMountParam)<EOL>


return _get_filtered_mounts(mounts, job_model.PersistentDiskMountParam)<EOL>

Returns the persistent disk mounts from mounts.


return _get_filtered_mounts(mounts, job_model.LocalMountParam)<EOL>


return set([mount for mount in mounts if isinstance(mount, mount_param_type)])<EOL>

Helper function to return an appropriate set of mount parameters.


if not logging_uri:<EOL><INDENT>return job_model.LoggingParam(None, None)<EOL><DEDENT>recursive = not logging_uri.endswith('<STR_LIT>')<EOL>oututil = util_class('<STR_LIT>')<EOL>_, uri, provider = oututil.parse_uri(logging_uri, recursive)<EOL>if '<STR_LIT:*>' in uri.basename:<EOL><INDENT>raise ValueError('<STR_LIT>' % uri)<EOL><DEDENT>return job_model.LoggingParam(uri, provider)<EOL>

Convenience function simplifies construction of the logging uri.


def split_pair(pair_string, separator, nullable_idx=<NUM_LIT:1>):

pair = pair_string.split(separator, <NUM_LIT:1>)<EOL>if len(pair) == <NUM_LIT:1>:<EOL><INDENT>if nullable_idx == <NUM_LIT:0>:<EOL><INDENT>return [None, pair[<NUM_LIT:0>]]<EOL><DEDENT>elif nullable_idx == <NUM_LIT:1>:<EOL><INDENT>return [pair[<NUM_LIT:0>], None]<EOL><DEDENT>else:<EOL><INDENT>raise IndexError('<STR_LIT>')<EOL><DEDENT><DEDENT>else:<EOL><INDENT>return pair<EOL><DEDENT>

Split a string into a pair, which can have one empty value.

    Args:
      pair_string: The string to be split.
      separator: The separator to be used for splitting.
      nullable_idx: The location to be set to null if the separator is not in the
                    input string. Should be either 0 or 1.

    Returns:
      A list containing the pair.

    Raises:
      IndexError: If nullable_idx is not 0 or 1.


job_params = []<EOL>for col in header:<EOL><INDENT>col_type = '<STR_LIT>'<EOL>col_value = col<EOL>if col.startswith('<STR_LIT:->'):<EOL><INDENT>col_type, col_value = split_pair(col, '<STR_LIT:U+0020>', <NUM_LIT:1>)<EOL><DEDENT>if col_type == '<STR_LIT>':<EOL><INDENT>job_params.append(job_model.EnvParam(col_value))<EOL><DEDENT>elif col_type == '<STR_LIT>':<EOL><INDENT>job_params.append(job_model.LabelParam(col_value))<EOL><DEDENT>elif col_type == '<STR_LIT>' or col_type == '<STR_LIT>':<EOL><INDENT>name = input_file_param_util.get_variable_name(col_value)<EOL>job_params.append(<EOL>job_model.InputFileParam(<EOL>name, recursive=(col_type.endswith('<STR_LIT>'))))<EOL><DEDENT>elif col_type == '<STR_LIT>' or col_type == '<STR_LIT>':<EOL><INDENT>name = output_file_param_util.get_variable_name(col_value)<EOL>job_params.append(<EOL>job_model.OutputFileParam(<EOL>name, recursive=(col_type.endswith('<STR_LIT>'))))<EOL><DEDENT>else:<EOL><INDENT>raise ValueError('<STR_LIT>' % col)<EOL><DEDENT><DEDENT>return job_params<EOL>

Parse the header from the tasks file into env, input, output definitions.

    Elements are formatted similar to their equivalent command-line arguments,
    but with associated values coming from the data rows.

    Environment variables columns are headered as "--env <name>"
    Inputs columns are headered as "--input <name>" with the name optional.
    Outputs columns are headered as "--output <name>" with the name optional.

    For historical reasons, bareword column headers (such as "JOB_ID") are
    equivalent to "--env var_name".

    Args:
      header: Array of header fields
      input_file_param_util: Utility for producing InputFileParam objects.
      output_file_param_util: Utility for producing OutputFileParam objects.

    Returns:
      job_params: A list of EnvParams and FileParams for the environment
      variables, LabelParams, input file parameters, and output file parameters.

    Raises:
      ValueError: If a header contains a ":" and the prefix is not supported.