Module Expr Str ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias Assign NameStore Str ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad body Expr Str If BoolOpOr NameLoad NameLoad body Assert BoolOpOr BoolOpAnd NameLoad UnaryOpNot NameLoad BoolOpAnd NameLoad UnaryOpNot NameLoad Str Assign AttributeStore NameLoad attr Call NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str If AttributeLoad NameLoad attr body If BoolOpOr AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Expr Call NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr orelse Raise Call NameLoad Str decorator_list FunctionDef arguments args NameParam defaults body Expr Str If AttributeLoad NameLoad attr body Return AttributeLoad NameLoad attr orelse If AttributeLoad NameLoad attr body With Call NameLoad AttributeLoad NameLoad attr Str NameStore body Return Call AttributeLoad NameLoad attr Return Call NameLoad AttributeLoad NameLoad attr decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Str If AttributeLoad NameLoad attr body Return Call AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr Str attr orelse Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Str Return AttributeLoad NameLoad attr decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str If UnaryOpNot Call NameLoad NameLoad NameLoad body Raise Call NameLoad Str Assign AttributeStore NameLoad attr Call NameLoad keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr decorator_list AttributeLoad NameLoad attr decorator_list
N/A N/A 
Base class for storing a S3 File
 utils.exceptions ETLInputError utils.helpers parse_path s3_path S3Path utils read_from_s3 utils upload_to_s3 N/A DEFAULT_FILE_NAME file S3File N/A object N/A N/A S3 File object that provides functions to operate with a file on S3

   The S3 file unifies the file concept, which could be stored on the
   local file system, as a string, or already in s3.

     __init__ N/A N/A self path text s3_path N/A None None None N/A N/A Constructor for the S3 File object

        Args:
            path (str): Local path to file
            text (str): Text of a file
            s3_path (S3Path, optional): s3_path of the file

         N/A N/A path text N/A N/A N/A N/A path N/A text N/A text N/A path Cannot specify both path and text for s3 file. N/A N/A self _path N/A parse_path path N/A N/A self _text text N/A N/A self _s3_path s3_path N/A upload_to_s3 N/A N/A self N/A N/A N/A Sends file to URI. This action is idempotent.

        Raises:
            ETLInputError: If no URL is provided
         N/A N/A self _s3_path N/A N/A N/A N/A self _path N/A self _text N/A N/A N/A upload_to_s3 N/A self _s3_path N/A self _path N/A self _text N/A N/A N/A ETLInputError No URI provided for the file to be uploaded N/A text N/A N/A self N/A N/A N/A Outputs the text of the associated file

        Returns:
            result(str): The text of the file. Can be local or on S3
         N/A N/A self _text N/A N/A N/A self _text N/A N/A N/A self _path N/A N/A N/A open N/A self _path r f N/A N/A N/A N/A f read N/A N/A read_from_s3 N/A self _s3_path N/A property file_name N/A N/A self N/A N/A N/A The file name of this file

        Returns:
            file_name(str): The file_name of this file
         N/A N/A self _path N/A N/A N/A N/A N/A N/A N/A self _path split / pop N/A N/A DEFAULT_FILE_NAME N/A property s3_path N/A N/A self N/A N/A N/A Outputs the s3_path
         N/A N/A self _s3_path N/A property s3_path N/A N/A self s3_path N/A N/A N/A Set the S3 path for the file

        Args:
            s3_path(S3Path): If the path is a directory, a
            name will be assigned based on the path variable.
            If there is no path, the name "file" will be applied.
         N/A N/A N/A isinstance s3_path S3Path N/A N/A N/A ETLInputError Input path should be of type S3Path N/A N/A self _s3_path N/A S3Path key N/A s3_path key bucket N/A s3_path bucket is_directory N/A s3_path is_directory N/A N/A s3_path is_directory N/A N/A N/A N/A N/A self _s3_path append N/A self file_name N/A N/A s3_path setter N/A