Package: gcloudy
ABOUTgcloudyis a wrapper for Google's GCP Python package(s) that aims to make interacting with GCP and its services more intuitive, especially for new GCP users. In doing so, it adheres topandas-likesyntax for function/method calls.Thegcloudypackage is not meant to be a replacement for GCP power-users, but rather an alternative for GCP users who are interested in using Python in GCP to deploy Cloud Functions and interact with certain GCP services, especially BigQuery and Google Cloud Storage.Thegcloudypackage is built on top of cononical Google Python packages(s) without any alteration to Google's base code.INSTALL, IMPORT, & INITIALIZEgcloudyis installed using pip with theterminalcommand:$ pip install gcloudyOnce installed, theBigQueryclass can be imported from the mainGoogleCloudmodule with:from gcloudy.GoogleCloud import BigQueryThen, thebqobject is initialized with the following (where "gcp-project-name" is your GCP Project ID / Name):bq = BigQuery("gcp-project-name")NOTE: It is important to also import the Pandas package:import pandas as pdMETHODSThe following section contains the methods and their usage.----------------------------bq.read_bigquery- Read an existing BigQuery table into a DataFrame.read_bigquery(bq_dataset_dot_table = None, date_cols = [], preview_top = None, to_verbose = True)bq_dataset_dot_table: the "dataset-name.table-name" path of the existing BigQuery tabledate_cols: [optional] column(s) passed inside a list that should be parsed as datespreview_top: [optional] only read in the topNrowsto_verbose: should info be printed? defaults toTrueEX:my_table = bq.read_bigquery("my_bq_dataset.my_bq_table")
my_table = bq.read_bigquery("my_bq_dataset.my_bq_table", date_cols = ['date'])-----------bq.write_bigquery- Write a DataFrame to a BigQuery table.write_bigquery(df, bq_dataset_dot_table = None, use_schema = None, append_to_existing = False, to_verbose = True)df: the DataFrame to be written to a BigQuery tablebq_dataset_dot_table: the "dataset-name.table-name" path of the existing BigQuery tableuse_schema: [optional] a custom schema for the BigQuery table.NOTE: seebq.guess_schemabelowappend_to_existing: should the DataFrame be appended to an existing BigQuery table? defaults toFalse(create new / overwrite)to_verbose: should info be printed? defaults toTrueEX:bq.write_bigquery(my_data, "my_bq_dataset.my_data")
bq.write_bigquery(my_data, "my_bq_dataset.my_data", append_to_existing = True)-----------bq.guess_schema- A helper forbq.write_bigquery, passed to itsuse_schemaarg. Creates a custom schema based on thedtypesof a DataFrame.guess_schema(df, bq_type_default = "STRING")df: the DataFrame to be written to a BigQuery tablebq_type_default: default BQ type passed todtype'object'EX:bq.write_bigquery(my_data, "my_bq_dataset.my_data", use_schema = bq.guess_schema(my_data))-----------bq.read_custom_query- Read in a custom BigQuery SQL query into a DataFrame.read_custom_query(custom_query, to_verbose = True)custom_query: the custom BigQuery SQL query that will produce a table to be read into a DataFrameto_verbose: should info be printed? defaults toTrueEX:my_custom_table = bq.read_custom_query("""
    SELECT
        date,
        sales,
        products
    FROM
        my_bq_project_id.my_bq_dataset.my_bq_table
    WHERE
        sales_month = 'June'
""")-----------bq.send_query- Send a custom SQL query to BigQuery. Note, does not return anything as the process is carried out within BigQuery.send_query(que, to_verbose = True)que: the custom SQL query to be sent and carried out within BigQueryto_verbose: should info be printed? defaults toTrueEX:bq.send_query("""
    CREATE TABLE my_bq_project_id.my_bq_dataset.my_new_bq_table AS 
    (
        SELECT
            date,
            sales,
            products
        FROM
            my_bq_project_id.my_bq_dataset.my_bq_table
        WHERE
            sales_month = 'June'
    )
""")-----------bq.read_gcs- Read a CSV file stored within a Google Cloud Storage (GCS) Bucket into a DataFrame.read_gcs(gsutil_uri, date_cols = None, to_verbose = True)gsutil_uri: the GCS Bucket path of the existing CSV filedate_cols: [optional] column(s) passed inside a list that should be parsed as datesto_verbose: should info be printed? defaults toTrueEX:my_table = bq.read_gcs("gs://my-bucket/my_data.csv")
my_table = bq.read_gcs("gs://my-bucket/my_data.csv", date_cols = ['date'])-----------bq.write_gcs- Write a Pandas DataFrame to a Google Cloud Storage (GCS) Bucket as a CSV.write_gcs(pandas_df, gsutil_uri, keep_index = False, to_verbose = True)pandas_df: the Pandas DataFrame to be written to a Google Cloud Storage (GCS) Bucket as a CSVgsutil_uri: the GCS Bucket pathkeep_index: should the DataFrame index be written as well? defaults toFalseto_verbose: should info be printed? defaults toTrueEX:bq.write_gcs(my_data, "gs://my-bucket/my_data.csv")