Package: labelbigquery
Labelbox Connector for Google BigQueryAccess the Labelbox Connector for Google BigQuery to easily upload your CSV of text snippets to BigQuery, select columns, and add that dataset to Labelbox for annotation in our text tool. This is a very specific library for text use-cases, although it may be expanded to support other use-cases as needed in BigQuery.The Demo code supplied in this Github is designed to run in a Google Co-Lab, but the code can be adapted to any notebook environment.Labelbox is the enterprise-grade training data solution with fast AI enabled labeling tools, labeling automation, human workforce, data management, a powerful API for integration & SDK for extensibility. VisitLabelboxfor more information.This library is currently in beta. It may contain errors or inaccuracies and may not function as well as commercially released software. Please report any issues/bugs viaGithub Issues.Table of ContentsRequirementsInstallationDocumentationAuthenticationContributionRequirementsGoogle Cloud BigQuery Authenticated ClientGoogle BigQuery SDKLabelbox accountGenerate a Labelbox API keyInstallationInstall LabelBigQuery to your Python environment. The installation will also add the Labelbox SDK and BigQuery SDK.pip install labelbigqueryDocumentationLabelBigQuery includes several methods to help facilitate your workflow between BigQuery and Labelbox.Add your CSV contents to BigQuery (only necessary if you don't have your data in BigQuery yet):#define headers and fields for BigQuery data load
    SELECTED_HEADERS = {
        'conversation_id',
        'normalized_query'
    }

    SCHEMA_FIELDS = [
        bigquery.SchemaField("conversation_id", "STRING"),
        bigquery.SchemaField("normalized_query", "STRING"),
    ]

    labelbigquery.load_data_to_big_query(bq_client, args.table_name, args.csv_file_name,
                                         SELECTED_HEADERS = SELECTED_HEADERS,SCHEMA_FIELDS = SCHEMA_FIELDS)Where "SELECTED_HEADERS" and "SCHEMA_FIELDS" specifies the columns of your CSV that you want to send to BigQuery, along with the type definitions for proper storage in BigQuery.Labelbigquery for text requires two columns of data; a unique identifier (becomes the "External ID" in our system), and a corresponding text string. Here is a chatbot example table:conversation_idnormalized_querysample_1Some text string here for labeling.sample_2Some text string here for labeling.sample_3Some text string here for labeling.Submit a query to BigQuery for your target columns. This will also write individual text files to a "data" folder. The file names are based off the unique identifier ("conversation id" in the above example).query = fr'SELECT conversation_id, STRING_AGG(normalized_query, "\n") FROM {args.table_name} GROUP BY conversation_id'
    file_names = labelbigquery.fetch_and_write_rows(bq_client, query=query)Submit your files to Labelbox for annotation in the text editor.lb_dataset = labelbigquery.make_dataset_and_data_rows(lb_client, file_names, args.dataset_name)
    print("Dataset unique identifier: " + lb_dataset.uid)While using LabelBigQuery, you will likely also use the Labelbox SDK (e.g. for programmatic ontology creation). These resources will help familiarize you with the Labelbox Python SDK:Visit our docsto learn how the SDK worksView ourLabelBigQuery demo codefor inspiration.view ourAPI reference.AuthenticationLabelbox uses API keys to validate requests. You can create and manage API keys onLabelbox.ContributionPlease consultCONTRIB.md