Package: visier-connector
Visier Python ConnectorThe Visier Python Connector allows Python developers to query against the Visier platform in Visier's SQL-like syntax.PrerequisitesThis connector acts as a bridge between your Python application and Visier's cloud-hosted service infrastructure. To successfully connect to your Visier data, you need:The URL domain name prefix. For example:https://{vanity-name}.api.visier.io.An API key issued by Visier.One of:A registered OAuth client application in your Visier tenant.A Visier username and password with API access capabilities.Authentication EnvironmentAs of version0.9.8, the Visier Python Connector supports two authentication methods:OAuth 2.0: A three-legged authentication flow that authenticates through the authorization server. In OAuth 2.0 protocol, no user credentials are provided directly to Visier. We recommend using the OAuth 2.0 authentication method.Basic authentication: A traditional authentication flow that authenticates through a Visier username and password.To avoid passing authentication credentials in with command-line arguments, Visier recommends that basic authentication credentials, such as username and password, are passed in through environment variables. You can use themake_auth()function to create the appropriate authentication configuration object fromVISIER_-prefixed environment variables, as described below.OAuth 2.0The following list defines the OAuth 2.0 authentication parameters. These are also the environment anddotenvvariables themake_auth()utility function will use when instantiating an authentication object.VISIER_HOST: The fully qualified domain name and protocol to access your Visier tenant and initiate the OAuth 2.0 authentication process.VISIER_APIKEY: The API key granted by Visier.VISIER_CLIENT_ID: The identifier of the registered client application.VISIER_CLIENT_SECRET: The generated secret of the registered client application. This is required for customer-registered applications.VISIER_REDIRECT_URI: The URI theauthorizecall will redirect to upon a successful authorization code generation. By default, this ishttp://localhost:5000/oauth2/callbackbut the URI must match theredirect_uriin the client application registration exactly. If the client application URI is different, it is essential that that exact value is provided to the Python connector.VISIER_TARGET_TENANT_ID: The tenant code of the target tenant. This is only applicable in partner configurations.The following example illustrates an authentication environment. Let's say you're using a Linux-like system with an X-display available. First, create a file named.envand then populate the file as shown next, substituting the example values with actual values as appropriate.exportVISIER_HOST=https://customer-specific.api.visier.ioexportVISIER_CLIENT_ID=client-idexportVISIER_APIKEY=the-api-key-issued-by-visierexportVISIER_REDIRECT_URI=# export VISIER_REDIRECT_URI=http://localhost:5000/oauth2/callbackexportVISIER_TARGET_TENANT_ID=exportVISIER_USERNAME=exportVISIER_PASSWORD=echo-n"Enter the client secret for client with id$VISIER_CLIENT_ID: "read-ssecretexportVISIER_CLIENT_SECRET=$secretNote: You may provide a valid username and password combination with the variables above. If a username and password are provided along with the client ID and secret, the connector will use the password grant type instead of the authorization code grant type. We do not recommend the password grant type in a production environment.Next, source the file below in and then the environment is ready to use the connector with OAuth 2.0 authentication.$source.envBecause the connector supportspython-dotenv, some users may prefer to define assignments directly in.envinstead of sourcing it into the OS environment. In the following snippet, the connector uses credentials obtained with dotenv.fromdotenvimportdotenv_valuesfromvisier.connectorimportVisierSession,make_authfromvisier.apiimportModelApiClientenv_creds=dotenv_values()auth=make_auth(env_values=env_creds)withVisierSession(auth)ass:...Callback URIThe OAuth 2.0 authentication flow requires that the authorizing server can call back to the initiating client with an authorization code. In OAuth mode, the connector starts a transient web server that listens for an authorization code onhttp://localhost:5000/oauth2/callback. You can modify the URL by setting a different value forVISIER_REDIRECT_URI. TheVISIER_REDIRECT_URIvalue must exactly match the URI value in your Visier OAuth 2.0 client registration and must respect Visier's callback URI rules, such as a limited set of permissible subnets.Basic AuthenticationThe Visier Python Connector doesn't directly interact with the environment variables. The following list defines the basic authentication parameters. The basic authentication parameters are also the environment variables that themake_auth()utility function uses.VISIER_HOST: The fully qualified domain name and protocol to access your Visier tenant.VISIER_USERNAME: The Visier user that has sufficient API capabilities.VISIER_PASSWORD: The password of that user.VISIER_APIKEY: The API key granted by Visier.VISIER_VANITY: The vanity name of the Visier tenant.VISIER_TARGET_TENANT_ID: The tenant code of the target tenant. This is only applicable in partner configurations.The following example illustrates an authentication environment. The example is suitable in a non-production environment.Let's say you're using a Linux-like system. First, create a file named.envand then populate the file as shown next, substituting the example values with actual values as appropriate.echo-n"Enter the password for the Visier API User: "read-svpwdexportVISIER_VANITY=customer-specificexportVISIER_HOST=https://$VISIER_VANITY.api.visier.ioexportVISIER_USERNAME=apiuser@example.comexportVISIER_PASSWORD=$vpwdexportVISIER_TARGET_TENANT_ID=tenant-codeexportVISIER_APIKEY=the-api-key-issued-by-visierexportVISIER_CLIENT_ID=Next, source this environment in and provide the password when prompted.$source.envJupyter NotebooksJupyter notebooks and lab are well-suited to run Visier connector code. However, some users may not find OS-level variables ideal. As of version0.9.9, the Visier Python connector supportsdotenvto facilitate a more dynamic switching of Visier authentication parameters. If the file is called.env, the Python packagedotenvattempts to load the file. If the file has a different name, you must provide that file name when loading the environment withdotenv.Jupyter Basic Authentication ExampleBasic authentication is the most practical means of authenticating against Visier for Jupyter notebooks.Create an environment file to store the authentication parameters.Example environment file:VISIER_VANITY=customer-specific
VISIER_HOST=https://customer-specific.api.visier.io
VISIER_APIKEY=the-api-key-issued-by-visier
VISIER_USERNAME=apiuser@example.com
VISIER_PASSWORD=password-or-variable-referenceCreate a basic authentication object as described in the following snippet:fromdotenvimportdotenv_valuesfromvisier.connectorimportVisierSession,make_authfromvisier.apiimportQueryApiClientenv_creds=dotenv_values()auth=make_auth(env_values=env_creds)withVisierSession(auth)ass:query_client=QueryApiClient(s)...Jupyter OAuth 2.0 ExampleOAuth authentication to Visier in Jupyter notebooks is only supported when the Jupyter server runs on your local computer, bound tolocalhost. We only recommend using OAuth in Jupyter notebooks against Visier for test and development uses.To authenticate with OAuth, you must first register an OAuth 2.0 client in Visier. Visier administrators can register OAuth clients.After the OAuth client is registered in Visier, create an environment file to store the authentication parameters.Example environment file:VISIER_HOST=https://customer-specific.api.visier.io
VISIER_CLIENT_ID=client-id-from-registration
VISIER_APIKEY=the-api-key-issued-by-visierCreate an OAuth authentication object, as described next.fromdotenvimportdotenv_valuesfromvisier.connectorimportVisierSession,make_authfromvisier.apiimportQueryApiClientenv_creds=dotenv_values()auth=make_auth(env_values=env_creds)withVisierSession(auth)ass:query_client=QueryApiClient(s)...Connector SeparationAs of version0.9.5, the Python connector separates API calls from theVisierSessionobject. As a result of this change, query execution methods on theVisierSessionare deprecated and will be removed in a future release.As of version0.9.5, you can invoke Visier public APIs through the Visier Python connector by instantiating the appropriate API client and calling the methods defined on the client object. The following example invokes theanalytic-objectsData Model endpoint to retrieve metadata for two analytic objects:RequisitionandEmployee_Exit.withVisierSession(auth)assession:model_client=ModelApiClient(session)objs=model_client.get_analytic_objects(["Requisition","Employee_Exit"])print(objs.text)Error HandlingBy default, a failed API call returnsNoneand information about the error is available on the client object. Using the example above, if the call failed, the value ofobjsisNone(no value). To investigate why the call failed, you can callmodel_client.last_error()to find the issue.It's possible to force the API client to instead raise aQueryExecutionException. To do so, instantiate the API client with the parameter valueraise_on_error=True. Using the example above, themodel_clientinstantiation would appear asmodel_client = ModelApiClient(session, raise_on_error=True).ExamplesData Query APIThe Query API client makes calls to Visier's Data Query API. The Data Query API examples usePandasto illustrate common data engineering and data science workflows using Visier data.Note: Theexamplesin this repository are not included in thevisier-connectorpackage. To use theexamples, copy them into a sample application or run the example queries with a test script in this repository as shown in the samples below.Generally, Visier Query API queries are one of:List query: List queries provide a detailed, non-aggregated view of underlying analytic objects in a tabular format. Each table attribute represents a column in the result set. List queries are also referred to asdetailordrill throughqueries.Aggregate query: Aggregate queries aggregate metric values. The API aggregates the query's defined axes and then produces multi-dimensional cell sets. To get a flat response rather than a multi-dimensional response, you can provide anAcceptheader whose first value is eitherapplication/jsonlinesortext/csv. You must always includeapplication/jsonafter the flat format to successfully get error responses.If you'd prefer not to make JSON-based queries, Visier offers a SQL-like alternative that allows you to make queries in a language close to SQL. This is preferable for users who prefer the more compact and intuitive aspects of SQL. You can define both aggregate and list queries in Visier's SQL-like dialect.In this repository, example queries are provided in individual files for your convenience. However, SQL-like is formed with simple strings and can be provided to the call itself rather than in files.To reduce duplication, ensure that you precede each sample with the necessaryimportstatements and authentication credential definition, as shown next.Note: The use of pandas here is for demonstration purposes. The Visier Python Connector does not depend on pandas.importosfromvisier.connectorimportVisierSession,make_authfromvisier.apiimportQueryApiClientfromexamplesimportload_json,load_strimportpandasaspdauth=make_auth()List QueryList data is granular, non-aggregated data from Visier objects like theEmployeesubject or theCompensation_Payoutevent. In this example, we define a snippet to getEmployee.Pay_Levelthat you can add to a third-party resource that loads detailed data; for example, a Jupyter Notebook.withVisierSession(auth)ass:client=QueryApiClient(s)# List query from JSON query definitionlist_query=load_json("detail/employee-pay_level.json")list_result=client.list(list_query)df_list=pd.DataFrame.from_records(data=list_result.rows(),columns=list_result.header)# ...print(df_list.head)Snapshot QuerySnapshot queries retrieve a collection oflistquery-style snapshots taken at the defined intervals. Asnapshotquery's structure and result granularity is similar to alistquery. You can think of asnapshotquery as a sequence oflistqueries executed at specific times.You can augment thesnapshotresult with theeffectiveDatePropertyproperty to include the time of each snapshot. For more information about using theeffectiveDatePropertyproperty, see the referenced sample JSONquery definition.withVisierSession(auth)ass:client=QueryApiClient(s)# Snapshot query from JSON query definitionsnapshot_query=load_json("snapshot/employee-info.json")snapshot_result=client.snapshot(snapshot_query)df_snapshot=pd.DataFrame.from_records(data=snapshot_result.rows(),columns=snapshot_result.header)# ...print(df_snapshot.head)Aggregate QueryAggregate queries execute queries using Visier metrics. A metric is a business question or concern that is quantifiable as a number. Visier metrics range from simple metrics likeemployeeCountto more complex metrics likehrRecruitingBudgetedLaborCostPerFTE. In this example, we define a query to aggregateapplicantCountbyApplication_SourceandApplicant_Stage.With aVisierSessionavailable, an aggregate query is executed functionally identically.withVisierSession(auth)ass:client=QueryApiClient(s)# Aggregate query from JSON query definitionaggregate_query=load_json("aggregate/applicants-source.json")aggregate_result=client.aggregate(aggregate_query)df_aggregate=pd.DataFrame.from_records(data=aggregate_result.rows(),columns=aggregate_result.header)# Now that the data is in a Pandas Data Frame, do something with it, or just...print(df_aggregate.head)SQL-Like QueryYou can write SQL-like queries to define both list and aggregate queries.SQL-Like List QueryIn this example, we define a snippet to get theEmployeeID,Union_Status,Direct_Manager.Gender,Direct_Manager.Vaccination_StatuswhereisFemale=TRUEandisHighPerformer=TRUEbetween January 1, 2020 and December 31, 2021.withVisierSession(auth)ass:client=QueryApiClient(s)# SQL-like detail querysql_detail_query=load_str("sql-like/detail/employee-demo.sql")list_result=client.sqllike(sql_detail_query)df_list=pd.DataFrame.from_records(data=list_result.rows(),columns=list_result.header)# ...print(df_list.head)SQL-Like Aggregate QueryIn this example, we define a snippet to aggregateemployeeCountbyLocation_0,Gender,Union_Status, andLocation_1for 4 periods of 3 months each starting from April 1, 2020. In aggregate SQL-like queries, you can useoptionsto eliminate cells with zero and null values. This reduces the size of the overall result set to only include rows whose metric value is more than 0.withVisierSession(auth)ass:client=QueryApiClient(s)# SQL-like aggregate querysql_aggregate_query=load_str("sql-like/aggregate/employee-count.sql")sparse_options=load_json("sql-like/options/sparse.json")aggregate_result=client.sqllike(sql_aggregate_query,sparse_options)df_aggregate=pd.DataFrame.from_records(data=aggregate_result.rows(),columns=aggregate_result.header)# ...print(df_aggregate.head)Data Model APIThe Model API client makes calls to Visier's Data Model API.To run the example below, add the following import statement to your program.fromvisier.apiimportModelApiClientIn this example, we query for the metadata of selection concepts on theRequisitionanalytic object:isRequisitionbyOtherIncomingReasonsandisActiveRequisition.withVisierSession(auth)assession:model_client=ModelApiClient(session)concepts=model_client.get_selection_concepts("Requisition",["isRequisitionbyOtherIncomingReasons","isActiveRequisition"])print(concepts)Direct Data Intake APIThe Direct Intake API allows users to load data whose structure already matches the target analytic object. To instantiate the Direct Data Intake API, follow the same pattern as the Data Query API and Data Model API.
First, import the appropriate API client:from visier.api import DirectIntakeApiClientNote:The Direct Data Intake API requires that your source data is  cleansed, deduplicated, and transformed before sending the data to Visier. If any of these criteria are not met, the Direct Data Intake API is not suitable for loading data. You may instead send data to Visier through alternative methods, such as SFTP or data connectors.The Direct Data Intake API call sequence follows a transactional pattern. You first start a transaction, then make the required file uploads to the transaction, and finally commit the  transaction. If a transaction must be aborted, you can instead roll back the transaction instead of committing it.:warning: The API caller must have sufficient capabilities in Visier to successfully call the Direct Data Intake API.Data Load SchemaTo understand the required structure to load data into target objects, use theschemasendpoint. Theschemasendpoint returns the data load schema, or staging schema, of a specified target object.schema=intake_client.get_object_schema("Employee_Exit")The data load schema is distinct from the analytic schema retrieved through the Data Model API. The analytic schema defines an object's attributes, such as its display name and associated dimensions. The data load schema defines an object's mandatory and optional data columns, data type, and other schema details.File UploadIn this example, we load data forEmployeeandEmployee_Exit.withVisierSession(auth)assession:intake_client=DirectIntakeApiClient(session,raise_on_error=True)try:response=intake_client.start_transaction()tx_id=response.json()["transactionId"]print(f"Transaction ID:{tx_id}")response=intake_client.upload_file(tx_id,"Employee","/tmp/data/employee-data.zip")print(response)response=intake_client.upload_file(tx_id,"Employee_Exit","/tmp/data/exits.csv")print(response)response=intake_client.commit_transaction(tx_id)print(response)except:print(f"Intake failed. Rolling back{tx_id}")intake_client.rollback_transaction(tx_id)Other UsesIn addition to querying data, this connector provides a lower level, generic function for executing other public Visier APIs. The following example illustrates how this connector can determine all the plans defined on a plan model.defget_location_levels(context:SessionContext)->Response:path="/v1/data/model/plan-models/WorkforcePlanModel/plans"returncontext.session().get(url=context.mk_url(path))withVisierSession(auth)ass:levels=s.execute(get_location_levels)print(levels.json())InstallationAddvisier-connectoras a dependency to your module or installpip install -U visier-connectordirectly.