tinybird-cli 6.2.1.dev0
Tinybird Command Line Tool
Requires Python: <3.14,>=3.10

Tinybird CLI
=============

The Tinybird command-line tool allows you to use all the Tinybird functionality directly from the command line. Additionally, it includes several functions to create and manage data projects easily.

Changelog
----------

6.2.0
***********

- `Fixed` `tb push` with `SHARED_WITH` now uses the user token to fetch workspaces, avoiding failures caused by using the wrong token

6.0.2
***********

- `Changed` `IMPORT_FORMAT` is now optional for S3/GCS data sources and will be automatically inferred from the file extension if not provided

6.0.1
***********

- `Fixed` false circular dependency detection when a shared datasource has the same name as a local pipe

6.0.0
***********

- `Removed` support for Python 3.9. The minimum supported Python version is now 3.10.
- `Removed` BigQuery and Snowflake CDK connector commands from `tb connection create`

5.22.2
***********

- `Fixed` tokens in existing datasource files not being updated when using `tb push -f`

5.22.1
***********

- `Fixed` RuntimeWarning for unawaited coroutines when `tb pull` encounters server errors

5.22.0
***********

- `Added` `tb init --git --override-commit` now works on workspaces without an existing release, allowing you to initialize git deployments from scratch
- `Added` `tb release create --commit` option to specify a specific git commit instead of using HEAD
- `Improved` `tb fmt` now supports `{% end if %}` and `{% end for %}` tag variations in addition to the standard Tornado-style `{% end %}` tags
- `Improved` Error message when trying to format a file using `tb fmt` or when we are running `tb init --git` to include the resource name and source (local/remote)

5.21.1
***********

- `Fixed` The `--on-demand-compute` flag will now ask for confirmation.

5.21.0
***********

- `Added` `tb pipe populate` and `tb push <PIPE> --populate` has a new `--on-demand-compute` flag to enable Compute-Compute Separation in populates.
- `Added` When trying to push a materialized view with an unoptimized join, the CLI will now warn the user.

5.19.0
***********

- `Added` `northamerica-northeast2` region to the list of available regions.

5.18.0
***********

- `Deprecated` `tb connection` no longer offers the S3 (AccessKey + Secret) option. Use S3 IAM role instead.
- `Deprecated` `tb connection` no longer offers the BigQuery option. The BigQuery connector is deprecated. Use ingestion from GCS instead.

5.17.0
***********

- `Added` internal changes

5.16.0
***********

- `Added` support for organizations on `tb workspace create`
- `Added` support for Python 3.13
- `Updated` dependencies:
    - croniter: from 1.3.8 to 1.3.15
    - clickhouse-driver: from 0.2.7 to 0.2.9
    - clickhouse-toolset: from 0.33.dev0 to 0.34.dev0
    - click: from 8.1.7 to 8.1.8

5.15.1
***********

- `Fixed` `unsupported operand type(s) for |: 'type' and '_GenericAlias` bug that was causing the CLI to fail when using Python 3.9 in some cases.
- `Added` support for `s3_iamrole` as service type for S3 Data Sources on `tb datasource sync`.

5.15.0
***********

- `Removed` Snowflake connector command from `tb connection create`.

5.14.0
***********

- `Added` support for `KAFKA_KEY_FORMAT` and `KAFKA_VALUE_FORMAT` parameters for kafka datasources.

5.13.3
***********

- `Fixed` updating the cron schedule of a BigQuery data source when having a query but no external_data_source setting.
- `Fixed` handle Kafka connector with multiple brokers properly.

5.13.2
***********

- `Fixed` Correctly handle Kafka connector with multiple brokers

5.13.1.dev2
***********

- `Added` comply with [PEP 625](https://peps.python.org/pep-0625/)
- `Added` `eu-west-1` region to the list of available regions.

5.13.1
***********

- `Added` support for `EXPORT_WRITE_STRATEGY` parameter for sinks, in `.pipe` files.
- `Changed` environment variables are now evaluated in the SQL nodes.

5.13.0
***********

- `Added` additional metadata to `tb branch ls` command
- `Changed` dependencies to be less restrictive:
    - cryptography: from ``>=41.0.0`` to ``~=41.0.0``
    - snowflake-connector-python: from ``==3.12.3`` to ``~=3.12.3``

5.12.0
***********

- `Added` support for `KAFKA_SASL_MECHANISM` in `.datasource` files.
- `Changed` Upgrade clickhouse-toolset to 0.33.dev0 to support new ClickHouse JSON type.
- `Changed` multiple dependencies:
    - cryptography: from ``>=3.4.8`` to ``>=41.0.0``
    - GitPython: from ``==3.1.32`` to ``~=3.1.32``
    - humanfriendly: from ``==8.2`` to ``~=8.2``
    - pydantic: from ``==2.5.2`` to ``~=2.8.0``
    - tornado: from ``==6.0.0`` to ``~=6.0.0``
    - snowflake-connector-python: from ``==2.7.1`` to ``==3.12.3``

5.11.0
***********

- `Changed` Drop support for Python 3.8

5.10.2
***********

- `Changed` Bump tinybird-cli version in requirements.txt when running `tb init --git`
- `Changed` Use `aiofiles` for async operations
- `Fixed` `TAGS` format was missing quotes in `tb pull``
- `Fixed` some internal typing issues from the CLI. Something internal that we don't include in the changelog of the CLI.

5.10.1
***********

- `Fixed` Internal Error while pushing S3 Data Source which provided the wrong cron expression to the backend.

5.10.0
***********

- `Added` support for Kafka Data Sources with CA certificate.
- `Fixed` Correctly parse lambda expressions in indexes
- `Changed` Upgrade clickhouse-toolset to 0.32.dev0
- `Added` new "File not found" error to `tb check` when including files from missing paths.
- `Added` support for Kafka Data Sources with CA certificate.
- `Changed` Upgrade clickhouse-toolset to 0.32.dev0
- `Fixed` Correctly parse lambda expressions in indexes

5.9.0
***********

- `Added` `tb pull` has a new `--fmt` flag to apply the same formatting as `tb fmt` when pulling a datafile.
- `Changed` `tb auth info` and `tb auth ls` now show the region API host properly.
- `Removed` misleading output message when `tb auth --token` with TB_TOKEN env var also defined.
- `Fixed` error messages when performing operations without being authenticated or using a token.
- `Fixed` `tb auth` uses explicitly the `--host` if provided.

5.8.1
***********

- `Improved` error message of `tb push` when pushing .datasource files.

5.8.0
**********

- `Added` `tb datasource scheduling` commands to manage the scheduling of a Data Source
- `Added` `tb tag` commands to manage tags from the CLI.
- `Added` support to `TAGS` in `tb fmt`.
- `Added` support to `TAGS` in `tb pull` and `tb push`. Allows tagging resources for filtering in the UI.
- `Changed` Improved message when creating a DynamoDB connection

5.7.0
**********

- `Changed` support `prod` as main git branch name when running `tb init --git`
- `Fixed` issue with `tb auth` ignoring the token value when using `--region` and `--token` options together.
- `Fixed` `tb auth` not getting the proper value when using the `TB_TOKEN` env var.

5.6.0
**********

- `Added` Add `IMPORT_FROM_TIMESTAMP` to the list of supported directives in the CLI for S3 Data Sources, files added to the S3 bucket before that time will be ignored. [Read our docs for more information.](https://www.tinybird.co/docs/ingest/s3)
- `Fixed` region list for AWS and custom regions when usign `tb auth --interactive`

5.5.0
**********

- `Added` Support `gte` and `lte` in `sql_and` directive. See https://www.tinybird.co/docs/cli/advanced-templates#sql-and
- `Changed` info and error messages when using "user token" vs "admin your@email.com" token
- `Fixed` regions list when usign `tb auth --interactive`, `tb auth ls` and `tb auth info`

5.4.0
**********

- `Added` support to Python 3.12
- `Added` support to Stream Pipes in `tb push` and `tb pull` commands. (Feature in development. It requires access)
- `Added` `json` and `jsonl` as supported formats for S3 Data Sources
- `Changed` `tb init --git` now supports run from a `develop` branch in git
- `Fixed` Do not try to deploy shared Datafiles from the `vendor` folder on `tb deploy`

5.3.0
**********

- `Added` `tb push` now supports `dynamodb` as service type
- `Added` `tb connection create` now supports `dynamodb` as service type
- `Added` error when trying to push a data source with `SETTINGS` instead of `ENGINE_SETTINGS`
- `Added` support for buckets with gzip files when creating S3 Data Sources

5.2.1
******

- `Added` `tb push` prevents pushing pipes that have nodes with the same name as the pipe.
- `Changed` helper messages to rename `Token` to `Static Token`
- `Fixed` error messages when using `tb push` with already taken names for other resources.

5.2.0
******

- `Added` internal changes
- `Added` `tb pipe unlink` command
- `Changed` No need for `--node` param in `tb pipe populate`

5.1.0
******

- `Added` COPY pipes now support `COPY_MODE replace`.
- `Added` improve checks for dependencies in pipes when altering Data Sources.

5.0.0
******

- `Breaking change` Make `insertion_date` column explicit. This column is no longer inferred, it must be present in the Data Source file.

  Detailed info at https://www.tinybird.co/docs/changelog/2024-06-17-insertion_date-deprecation

- `Added` parameter to `tb pipe regression-test` --relative-change
- `Added` `tb push` displays warnings when using a reserved parameter in a pipe.
- `Added` --role parameter to `tb workspace members add`
- `Changed` Point region hosts to new `app.tinybird.co` domain
- `Improved` help message of `tb workspace members set-role` now displays the 3 valid roles: viewer|guest|admin.
- `Improved` syntax error messages when parsing endpoints.

4.1.1
************

- `Fixed` workspace info gathering when switching branches. We were prioritizing the general workspace properties over the user ones.

4.1.0
************

- `Added` `tb token create` command to be able to create static and JWT tokens from the CLI. You can check more information at https://www.tinybird.co/blog-posts/jwt-api-endpoints-public-beta
- `Fixed` `tb init --git` to pin `tinybird-cli>=4,<5` in `requirements.txt` to avoid issues with the latest version of the CLI.

4.0.0
************

This is a major release, please read the commands affected below and consider updating your scripts and workflow before upgrading to this version.

- `Deprecated` `--semver` flag and `tb release` commands are now deprecated. You can keep using `tb deploy` to integrate and deploy from git. Changes are deployed to the main Workspace instead of to a Release.
- `Removed` `--cicd` flag and CI/CD templates generation from `tb init`. You can still use the git integration, just create your own pipelines. You can use the ones in this repo as an example https://github.com/tinybirdco/ci
- `Removed` `tb env` command is removed, use `tb branch` instead.
- `Deprecated` .datasource files with `ENGINE "Join"` is deprecated, use `Engine "MergeeTree"` instead.
- `Deprecated` `tb materialize`
- `Removed` Drop the `--timeout` flag from `tb push` which made the populate job to timeout. You can use now `--wait` to wait for the job to finish or nothing to just create the job and return.
- `Removed` Support for `KEY` directive is removed. The `KEY` was used to create a Data Source with Join engine by the given `KEY` column name. Join engines are also deprecated, you can use a regular `MergeTree` Data Source instead and adapt the pipes SQL accordingly.

3.12.0
************

- `Added` new "us-west-2" region to `tb auth ls`
- `Fixed` "Appended 0 new rows" message when appending NDJSON

3.11.0
************

- `Added` confirmation step to `tb datasource sync` command
- `Fixed` support for KAFKA_STORE_BINARY_HEADERS for new Data Sources. Only relevant when KAFKA_STORE_HEADERS is also enabled, used to store Kafka headers as binary data (default option from now on).
- `Fixed` Report error when `tb push --force --yes` cannot apply a change in a .datasource schema
- `Fixed` Skip error when using `tb push --only-changes` and no local .git directory is present

3.10.0
************

- `Added` support for unlink materialized pipes doing `tb workspace clear`
- `Added` support for adding multiple tokens when pushing a `.datasource` file
- `Fixed` s3 iamrole connection creation will not fail when `pbcopy` dependency is not available
- `Fixed` Fixed a bug on data branching
- `Fixed` Support ngram index

3.9.0
************

- `Added` Support for connection names when doing `tb connection rm`
- `Added` New `--policy` option for `create s3_iamrole` command that will generate different hints depending on the case
- `Fixed` Avoid system vars evaluation when doing `tb fmt`
- `Fixed` environment variables substitution for Data Source engine parameters.


3.8.0
************

- `Added` Support for S3IAM role for ingest
- `Changed` Upgrade clickhouse-toolset to 0.30dev0
- `Changed` Decouple cli from connector_settings module

3.7.0
************

- `Changed` fixed major version of tinybird-cli to lower than 4 when using `tb init --git`
- `Changed` behavior when running `tb deploy` on a branch to push the connection settings to the backend. This change is the backend that decides what to do.
- `Changed` `tb pipe sink` commands are now available
- `Fixed` regression tests query when filtering by specific parameter
- `Fixed` Avoid `tb fmt` to error if there's a `CASE` in the sql
- `Fixed` Some issues when using `--fmt: off` with `CASE` in the sql
- `Fixed` `tb deploy` supports deployment when a `.incl` file is removed
- `Fixed` Report better error message on `tb diff` when two Datafiles have the same name
- `Fixed` Added correct URL for checking available regions
- `Fixed` Support `.incl` files with environment variables in `tb fmt`

3.6.0
************

- `Added` Add new envvar `TB_INCL_RELATIVE_PATH` in .tinyenv to look for Datafiles out of the Data Project directory
- `Changed` On-demand Sink run checks the pipe type instead of checking the workspace connectors

3.5.0
************

- `Added` Token management in Copy pipes datafiles
- `Added` Better error report on `tb push` when trying to push a pipe with bad templating
- `Added` credential validation for S3 connections
- `Added` `tb connection create` now supports `s3_iamrole` as service type
- `Changed` error message when trying to modify a datasource without bumping the major semver version to add a link to the documentation.
- `Changed` GH workflow files and secret includes the workspace name when generated from the UI and CLI
- `Changed` rename `INDICES` to `INDEXES`
- `Changed` Save git commit to workspace on `tb push --only-changes`
- `Fixed` Fix error message when deleting an `.incl` file with the git workflow
- `Fixed` support `tb deploy --dry-run`


3.4.0
************

- `Added` Support `tb deploy` .datasource with `INDEXES`
- `Added` Support `tb push` .datasource with `INDEXES`
- `Added` Support `tb deploy` in a post release of a .datasource with `INDEXES`
- `Added` Check backfill on preview is required when `tb deploy` modified datasources. Disable check with `TB_CHECK_BACKFILL_REQUIRED=0`.
- `Changed` Allow spaces when managing multiple Workspace members
- `Changed` `tb deploy` to not allow pushing materialized views with datasource definition in the same datafile. We are blocking this as has unsupported behavior and it misleading and less flexible than having 2 files one for the datasource and one for the materialized view.
- `Changed` `tb deploy` to deploy the datasources from top to bottom to make sure that if we modified the column in some datasource in the top. It will be deployed before the datasources that depend on it.
- `Changed` Post-release required when deploying changes that include changing a Copy Pipe.
- `Changed` script `exec_test.sh`  from `tb init --git` to run retries to avoid flakiness in tests
- `Changed` Update `tb release rm --oldest-rollback` to use new API
- `Fixed` Post-release recommended instead of required when deploying changes that include changing a Copy Pipe. Displays a warning when other number of the semver is increased.
- `Fixed` Fixed bug when a Data Source was created using an INDEX
- `Fixed` Avoid None git diff change type
- `Fixed` `tb deploy` to deploy the endpoints in the correct order when there are endpoints depending on other endpoints.
- `Fixed` `tb push` pipe with multiple tokens
- `Fixed` `IMPORT` and `ENGINE` changes were not detected `tb deploy`
- `Removed` part of the code supporting python 3.6 as we stopped giving support 18 month ago


3.3.0
************

- `Added` Show warning when doing data or pipe operations directly against Live.
- `Added` `bi_stats`, `bi_stats_rt` and `endpoint_errors` to the list of service data sources that the CLI ignore when looking for dependencies of pipes being pushed.
- `Added` S3 Iam Role connection creation
- `Added` Connector credential validation
- `Added` Data quality tests inside a `skip_in_branch` folder inside the `tests` folder are skipped when running on a branch. Use this to skip tests in the CI piipeline that should only run with production data.
- `Changed` Stop creating endpoints folder on tb init
- `Changed` Small refactoring of the connector credentials
- `Fixed` Progress bar in `tb init --git`.
- `Fixed` Don't do diff when there are no modified resources on `tb deploy`.
- `Fixed` Upgrade GitPython dependency to 3.1.32. That version fixes the security vulnerability `CVE-2022-24439 <https://github.com/advisories/GHSA-hcpj-qp55-gfph>`
- `Fixed` Homepage URL in PyPI

3.2.0
************

- `Changed` `tb init --git` uses v3.1.0 of the CI templates (See release notes: https://github.com/tinybirdco/ci/releases/tag/v3.1.0)
- `Changed` Allow alter data source with JSON paths to add DEFAULTS if the data source already has any (See release notes: https://github.com/tinybirdco/ci/releases/tag/v3.1.0)

3.1.0
************

- `Added` `tb init --cicd` to generate CI/CD files not only when using `--git`. Combine with `--force` to overwrite
- `Added` Ignore format changes on `tb deploy` by default
- `Changed` `tb init --git` and `tb release generate` include `set -euxo pipefail` in the generated bash scripts
- `Changed` `tb init --git` uses github.com/tinybirdco/ci => v3.0.0
- `Changed` Internal changes to allow parametrize fmt
- `Changed` `tb release promote` automatically deletes de oldest rollback Release if `export TB_FORCE_REMOVE_OLDEST_ROLLBACK=1`
- `Changed` option `--node` in `tb pipe populate` to required as it is already mandatory to specify the node to populate. Now, it will provide an error message if the `--node` option is not provided.
- `Changed` Better error report on `tb fmt --diff`. Use it in CI to standardize Datafile formatting
- `Changed` `tb init --git` generates a requirements.txt file to install the latest tinybird-cli release from PyPI
- `Changed` `tb deploy` shows an example when failing because of a missing `--semver` argument
- `Fixed` When parsing the response of a CSV `tb sql` command, the CLI was interpreting the result as an error in some cases.

3.0.0
************

Major version updated to 3.0.0 due to the change in the logic of `tb deploy`. From this version on, `tb deploy` promotes Releases to Live or Preview based on the semver version.

See `our Version Control guides <https://www.tinybird.co/docs/version-control/deployment-strategies.html#semver-deployment-behaviour>`_ for more details about our semver deployment behaviour.

- `Added` `syncing` status to git remote settings
- `Added` `tb release rm --oldest-rollback --force --yes` deletes the oldest rollback Release by creation date
- `Changed` Updated logic in `tb deploy`
- `Changed` Updated `scripts/exec_test.sh` generated from `tb init --git`
- `Changed` Pointing to versions.tinybird.co for docs pages related to Versions
- `Changed` Initial git files to include mandatory directories
- `Changed` Unify generation of ci/cd script resources
- `Changed` Update clickhouse-toolset to 0.29.dev0
- `Changed` When removing a Release, the CLI will now inform if Share Data Sources are still being used in a Materialized View in other Workspaces.
- `Changed` Update CLI to generate ci/cd resources: new `append_fixtures.sh` and updated `exec_tests.sh`
- `Changed` Update `tb release generate` to generate `deploy.sh` and `postdeploy.sh` with instructions on how they work
- `Changed` `.tinyenv` default template includes all possible vars
- `Changed` behaviour of `tb env regression-test` when no requests are found. Now, it will show a warning message and will not fail the command.
- `Changed` Add usage validation on `tb release rm` and allow `--force` to avoid the validation
- `Changed` Support raw content on datafile parses
- `Fixed` `tb deploy` now fails with exit 1 in case a release limit has been reached.
- `Fixed` `tb deploy` validates post --semver
- `Fixed` `append_fixtures.sh` script to exit with 1 when there's an error
- `Fixed` Error while pushing Sink Pipes using IAM Role Connector for S3
- `Fixed` Deploy pipes with shared Data Sources using `tb deploy`
- `Fixed` Bug on `tb deploy` which did not preserve a proper order when deploying resources
- `Fixed` Small error message misspell. The argument to use the User token in certain operations like sharing a Data Source is `--user_token` and was written as `--user-token`.
- `Removed` `--all` argument from `tb branch`


2.2.0
************

- `Changed` `tb init --git` update to get ready to use cicd with releases
- `Changed` Use `tb branch` instead of `tb env`. `tb env` is still usable but deprecated and will be removed in the next version.
- `Fixed` command `tb pipe regression-tests` to handle the edge case when all the requests to the endpoint that we want to check fails as the value returned is no what we expect.
- `Fixed` `tb deploy` fixes to work with `--dry-run` and push changes of new and modified files
- `Fixed` `tb check` doesn't exit with error when checking a `.incl` file.
- `Fixed` `tb fmt` adds empty quoted strings in .datasource connection section.
- `Fixed` Be smarter in `tb fmt` when detecting if `.incl` file contains Data Source or Pipe definitions.
- `Fixed` `tb env use` when using a token directly, without a `.tinyb` file.


2.1.0
************

- `Changed` `exec_test.sh` now runs with `parallel -j 4` to speed up CI cycles
- `Changed` `tb release rollback` now requires double confirmation
- `Fixed` Ignore .diff_tmp/ on `tb deploy`
- `Changed` Regression tests select faster the requests used for the checks.
- `Improve` peformance when using the option `--only-change` in the `tb push` command. This was a regression introduced in the version 2.0.0
- `Added` Add option `c` or `--concurrency` to `tb run test` to run tests in parallel
- `Removed` hidden option `--only-changes` from `tb deploy` as it's no longer required.
- `Changed` Speed-up CLI startup by lazy-importing some dependencies.
- `Added` `tb auth` shows and uses the new AWS regions
- `Added` Add `--auto-promote` to `tb deploy` to automatically promote the Release created. It automatically applies to Environments created by CI.
- `Added` Support `TAGS "backfil_column=<column_name>"` in .datasource definition to do backfill operations when running a populate job
- `Changed` Regression tests now also use Pipe requests done with the POST HTTP method.
- `Changed` `tb env` commands to use the new `CLIConfig` class to manage the authentication and the configuration of the CLI. This change allows us to have a better control of the configuration and the authentication of the CLI. This should be transparent for the users.
- `Changed` Internal changes in `tb deploy`
- `Added` `tb deploy` removes resources from datafiles deleted


2.0.0
************

Released new version 2.0.0 with all these changes:

- `Changed` All commands exit with 1 when there's an exception
- `Removed` Prefixes are not supported anymore ([context](https://www.tinybird.co/changelog/2023-10-09-versions), and [how to migrate](https://www.tinybird.co/docs/guides/staging-and-production-workspaces.html#migrating-from-prefixes))
- `Removed` `tb pipe create`


1.3.0
************

Released new version 1.3.0 with all these changes:

- `Added` Support for Copy pipes in `tb fmt`
- `Added` Support for `.datasource.incl` files in `tb fmt`
- `Fixed` `tb auth use` to move between regions saved in the config file
- `Fixed` Skip `regression.yaml` in `tb test run` since it's reserved for regression tests
- `Fixed` Skip replacing `.incl` files in `tb fmt`


1.2.0
************

Released new version 1.2.0 with all these changes:

- `Changed` Upgrade CICD version templates for `tb init --git`
- `Changed` Support `tb release` commands in Environments
- `Fixed` Correct paths for triggering CICD generated by `tb init --git`
- `Fixed` rolled back the version of the snowflake-connectors-python package.
- `Fixed` Do not print data branching summary in `tb env create <env_name>`


1.1.0
************

Released new version 1.1.0 with all these changes:

- `Added` Support `skip` individual regression tests through the regression.yaml configuration file. CI/CD guide at https://www.tinybird.co/docs/guides/continuous-integration.html
- `Added` Fail `tb env regression-tests` if no tests are run for any pipe, use `skip` to avoid the error
- `Added` Skip POST requests by default in `tb env regression-tests` until it's supported
- `Added` Support append `--fixtures` to shared Data Sources in Environments
- `Added` support `--assert-max-time` in `tb env regression-tests` so that serves as threshold for response time and avoid flakiness on fast endpoints.
- `Added` support `tb env regression-tests --main -f regression.yaml` so you can run regression tests agains the main Environment together with a configuration file.
- `Added` `tb env regression-tests coverage --main` runs coverage regression tests in the main Environment. This command is experimental and undocumented.
- `Changed` `tb init --git --folder <folder>` includes the `<folder>` in the CI/CD templates as `data_project_dir` and in the paths to track changes so CI and CD only run when there's a change on that folder. If `<folder>` is not provided then it is set to the current directory `.`
- `Changed` Force `tb init --git` to run in the `main` or `master` git branch
- `Changed` the creation of the CD/CI workflow when running `tb init --git` to use the admin workspace token instead of the token of your admin user.
- `Changed` Expose resources and ids when removing releases `tb release rm`
- `Changed` `--override-commit` option to not require needing to pass the `--git` option as well. Now, running `tb init --override-commit` should work as expected.
- `Changed` Copy and Sink on-demand jobs now return the job URL.
- `Changed` allow Workspace admin tokens for CI/CD workflows. Previously, you were asked for user admin token. You can update your TB_ADMIN_TOKEN to be the Workspace admin token instead the admin token from a specific user. CI/CD guide at https://www.tinybird.co/docs/guides/continuous-integration.html
- `Changed` Improve CI/CD output. Added warning hint in custom deployments.
- `Fixed` `tb deploy` raises an error if it detects resources renamed in git
- `Fixed` On `tb push --force --yes` a BigQuery Data Source, it updates both the `SCHEMA` and connector settings.
- `Fixed` Connection settings of BigQuery Data Sources are preserved in Environments, although the connection does not sync by design, to ingest data use `--fixtures`
- `Removed` warning message that recommended not using the admin token for CI/CD workflows as we introduced a fix in the version 1.0.1 to avoid the problems that the admin token was causing.
- `Removed` fallback when running `tb pipe regression-test` does was doing requests to `/v0/pipes/XXX/requests` as it was mostly not used. This endpoint was being called when there weren't requests in `tinybird.pipe_stats_rt` in the last 7 days. This endpoint was really heavy and most of the time wouldn't return any actual data as endpoint not being called after 7 days most likely are not in production.
- `Removed` `async` parameter in the `/v0/environments` requests as it's not being used.


1.0.1
************

Released new version 1.0.1 with all these changes:

- `Added` Google Cloud Storage as a Data Source connector (BETA)
- `Added` support for fixtures using Parquet format. Also, we have modified the message when no fixture is found. Before it would show something like `** Warning: datasources/fixtures/current_events.ndjson file not found` as it was the last format was looking for, now it will display `** Warning: No fixture found for the datasource current_events`
- `Added` `tb deploy` output now exposes what is detecting for deploying
- `Added` `tb env regression-tests` commands now support flag `--skip-regression-tests` or `export TB_SKIP_REGRESSION=1` to skip execution of regression tests. Handy for CI environments were regression tests might be flaky
- `Changed` `tb init --git` uses v2.3.0 of the CI/CD templates. See [release notes](https://github.com/tinybirdco/ci/releases/tag/v2.3.0)
- `Changed` `tb auth info` to give better information in case of authentication problems
- `Changed` When running `tb init --git` for the first time the `exec_test.sh` script to run fixtures tests now includes an environment variable `export TB_VERSION_WARNING=0` to skip the CLI version warning
- `Changed` Support `OBFUSCATE_REGEX_PATTERN` and `OBFUSCATE_PATTERN_SEPARATOR` environment variables to define a regex pattern and a separator (in case of a single string with multiple regex) to obfuscate secrets in the CLI output
- `Changed` Update clickhouse-toolset to 0.28.dev0
- `Fixed` Skip error when `tb deploy --populate --wait` and a copy pipe has been deployed
- `Fixed` Typo when there is an error with a connection
- `Fixed` that all the HTTP requests follow the environment variable `TB_DISABLE_SSL_CHECKS`. This environment variable allows users to disable the SSL verification if needed. By default is not disabled. Before this version, some HTTP request were not disabling the SSL validation
- `Fixed` `tb env rm` and `tb env use` when using the admin workspace token. Before when using these command and using the admin token, this command would return an error saying that the workspace was not found. Now, these commands should work as expected
- `Fixed` `tb connection ls` now lists BigQuery connections
- `Fixed` `tb pull` and `tb diff` now respect multiline in `IMPORT_QUERY`


1.0.0
*******

- `Released` Version 1.0.0, from now on the tinybird-cli package uses the standard semver convention for stable versions. Development versions will be tagged as with the `.devX` suffix where `X` is an integer number.

1.0.0b448
*********

- `Changed` Better error feedback when Environment is not ready for `tb deploy`

1.0.0b447
*********

- `Added` `tb datasource delete` new option `dry-run` to check number of maching rows before execute

1.0.0b446
*********

- `Added` `tb token ls` to list all the available tokens
- `Added` `tb token copy` to copy a token to the clipboard
- `Added` `tb token rm` to remove a token
- `Added` `tb token refresh` to refresh a token
- `Added` `tb token scopes` to list the scopes of a token

1.0.0b445
*********

- `Changed` `tb env create` waits job by default. Use `--no-wait` in case you wan't to avoid waiting.
- `Changed` Better error feedback for `tb deploy` when Environment creation is not completed

1.0.0b444
*********

- `Fixed`: Data Sources definition: Default column modifier not allowed when there's json paths

1.0.0b443
*********

- `Changed` Bump CI/CD release to v2.2.0 => https://github.com/tinybirdco/ci/releases/tag/v2.2.0

1.0.0b442
*********

- `Fixed` Check if not main when creating an Environment

1.0.0b441
*********

- `Changed` `tb token --git` unhidden

1.0.0b440
*********

- `Changed` `tb token rm` needs to be confirmed or run with `--yes` flag to be removed

1.0.0b439
*********

- `Added` References to the latest release of our CI/CD actions

1.0.0b438
*********

- `Fixed` `tb diff` just using progressbar in `tb init --git`
- `Fixed` `tb env rm` returns proper error when trying to remove from another Environment
- `Changed` `tb auth` better output suggesting to use region host

1.0.0b437
*********

- `Fixed` `tb check` compatility with tokens Datafiles
- `Fixed` `tb token rm` returns proper error when token does not exists

1.0.0b436
*********

- `Changed` Internal changes

1.0.0b435
*********

- `Fixed` `tb init --git` fails if diffs detected and not executing `tb pull --auto --force`

1.0.0b434
*********

- `Changed` Versions Gitlab ci/cd pipeline to ensure temporal envs are removed

1.0.0b433
*********

- `Addded` `tb deploy` exposes `git diff` output before pushing resources.

1.0.0b432
*********

- `Changed` multiple dependencies to be less strict:
    - click: From ``==8.1.6`` to ``>=8.1.6,<8.2``
    - cryptography: From ``==3.4.8`` to ``>=3.4.8``
    - pyyaml: From ``==6.0`` to ``>=6.0,<6.1``
    - requests: From ``==2.28.1`` to ``>=2.28.1,<3``
    - urllib3: From ``==1.26.14`` to ``>=1.26.14,<2``

1.0.0b431
*********

- `Changed` `tb pull` defaults now to `--auto` True, that means that it preserves the Data Project default folder structure. You can still pull all files to the current folder by using the `--no-auto` flag.

1.0.0b430
*********

- `Deprecated` `tb pipe new` command is deprecated

1.0.0b429
*********

- `Changed` `--prefix` deprecation message to indicate next EOL on 20th Dec.

1.0.0b428
*********

- `Added` support for the new Versions feature.

1.0.0b427
*********

- `Changed` `tb pull` omits `insertion_date` meta-column

1.0.0b426
*********

- `Added` Commands for the new Versions feature (https://www.tinybird.co/blog-posts/git-for-real-time-data-projects).

1.0.0b425
*********

- `Changed` Don't run regression checks on `tb deploy`

1.0.0b424
*********

- `Fixed` Bug in `tb env regression-tests` when `--no-assert-result-rows-count` provided

1.0.0b423
*********

- Tokens are hidden in the output by default
- `Added` option to show token values in the output using the `--show-tokens` flag.
- Deprecated to hide token values in the output using the `--hide-tokens` flag.

1.0.0b422
*********

- `Fixed` Errors in the last release.

1.0.0b421
*********

- `Fixed` Provide feedback error message for erroneous commit on 'tb deploy'

1.0.0b420
*********

- `Changed` The way the command `tb workspace connect` handles connections.

1.0.0b419
*********

- `Fixed` `tb init` Avoid to create example tests when tests folder already exists

1.0.0b418
*********

- `Added` Option '--commit' to provide and force git commit reference to 'tb init --git'

1.0.0b417
*********

- `Fixed` 'tb push' and 'tb deploy' just pushes pipe endpoints at the end when no forward deps

1.0.0b416
*********

- `Added` `tb init --git` Checks that .diff_tmp is added to .gitignore. Throws an error if it's not.

1.0.0b415
*********

- `Changed` the behaviour of `tb pull`, `tb datasource ls` and `tb pipe ls` to not detect the prefixes in the resources by default. In case, you want to use `tb pull`, `tb datasource ls` or `tb pipe ls` in the same way as before, you will need to specify which prefix you want to detect using `--prefix`.

1.0.0b414
*********

- `Fixed` `tb init --git` unexpected error when using token not associated to user

1.0.0b413
*********

- `Changed` Improved error outputs working with environments

1.0.0b412
*********

- `Changed` `tb init --git` cicd files are generated after release is finished

1.0.0b411
*********

- `Added` `tb test` pipe endpoint test example on 'tb init' data project generation

1.0.0b410
*********

- `Changed` `tb init --git` order of execution to be more reliable

1.0.0b409
*********

- `Changed` `tb deploy --fixtures` to append the fixtures instead of replacing the data.
- `Fixed` `tb deploy --fxitures` to append the fixtures even if there is no other change

1.0.0b408
*********

- `Deprecated` Hide and deprecate warning for the `--prefix` flag in all commands. The prefix feature is superseded by Workspaces, use `tb workspace create` instead.

1.0.0b407
*********

- `Fixed` `tb token copy` not working properly in Linux

1.0.0b406
*********

- `Fixed` Fail `tb init --git` if there are untracked files in project paths

1.0.0b405
*********

- `Added` Extra information about the current token and host to the ``tb auth info`` command.

1.0.0b404
*********

- `Fixed` Command `tb workspace create` when running in a CI environment.

1.0.0b403
*********

- `Changed` behaviour of `tb push` to only execute the `SHARED_WITH` logic when passing the `--user_token` option

1.0.0b402
*********

- `Fixed` Pushing a token with SQL condition

1.0.0b401
*********

- `Changed` Shows instructions snippet when creating a Snowflake connection and password not provided.

1.0.0b400
*********

- `Added` Support SETTINGS clause in query definition. Only a few of them are enabled in the backend.

1.0.0b399
*********

- `Added` Allow to copy data between Data Sources releases

1.0.0b398
*********

- `Fixed` Not shared resources now go to the correct folder when doing `tb pull --auto`

1.0.0b397
*********

- `Added` Experimental token management from the CLI (Feature in development. It requires access)

1.0.0b396
*********

- `Changed` Remove S3 feature flags

1.0.0b395
*********

- `Added` support to `sink` Pipes in `tb push` command (Feature in development. It requires access)

1.0.0b393
*********

- `Changed` Upgrade version of generated cicd templates

1.0.0b393
*********

- `Fixed` `tb diff` now shows the correct message when one of its arguments is deprecated

1.0.0b392
*********

- `Changed` Internal changes

1.0.0b391
*********

- `Changed` Internal changes

1.0.0b390
*********

- `Improve` Feedback messages on `tb init`

1.0.0b389
*********

- `Changed` Python 3.7 is no longer supported. The minimum supported version is now Python 3.8.

1.0.0b388
*********

- `Fixed` `tb fmt` when used with include files that contains nodes that are not being used in the parent pipe

1.0.0b387
*********

- `Changed` creation of gcs connection when creating data connector for sinks to `gcs_hmac` from `gcs`

1.0.0b386
*********

- `Fixed` Adding a column to Kafka Data Sources with `tb push -f`

1.0.0b385
*********

- `Changed` Added an 'End of support for Python 3.7 starting on 2023-08-01' notice.

1.0.0b384
*********

- `Fixed` integration and stage are not prompted anymore when creating a Snowflake connection

1.0.0b383
*********

- `Fixed` bug introduced in version `1.0.0b375` causing the regression-test to fail when the size of the response length was different

1.0.0b382
*********

- `Fix` Environments minor fixes

1.0.0b381
*********

- `Improve` feedback messages in different commands

1.0.0b380
*********

- `Fix` small bug when trying to unshare a datasource from all the workspaces removing the `SHARED_WITH` entry

1.0.0b379
*********

- `Added` S3 connector creation

1.0.0b378
*********

- `Added` support of `SHARED_WITH` in `tb pull` and `tb diff`

1.0.0b377
*********

- `Update` click package dependency from 8.1.3 to 8.1.6

1.0.0b376
*********

- `Changed` `tb init` now creates additional folders

1.0.0b375
*********

- `Changed` Internal changes

1.0.0b374
*********

- `Fixed`  Overriding a pipe when changing nodes from endpoint to copy and vice versa when running `tb push` with the `--force` flag

1.0.0b373
*********

- `Changed` Internal changes

1.0.0b372
*********

- `Changed` Internal changes


1.0.0b371
*********

- `Changed` Internal changes

1.0.0b370
*********

- `Changed` Internal improvements.

1.0.0b369
*********

- `Changed` Filter requests with environment parameters in regression tests when running `tb push` or `tb pipe regression-test`

1.0.0b368
*********

- `Fixed` `PLAIN` default `--sasl-mechanism` value in `tb connection create kafka` command

1.0.0b367
*********

- `Changed` Internal changes

1.0.0b366
*********

- `Fixed` `tb append` now returns the correct message when the append is successful

1.0.0b365
*********

- `Fixed` `tb fmt` removes empty space after empty comment

1.0.0b364
*********

- `Changed` Internal changes

1.0.0b363
*********

- `Changed` Internal