Package: bqup-spartez
bqupbqup is a backup tool for BigQuery projects. It can export a BigQuery
project's structure and source code while mimicking the hierarchy of
datasets and tables.For the full story of why we made bqup, check out ourblog
post!Installationbqup can be installed usingpip.$ pip install bqupAlternatively, you can also clone the repository then runinstall.$ git clone https://github.com/thinkingmachines/bqup.git
$ cd bqup
$ python3 setup.py installUsageCommand line optionsYou can list down the options by runningbqup --help.bqup [-p PROJECT_ID] [-d TARGET_DIR] [-fvx]

Options:
  -p PROJECT_ID, --project PROJECT_ID  Project ID to load. If unspecified,
                                       defaults to current project in
                                       configuration.
  -d TARGET_DIR, --dir TARGET_DIR      The target directory where the project
                                       will be written. Defaults to current
                                       timestamp.
  -f --force                           Overwrite target directory if it exists.
  -v --verbose                         Print a summary of the loaded project.
  -x --schema                          Export table schemata as json.DevelopmentSet upgcloudto run with your personal account (aka run with scissors).Set upapplication-default.$ gcloud auth application-default loginInstallwheel.$ pip3 install wheelInstall bqup.pip3 install -e .Alternatively, you can also install it using:python3 setup.py developRun bqup (seeUsage).ProductionNote: When deploying for a new GCP project, consider usingCloud
SchedulerTurn off your host Google Instance.Enable BigQuery on the instance's Cloud API Access ScopesStart the instance.sshinto the Google Instance you want to runbqupfrom.Authorize your Compute Instance Account to read from the target BigQuery project.Installbqupviapip install bqup, optionally inside avirtual environment.RunbqupIf it still doesn't work, check in IAM that the service account you are using has BigQuery read access.Setting up regular backupsOn the machine that will run your backups, set up your git config (username,
email, the usual).Make a directory to use as the Git repository. For this example, let's userepo:$ mkdir repo cd repo git initAdd the remote to the git repository (ideally a GCP repository). For this
example, let's usegoogle:$ git remote add google <url-to-remote-repository>Create a script calledbqup.shthat follows the following template. For
our example, our repository is dedicated to backups, so we just assume that
ourHEADis the latest and just push gently tomaster.#!/bin/bash
<absolute-path-to-bqup> -p <project-id> -d <absolute-path-to-repo>/projects -fv >> <absolute-path-to-log-file>
cd <absolute-path-to-repo>
date > last-updated.log
git add .
git commit -m "Automated bqup"
git push <remote> <branch>Add this script to yourcrontabto run as frequently as your heart desires.DistributionRunmake testto try a test upload.Runmake distto upload a distribution.Both of these will callmake build, which rebuilds the package locally.ContributingIf you wish to contribute, check out ourcontributing
guide!A list is maintained for all external contributors who have submitted pull
requests that were subsequently approved. Users are allowed and encouraged to
fork the project and submit pull requests and issues. We only request that
contributions adhere to these guidelines:Commit messagesGithub flowThe official maintainers in charge of responding to issues and merging pull
requests are:Pepe BawaganMark Steve SamsonCarlson ChengContributorsThanks to all these wonderful people who've helped out with bqup:JessRamPepe BerbaTim PronEnzoArdieDisclaimersbqup is maintained on abest effortbasis:No amount of official time is currently being dedicated to the regular
maintenance of this project.Thinking Machines does not make any guarantees about the quality of the
software.Thinking Machines reserves the rights to:refuse to resolve issuesclose issues without resolutionrequest changes to pull requestsreject pull requests outright