Package: datacatalog-fileset-exporter
Datacatalog Fileset ExporterA Python package to manage Google Cloud Data Catalog Fileset export scripts.Disclaimer: This is not an officially supported Google product.Table of ContentsExecuting in Cloud Shell1. Environment setup1.1. Python + virtualenv1.1.1. Install Python 3.6+1.1.2. Get the source code1.1.3. Create and activate an isolated Python environment1.1.4. Install the package1.2. Docker1.3. Auth credentials1.3.1. Create a service account and grant it below roles1.3.2. Download a JSON key and save it as1.3.3. Set the environment variables2. Export Filesets to CSV file2.1. A CSV file representing the Filesets will be created2.2. Run the datacatalog-fileset-exporter scriptExecuting in Cloud Shell# Set your SERVICE ACCOUNT, for instructions go to 1.3. Auth credentials# This name is just a suggestion, feel free to name it following your naming conventionsexportGOOGLE_APPLICATION_CREDENTIALS=~/datacatalog-fileset-exporter-sa.json# Install datacatalog-fileset-exporterpip3installdatacatalog-fileset-exporter--user# Add to your PATHexportPATH=~/.local/bin:$PATH# Look for available commandsdatacatalog-fileset-exporter--help1. Environment setup1.1. Python + virtualenvUsingvirtualenvis optional, but strongly recommended unless you useDocker.1.1.1. Install Python 3.6+1.1.2. Get the source codegitclonehttps://github.com/mesmacosta/datacatalog-fileset-exportercd./datacatalog-fileset-exporterAll paths starting with./in the next steps are relative to thedatacatalog-fileset-exporterfolder.1.1.3. Create and activate an isolated Python environmentpipinstall--upgradevirtualenv
python3-mvirtualenv--pythonpython3envsource./env/bin/activate1.1.4. Install the packagepipinstall--upgrade.1.2. DockerDocker may be used as an alternative to run the script. In this case, please disregard theVirtualenvsetup instructions.1.3. Auth credentials1.3.1. Create a service account and grant it below rolesData Catalog Admin1.3.2. Download a JSON key and save it asThis name is just a suggestion, feel free to name it following your naming conventions./credentials/datacatalog-fileset-exporter-sa.json1.3.3. Set the environment variablesThis step may be skipped if you're usingDocker.exportGOOGLE_APPLICATION_CREDENTIALS=~/credentials/datacatalog-fileset-exporter-sa.json2. Export Filesets to CSV file2.1. A CSV file representing the Filesets will be createdFilesets are composed of as many lines as required to represent all of their fields. The columns are
described as follows:ColumnDescriptionMandatoryentry_group_nameEntry Group Name.Yentry_group_display_nameEntry Group Display Name.Yentry_group_descriptionEntry Group Description.Yentry_idEntry ID.Yentry_display_nameEntry Display Name.Yentry_descriptionEntry Description.Yentry_file_patternsEntry File Patterns.Yschema_column_nameSchema column name.Nschema_column_typeSchema column type.Nschema_column_descriptionSchema column description.Nschema_column_modeSchema column mode.N2.2. Run the datacatalog-fileset-exporter scriptPython + virtualenvdatacatalog-fileset-exporterfilesetsexport--project-idsmy-project--file-pathCSV_FILE_PATHHistory0.1.0 (2020-04-28)First release on PyPI.0.2.0 (2020-05-08)ADD option to use filesets creation date.