Package: permifrost
permifrostWe welcome contributions, so please feel free to submit MRs orissuesif you'd like to help in any way. To get started with contributions read theContributingsection at the bottom of this README to get started.InstallationInstall the most stable version using the following command:pip install permifrostIf you would like to work with the most up-to-date functionality in permifrost install directly from GitLab using the following command:pip install git+https://gitlab.com/gitlab-data/permifrost.gitUsageUse this command to check and manage the permissions of a Snowflake account.permifrost[-v]run<spec_file>[--role][--dry][--diff][--user][--ignore-memberships]#> permifrost run --helpUsage:permifrostrun[OPTIONS]SPECGrantthepermissionsprovidedintheprovidedspecificationfileforspecificusersandroles

Options:--dryDonotactuallyrun,justcheck.--diffShowfulldiff,bothnewandexistingpermissions,usewith-v.--roleTEXTRungrantsforspecificroles.Usage:--roletestrole--roletestrole2.--userTEXTRungrantsforspecificusers.Usage:--usertestuser--usertestuser2.--ignore-membershipsDonothandlerolemembershipgrants/revokes--helpShowthismessageandexit.Use this utility command to run the SnowFlake specification loader to confirm that yourroles.ymlfile is valid.permifrost[-v]spec-test<spec_file>[--role][--user][--ignore-memberships]#>  permifrost spec-test --helpUsage:permifrostspec-test[OPTIONS]SPECLoadSnowFlakespecbasedontheroles.ymlprovided.CLIuseonlyforconfirmingspecificationsarevalid.

Options:--roleTEXTRungrantsforspecificroles.Usage:--roletestrole--roletestrole2.--userTEXTRungrantsforspecificusers.Usage:--usertestuser--usertestuser2.--ignore-membershipsDonothandlerolemembershipgrants/revokes--run-listTEXTRungrantsforspecificusers.Usage:--usertestuser--usertestuser2.--helpShowthismessageandexit.Given the parameters to connect to a Snowflake account and a YAML file (a
"spec") representing the desired database configuration, this command makes sure
that the configuration of that database matches the spec. If there are
differences, it will return the sql grant and revoke commands required to make
it match the spec. If there are additional permissions set in the database this
command will create the necessary revoke commands with the exception of:Object OwnershipWarehouse PrivilegesFurthermore, if you are using the recommended role ofSECURITYADMIN,ALTER USER ...commands will fail on users that are owned byACCOUNTADMIN. In these circumstances, it is highly recommended to log into the Snowflake instance and update ownership of all users to belong toUSERADMINas per Snowflake recommended best practices.Lastly, note that the default roles cannot have their role hierarchies modified. As such, anyGRANT ROLE <default role> TO ROLE <default role>;will be excluded from the permission set generated by Permifrost.For example:...roles:public:member_of:-useradminsecurityadmin:member_of:-useradmin...Both of the above relationships will be skipped as this attempts to modify a default Snowflake permission structure which would generate an error on attempting to implement.Permifrost is heavily inspired bypgbedrockwhich can be used for
managing the permissions in a Postgres database.spec_fileThe YAML specification file is used to define in a declarative way the
databases, roles, users and warehouses in a Snowflake account, together with the
permissions for databases, schemas and tables for the same account.All permissions are abbreviated asreadorwritepermissions, with
Permifrost generating the proper grants for each type of object. This includes
shared databases which have simpler and more limited permissions than non-shared
databases.According to thereadvs.writepermissions approach, you should be able to
grant granular access likereadpermissions for usage of database and schema
andwritepermissions to insert data into a specific table within that
database and schema.Please find below the links between Permifrost permissions and Snowflake grants.ObjectsPermifrost permissionsSnowflake grantsDatabasesreadusagewritemonitor, create schemaSchemasreadusagewritemonitor, create table, create view, create stage, create file format, create sequence, create function, create pipeTablereadselectwriteinsert, update, delete, truncate, referencesTables and views are listed undertablesand handled properly behind the
scenes.If*is provided as the parameter for tables the grant statement will use theALL <object_type>s in SCHEMAsyntax. It will also grant to future tables and
views. See Snowflake documentation forON FUTUREIf a schema name includes an asterisk (prefix or suffix), such assnowplow_*or*_snowplow, then all schemas
that match this pattern will be included in the grant statementunless it is
for ownership, in which case the asterisk is not supported. This can be coupled
with the asterisk for table grants to grant permissions on all tables in all
schemas that match the given pattern. This is useful for date-partitioned
schemas.All entities must be explicitly referenced. For example, if a permission is
granted to a schema or table then the database must be explicitly referenced for
permissioning as well. Additionally, role membership must be explicit in the
config file. If a role does not have amember_oflist, it will have all roles
it currently has revoked.Roles can accept "_" as a role name either alone or nested under theincludekey. There is optionally anexcludekey that can be used ifincludeis used."_"will grant membership to all roles defined in the spec. Any roles defined
inexcludewill be removed from the list defined ininclude.Objects like warehouses and integrations that only have one permifrost permission type just
needs to be specified in the role (see below).Objects can have ametadictionary which may contain information that is not relevant for Permifrost's
execution and are ignored by Permifrost itself.A specification file has the following structure:# Databasesdatabases:-db_name:shared:boolean-db_name:shared:booleanowner:role_namemeta:some_key:some_value............# Rolesroles:-role_name:warehouses:-warehouse_name-warehouse_name...member_of:-role_name-role_name...# ormember_of:include:-"*"exclude:-role_nameprivileges:databases:read:-database_name-database_name...write:-database_name-database_name...schemas:read:-database_name.*-database_name.schema_name-database_name.schema_partial_*-database_name.*_schema_partial...write:-database_name.*-database_name.schema_name-database_name.schema_partial_*-database_name.*_schema_partial...tables:read:-database_name.*.*-database_name.schema_name.*-database_name.schema_partial_*.*-database_name.*_schema_partial.*-database_name.schema_name.table_name...write:-database_name.*.*-database_name.schema_name.*-database_name.schema_partial_*.*-database_name.*_schema_partial.*-database_name.schema_name.table_name...owns:databases:-database_name...schemas:-database_name.*-database_name.schema_name...tables:-database_name.*.*-database_name.schema_name.*-database_name.schema_name.table_name...meta:some_key:some_value...-role_name:owner:role_name.........# Users# can_login is required the rest of the parameters are optional# None of the values are validated in Snowflake, hence default_warehouse, default_namespace and default_role# can contain invalid valuesusers:-user_name:can_login:booleanmember_of:-role_name...has_password:booleandisplay_name:stringfirst_name:stringmiddle_name:stringlast_name:stringemail:stringcomment:stringdefault_warehouse:stringdefault_namespace:stringdefault_role:stringmeta:some_key:some_value...-user_name:owner:role_name.........# Warehouses# Warehouse sizes are informative and not altered by Permifrost to align with the spec filewarehouses:-warehouse_name:size:x-smallmeta:some_key:some_value...-warehouse_name:size:x-smallowner:role_name.........# Integrations# Integration categories are informative and not altered by Permifrost to align with the spec fileintegrations:-integration_name:category:storagemeta:some_key:some_value...-integration_name:category:securityowner:role_name.........For a working example, you can checkthe Snowflake specification
filethat we are using for testingpermifrost permissions.Note: The spec file must all be in lowercase.SettingsAll settings are declared here with their default values and are described
below. These can be added to your spec.yaml file.require-owner:falserequire-owner: Set to true to force having to set theownerproperty on all
objects defined.--diffWhen this flag is set, a full diff with both new and already granted commands is
returned. Otherwise, only required commands for matching the definitions on the
spec are returned.--dryWhen this flag is set, the permission queries generated are not actually sent to
the server and run; They are just returned to the user for examining them and
running them manually.When this flag is not set, the commands will be executed on Snowflake and their
status will be returned and shown on the command line.Connection ParametersThe following environmental variables must be available to connect to Snowflake:$PERMISSION_BOT_USER$PERMISSION_BOT_ACCOUNT$PERMISSION_BOT_WAREHOUSEUsername and PasswordTo connect using a username and password, also include the following:$PERMISSION_BOT_PASSWORD$PERMISSION_BOT_DATABASE$PERMISSION_BOT_ROLECurrently, Permifrost assumes you are using the SECURITYADMIN role and will fail
validation if you are not.OAuthTo connect using an OAuth token, also include the following:$PERMISSION_BOT_OAUTH_TOKENExternal Browser SSO AuthTo connect using an external browser with SSO Auth, also include the following:$PERMISSION_BOT_AUTHENTICATOR='externalbrowser'Key Pair AuthenticationRather than supplying a password or an oauth token, it's possible to connect via
Snowflake's Key Pair authentication by setting the following:$PERMISSION_BOT_KEY_PATH$PERMISSION_BOT_KEY_PASSPHRASESeeSnowflake-sqlalchemyfor more info.ContributingContributing to Permifrost is easy, and most commands to do so are available
within the Makefile.The easiest way to start developing is to runmake initial-setupto install
all the necessary packages to develop on the project. Next runmake permifrostin a second terminal, this will open a shell in a docker container
with the local version of Permifrost installed.You can now make changes to the files in your editor and it will be reflected in
the commands that you run from the docker shell.To check code quality prior to committing changes, you can usemake local-lint.See theMakefilefor more details.WARNINGSDO NOT name git branches with forward slashes/as the current CI pipeline is
unable to manage names like this. (i.e.username/feature/feature-namewill
break the CI pipeline sousername.feature.feature-nameshould be used
instead)This project haspre-commit
hooksinstalled to maintain
the existing code quality. As such, we strongly recommend you use a terminal tocommitandpushcode changes. Specifically, avoid using git
integrations on IDEs to makecommitsorpushes.Addingfiles
through the IDE git integrations are okay, but do notcommitthrough the
IDE. Use the terminal to commit changes because it will show the output of each
of the pre-commit checks to allow you to make changes as needed.For committing work-in-progress changes usegit commit --no-verify -m "WIP: <message>".For committing finalized changes, the below workflow will identify errors and allow for easier development:Make your changes andgit add <file name(s)>git committo identify/format errors in the changed filesRepeat the following steps until all checks passgit add <file name(s)>git commitAdd message at the prompt and save/exit the commit fileWhen you are ready to push changes to the remote host, rungit push origin <branch name>. This will perform additional linting/formatting checks.Repeat the following steps until all checks passgit push origin <branch name>git add <file name(s)>git commitAdd message at the prompt and save/exit the commit filegit push origin <branch name>until all checks passReleasingSee theissue templatefor guidance on how to release a new version of this project to PyPi