Package: rpe-lib
resource-policy-evaluation-libraryrpe-lib is made up ofPolicy Engines (rpe.engines.Engine),Resources (rpe.resources.Resource), andExtractors (rpe.resources.Extractor).Resourcesproduce details about the current state of a given type of resource. A resource can be just about anything, but the initial goal of the project was to support Google Cloud Platform (GCP) resources. The implemention is intentionally generic to allow support for other resource typesPolicy EnginesevaluateResourcesagainst their configured policies. Given a resource, they can return a list ofEvaluations (rpe.policy.Evaluation)indicating the name of each policy that applies to the resource (by resource type), and whether or not the resource is compliant.Resourcealso define aremediate()function, that should be able to take a dictionary description of how to manipulate a resource to make it compliant.Extractorsparse structured data and return a list of resources, and optionally metadata about the data source.As an example, for GCP resources, theremediate()function expects details about what method to call in the Google REST API for the given resource, and what parameters to pass. So for a Google Cloud Storage Bucket, a policy that enforces bucket versioning could define remediation as a call to the bucketspatch()method with the appropriate arguments to enable versioning.Policy EnginesThere are currently 2Policy Enginesincluded with rpe-lib:Open Policy Agent (opa) EngineThe Open Policy Agent engine uses the Open Policy Agent REST API to evalute policy for a given resource. In order to use this engine, you'll need to run the opa server with the rpe-lib base policies, and whatever policies you'd like to evaluate and optionally enforce. The opa engine passes the result ofResource.get()to the opa server as input. All policies need to be defined in therpe.policynamespace, include a few specific rules, and operate on theinputdocument:applies_to: a list of resource types the policy applies todescription: a human-readable description of the policycompliant: returns true if the resource defined in theinputdocument adheres to the policyexcluded: returns true if there is a reason to exclude the resource from evaluation, for GCP we define resource labels that can mark a resource as excludedremediate (optional): returns a JSON object explaining how to remediate the resource for the given policyExample policy:# This is a real policy included with rpe-lib though slightly simplified
package rpe.policy.storage_buckets_require_object_versioning

description = "Require object versioning for storage buckets"

applies_to = ["storage.googleapis.com/Bucket"]

default compliant = false

default excluded = false

compliant {
        input.resource.versioning.enabled = true
}

excluded {
        input.resource.labels["forseti-enforcer"] = "disabled"
}

remediate = {
        "_remediation_spec": "v2",
        "steps": [
            "method": "patch",
            "params": {
                    "bucket": input.resource.name,
                    "body": {"versioning": {"enabled": true}},
            }
        ]
}Python EngineThe OPA engine is very powerful, but is limited by what OPA is capable of. For use-cases that are more complicated, you might want to use the python engine. This engine expects the path to a python package that contains classes that perform the evaluation, and optionally remediation.As mentioned, the python policies are actually classes. Here is an example python policy class:# Stubbed out example of the above OPA engine policy as a python engine policy
class GCPBucketVersioningPolicy:
    description = 'Require object versioning for storage buckets'
    applies_to = ['cloudresourcemanager.googleapis.com/Project']

    @classmethod
    def compliant(cls, resource):
        return resource.get()['resource']['versioning']['enabled'] == True

    @classmethod
    def excluded(cls, resource):
        return resource.get()['resource']['labels']['forseti-enforcer'] == 'disabled'

    @classmethod
    def remediate(cls, resource):
        # execute cloudfuntion to enable versioning,
        # or shell execute gsutil,
        # or anything you can do with python
        passWhen you configure your RPE object, you provide the path to your python package. That package doesn't need to be installed, it will be loaded dynamically. The path you provide should contain an__init__.pyfile with every policy class you wish to use. These can be defined directly in that file, or imported. Since RPElib takes care of importing the package, you will be able to use relative imports in your python files. For example your directory structure might look like this:/etc/rpe/policies-python
* __init__.py
* policy1.py
* policy2.pyAnd your__init__.pymight look like this:# __init__.pyfrom.policy1importMyFirstPythonPolicyfrom.policy2importAnotherPythonPolicy,YetAnotherPolicyResourcesResourcesmust define the following functionsget(): returns metadata describing the current state of the resourceremediate(remediation_spec): applies the remediation (the spec is specific to the resource type/implementation)type(): returns the type of resource, used by policy engines to determine which policies apply to a given resourceExamplesUsing the OPA engineThis assumes you have theopabinary in your path# First, start opa with our policies
opa run --server ./policy/Now we need to create an RPE instance with the opa engine configured to use the local OPA server:fromrpeimportRPEfromrpe.resourcesimportGoogleAPIResourceconfig={'policy_engines':[{'type':'opa','url':'http://localhost:8181/v1/data'}]}rpe=RPE(config)# Create a resource object for the resource we want to evaluateres=GoogleAPIResource.from_cai_data('//storage.googleapis.com/my-test-bucket','storage.googleapis.com/Bucket',project_id='my-test-project',)evals=rpe.evaluate(res)foreinevals:print(f'Policy:{e.policy_id}, Compliant:{e.compliant}')ifnote.compliantande.remediable:e.remediate()Using the Python engineUsing the Python policy engine is similar to the above:fromrpeimportRPEfromrpe.resourcesimportGoogleAPIResourceconfig={'policy_engines':[{'type':'python','path':'/etc/rpe/policies-python'}]}rpe=RPE(config)# Create resource objects, and evaluate as neededAnd your policies may look like this:/etc/rpe/policies-python/init.pyfrom.gcs_policyimportGCPBucketVersioningPolicy/etc/rpe/policies-python/gcs_policy.pyclassGCPBucketVersioningPolicy:description='Require object versioning for storage buckets'applies_to=['cloudresourcemanager.googleapis.com/Project']@classmethoddefcompliant(cls,resource):# Return true/false if the resource is compliant@classmethoddefexcluded(cls,resource):# Return true/false if the resource is excluded@classmethoddefremediate(cls,resource):# Enable versioningApplications using rpe-libForseti Real-time Enforcer- The Forseti Real-time enforcer uses rpe-lib for the evaluation and enforcement of policy for Google Cloud resources. It uses a Stackdriver log export to a Pub/Sub topic to trigger enforcement.