Repository: crossplane/crossplane

Issue:
Only watch, don't poll, composite resources
<!--
Thank you for helping to improve Crossplane!

Please be sure to search for open issues before raising a new one. We use issues
for bug reports and feature requests. Please find us at https://slack.crossplane.io
for questions, support, and discussion.
-->

### What problem are you facing?
<!--
Please tell us a little about your use case - it's okay if it's hypothetical!
Leading with this context helps frame the feature request so we can ensure we
implement it sensibly.
--->

The XR controller currently watches the XR type, but is also poll-triggered, by default polling desired state every 60 seconds. This interval can be changed by the `--poll-interval` flag. The XR reconciler is poll-triggered because it wants to know when composed resources change, in order to correct drift.

An XR controller doesn't know what kinds of resources it will compose at start time, which is typically when a controller's watches are configured. Furthermore, two different XRs of the same kind might compose completely different types of resources due to using different Compositions.

### How could Crossplane help solve your problem?
<!--
Let us know how you think Crossplane could help with your use case. 
-->

Ideally XR reconciliation would be purely watch-triggered. This would result in fewer invocations of the XR reconciler and thus presumably the ability for a particular Crossplane deployment to handle more XRs concurrently. This is also particularly important for Composition Functions, where every XR reconcile may call one or more Functions.

Hints from issue comments:
@negz: Thinking out loud, it  may be possible to:

1. Make one controller responsible (only) for selecting a Composition for XRs that don't have one.
2. Start another controller for every unique (XR, Composition) tuple.
3. Restart the (XR, Composition) controller whenever the XR's `spec.resourceRefs` changes

The idea being that each time the (XR, Composition) controller was restarted it would take a watch on all of the (types of?) composed resources that appeared in its `spec.resourceRefs` array.@negz: FWIW while this relates to Functions I'm not ready to say this is a blocker for Functions going to beta or even GA at this point. I'd like to see some indication that it's a real constraint before we spend time optimizing.@bobh66: @negz How would this account for the deletion of a composed resource?  I don't think that deletion of a composed resource would trigger an event on the owning composite?  The poll interval will catch this scenario and recreate the composed resource.@negz: > I don't think that deletion of a composed resource would trigger an event on the owning composite?

I believe it would as long as the controller of the owning composite was watching whatever type of resource was deleted. e.g. If we can notice that the XR has composed a `Subnet` and restart its controller to start watching for `Subnet` composed resources, the XR would have a reconcile queued any time any `Subnet` was deleted.@github-actions: Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as `stale` because it has had no activity in the last 90 days. It will be closed in 14 days if no further activity occurs. Leaving a comment **starting with** `/fresh` will mark this issue as not stale.@negz: /fresh

@sttts is actively working on this in #4637@negz: @sttts PS do you want to add this to the v1.14 milestone? I know its behind a feature flag so it would land as alpha.@sttts: yes, 1.14 is the plan. @negz: I think we can call this fixed by https://github.com/crossplane/crossplane/pull/4637. In practice we still watch _and_ poll, but I could imagine figuring out removing polling being part of the work to get the 'realtime Compositions' feature to beta in https://github.com/crossplane/crossplane/issues/4828.@negz: Reopening as I'd like to do this for reals as part of #4828.@negz: https://github.com/crossplane/crossplane/blob/b0cc38c104eb4f776a9ec3b7a131e457ea529372/internal/controller/apiextensions/composite/watch/watch.go#L148-L151

We'll need to figure this out before we remove polling.

Watch garbage collection is racy. Today it's possible we'll garbage collect a composed resource watch [^1] if the watch is needed, then not needed, then quickly needed again (e.g. because at least one XR composes the type, then no XRs do, then at least one XR does).

We could make this _less_ racy by falling back to an uncached client to double-check before we stop a watch. This way we'll only list XRs if we think we should stop a watch, not every interval.

There'd still be a race, but it'd be over a smaller window of time, e.g.:

1. We list current watches, and `kind: Foo` is one of them
2. We list XRs, nothing composes a `kind: Foo` anymore
3. A new XR starts composing `kind: Foo` <-- Race
4. We stop the watch on `kind: Foo`

Perhaps we could record when watches start, and refuse to GC watches that started very recently.

[^1]: Actually, any kind of watch until https://github.com/crossplane/crossplane/pull/6395 is merged@negz: Ah I just remembered this:

> I know the main constraint will be figuring out how/when to invoke the Function pipeline, assuming folks might want it to be invoked even when the XR/composed resources haven't changed.

Perhaps we expose XR poll interval as an XRD option, with the default being no polling?@negz: > Perhaps we could record when watches start, and refuse to GC watches that started very recently.

Actually, I'm inclined to skip this for now.

If we do stop a watch prematurely we'll start it again the next time any XR that composes the watched type is reconciled. Said XR would be reconciled if

* The XR itself was updated
* Any other resource the XR composes was updated
* Crossplane's `--sync-interval` was exceeded

The default sync interval is 1 hour, so worst case we'd go up to an hour not noticing that an XR needs reconciling because it's not watching one of its composed resource types. I feel comfortable with that. Especially since by double checking using an uncached client that no XR composes a type before we stop its watch, the likelihood that we ever stop a watch that we shouldn't becomes pretty small.@negz: 🤔 Actually, I also don't see why we couldn't have the inverse of our watch garbage collector running:

* Every $interval, list all XRs from cache
* Build the set of types composed by any XR
* Call `StartWatches` for those types

This should be pretty cheap - we'd list from cache, and `StartWatches` is a no-op if there's no watch to start. We'd only take the controller's read lock in the `ControllerEngine` for a very brief period.

That all said, I still wouldn't worry about building this proactively. It's an option we can keep around in case accidentally stopping watches becomes a problem (and we can't tolerate waiting for up to the sync interval to self heal).

Solution patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 9c6ba4d7b..e1ef44779 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -138,7 +138,7 @@ jobs:
 
       - name: Find the Go Build Cache
         id: go
-        run: echo "::set-output name=cache::$(make go.cachedir)"
+        run: echo "cache=$(make go.cachedir)" >> $GITHUB_OUTPUT
 
       - name: Cache the Go Build Cache
         uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3
@@ -158,12 +158,12 @@ jobs:
         run: make vendor vendor.check
 
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@46ed16ded91731b2df79a2893d3aea8e9f03b5c4 # v2
+        uses: github/codeql-action/init@489225d82a57396c6f426a40e66d461b16b3461d # v2
         with:
           languages: go
 
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@46ed16ded91731b2df79a2893d3aea8e9f03b5c4 # v2
+        uses: github/codeql-action/analyze@489225d82a57396c6f426a40e66d461b16b3461d # v2
 
   trivy-scan-fs:
     runs-on: ubuntu-22.04
@@ -206,7 +206,7 @@ jobs:
 
       - name: Find the Go Build Cache
         id: go
-        run: echo "::set-output name=cache::$(make go.cachedir)"
+        run: echo "cache=$(make go.cachedir)" >> $GITHUB_OUTPUT
 
       - name: Cache the Go Build Cache
         uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3
@@ -269,7 +269,7 @@ jobs:
 
       - name: Find the Go Build Cache
         id: go
-        run: echo "::set-output name=cache::$(make go.cachedir)"
+        run: echo "cache=$(make go.cachedir)" >> $GITHUB_OUTPUT
 
       - name: Cache the Go Build Cache
         uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3
@@ -331,7 +331,7 @@ jobs:
 
       - name: Find the Go Build Cache
         id: go
-        run: echo "::set-output name=cache::$(make go.cachedir)"
+        run: echo "cache=$(make go.cachedir)" >> $GITHUB_OUTPUT
 
       - name: Cache the Go Build Cache
         uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3
diff --git a/.github/workflows/scan.yaml b/.github/workflows/scan.yaml
index 26b6e2fbb..788497e57 100644
--- a/.github/workflows/scan.yaml
+++ b/.github/workflows/scan.yaml
@@ -131,7 +131,7 @@ jobs:
           retention-days: 3
 
       - name: Upload Trivy Scan Results To GitHub Security Tab
-        uses: github/codeql-action/upload-sarif@46ed16ded91731b2df79a2893d3aea8e9f03b5c4 # v2
+        uses: github/codeql-action/upload-sarif@489225d82a57396c6f426a40e66d461b16b3461d # v2
         with:
           sarif_file: 'trivy-results.sarif'
           category: ${{ matrix.image }}:${{ env.tag }}
diff --git a/cluster/images/xfn/Dockerfile b/cluster/images/xfn/Dockerfile
index 47325069d..16dfce7ac 100644
--- a/cluster/images/xfn/Dockerfile
+++ b/cluster/images/xfn/Dockerfile
@@ -1,5 +1,5 @@
 # This is debian:bookworm-slim (i.e. Debian 12, testing)
-FROM debian:bookworm-slim@sha256:d8f9d38c21495b04d1cca99805fbb383856e19794265684019bf193c3b7d67f9
+FROM debian:bookworm-slim@sha256:9bd077d2f77c754f4f7f5ee9e6ded9ff1dff92c6dce877754da21b917c122c77
 
 ARG TARGETOS
 ARG TARGETARCH
diff --git a/design/assets/design-doc-composition-functions/containerized-functions.png b/design/assets/design-doc-composition-functions/containerized-functions.png
new file mode 100644
index 000000000..819b348f6
Binary files /dev/null and b/design/assets/design-doc-composition-functions/containerized-functions.png differ
diff --git a/design/assets/design-doc-composition-functions/functions.png b/design/assets/design-doc-composition-functions/functions.png
new file mode 100644
index 000000000..c4f1bc4aa
Binary files /dev/null and b/design/assets/design-doc-composition-functions/functions.png differ
diff --git a/design/defunct/design-doc-composition-functions.md b/design/defunct/design-doc-composition-functions.md
new file mode 100644
index 000000000..1ca91d9b7
--- /dev/null
+++ b/design/defunct/design-doc-composition-functions.md
@@ -0,0 +1,778 @@
+ # Composition Functions
+
+* Owners: Nic Cope (@negz), Sergen Yalçın (@sergenyalcin)
+* Reviewers: Crossplane Maintainers
+* Status: Defunct
+
+## Background
+
+Crossplane is a framework for building cloud native control planes. These
+control planes sit one level above the cloud providers and allow you to
+customize the APIs they expose. Platform teams use Crossplane to offer the
+developers they support simpler, safer, self-service interfaces to the cloud.
+
+To build a control plane with Crossplane you:
+
+1. Define the APIs you’d like your control plane to expose.
+1. Extend Crossplane with support for orchestrating external resources (e.g.
+   AWS).
+1. Configure which external resources to orchestrate when someone calls your
+   APIs.
+
+Crossplane offers a handful of extension points that layer atop each other to
+help make this possible:
+
+* Providers extend Crossplane with Managed Resources (MRs), which are high
+  fidelity, declarative representations of external APIs. Crossplane reconciles
+  MRs by orchestrating an external system (e.g. AWS).
+* Configurations extend Crossplane with Composite Resources (XRs), which are
+  essentially arbitrary APIs, and Compositions. Crossplane reconciles XRs by
+  orchestrating MRs. Compositions teach Crossplane how to do this.
+
+The functionality enabled by XRs and Compositions is typically referred to as
+simply Composition. Support for Composition was added in Crossplane [v0.10.0]
+(April 2020). From our [terminology documentation][term-composition]:
+
+> Folks accustomed to Terraform might think of a Composition as a Terraform
+> module; the HCL code that describes how to take input variables and use them
+> to create resources in some cloud API. Folks accustomed to Helm might think of
+> a Composition as a Helm chart’s templates; the moustache templated YAML files
+> that describe how to take Helm chart values and render Kubernetes resources.
+
+A Crossplane `Composition` consists of an array of one or more 'base'
+resources. Each of these resources can be 'patched' with values derived from the
+XR. The functionality enabled by a `Composition` is intentionally limited - for
+example there is no support for conditionals (e.g. only create this resource if
+the following conditions are met) or iteration (e.g. create N of the following
+resource, where N is derived from an XR field).
+
+Below is an example `Composition`:
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1
+kind: Composition
+metadata:
+  name: example
+spec:
+  compositeTypeRef:
+    apiVersion: database.example.org/v1alpha1
+    kind: AcmeCoDatabase
+  resources:
+  - name: cloudsqlinstance
+    base:
+      apiVersion: database.gcp.crossplane.io/v1beta1
+      kind: CloudSQLInstance
+      spec:
+        forProvider:
+          databaseVersion: POSTGRES_9_6
+          region: us-central1
+          settings:
+            tier: db-custom-1-3840
+            dataDiskType: PD_SSD
+    patches:
+    - type: FromCompositeFieldPath
+      fromFieldPath: spec.parameters.storageGB
+      toFieldPath: spec.forProvider.settings.dataDiskSizeGb
+```
+
+The goals of the Crossplane maintainers in designing Composition were to:
+
+* Empower platform teams to provide a platform of useful, opinionated
+  abstractions.
+* Enable platform teams to define abstractions that may be portable across
+  different infrastructure providers and application runtimes.
+* Enable platform teams to share and reuse the abstractions they define.
+* Leverage the Kubernetes Resource Model (KRM) to model applications,
+  infrastructure, and the product of the two in a predictable, safe, and
+  declarative fashion using low or no code.
+* Avoid imposing unnecessary opinions, assumptions, or constraints around how
+  applications and infrastructure should function.
+
+The design document for Composition [captures these goals][composition-design]
+using somewhat dated parlance.
+
+Our approach to achieving our goals was heavily informed by Brian Grant’s
+[Declarative application management in Kubernetes][declarative-app-management].
+Brian’s document is an excellent summary of the gotchas and pitfalls faced by
+those attempting to design new configuration management tools, informed by his
+experiences designing Kubernetes, its precursor Borg, and many generations of
+configuration languages at Google, including [BCL/GCL][bcl]. In particular, we
+wanted to:
+
+* Avoid organically growing a new DSL. These languages tend to devolve to
+  incoherency as stakeholders push to “bolt on” new functionality to solve
+  pressing problems at the expense of measured design. Terraform’s DSL
+  unintuitively [supporting the count argument in some places but not
+  others][terraform-count] is a great example of this. Inventing a new DSL also
+  comes with the cost of inventing new tooling to test, lint, generate, etc,
+  your DSL.
+* Stick to configuration that could be modeled as a REST API. Modeling
+  Composition logic as a schemafied API resource makes it possible for
+  Crossplane to validate that logic and provide feedback to the platform team at
+  configuration time. It also greatly increases interoperability thanks to broad
+  support across tools and languages for interacting with REST APIs.
+
+It was also important to avoid the “worst of both worlds” - i.e. growing a fully
+featured ([Turing-complete][turing-complete]) DSL modeled as a REST API. To this
+end we omitted common language features such as conditionals and iteration. Our
+rationale being that these features were better deferred to a General Purpose
+Programming Language (GPL) designed by language experts, and with extensive
+existing tooling.
+
+Since its inception the Crossplane maintainers’ vision has been that there
+should essentially be two variants of Composition:
+
+* For simple cases, use contemporary "Patch and Transform" (P&T) Composition.
+* For advanced cases, bring your tool or programming language of choice.
+
+In this context a “simple case” might involve composing fewer than ten resources
+without the need for logic such as conditionals and iteration. Note that the
+Composition logic, whether P&T or deferred to a tool or programming language, is
+always behind the API line (behind an XR). This means the distinction is only
+important to the people authoring the Compositions, never to the people
+consuming them.
+
+Offering two variants of Composition allows Crossplane users to pick the one
+that is best aligned with their situation, preferences, and experience level.
+For simple cases you don’t need to learn a new programming language or tool, and
+there are no external dependencies - just write familiar, Kubernetes-style YAML.
+For advanced cases leverage proven tools and languages with existing ecosystems
+and documentation. Either way, Crossplane has no religion - if you prefer not to
+“write YAML”, pick another tool and vice versa.
+
+## Goals
+
+The proposal put forward by this document should:
+
+* Let folks use their composition tool and/or programming language of choice.
+* Support 'advanced' composition logic such as loops and conditionals.
+* Balance safety (e.g. sandboxing) with speed and simplicity.
+* Be possible to introduce behind a feature flag that is off by default.
+
+While not an explicit goal, it would also be ideal if the solution put forth by
+this document could serve as a test bed for new features in the contemporary
+'resources array' based form of Composition.
+
+The user experience around authoring and maintaining Composition Functions is
+out of scope for this proposal, which focuses only on adding foundational
+support for the feature to Crossplane. 
+
+## Proposal
+
+### Overview
+
+This document proposes that a new `functions` array be added to the existing
+`Composition` type. This array of functions would be called either instead of or
+in addition to the existing `resources` array in order to determine how an XR
+should be composed. The array of functions acts as a pipeline; the output of
+each function is passed as the input to the next. The output of the final
+function tells Crossplane what must be done to reconcile the XR.
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v2alpha1
+kind: Composition
+metadata:
+  name: example
+spec:
+  compositeTypeRef:
+    apiVersion: database.example.org/v1alpha1
+    kind: XPostgreSQLInstance
+  functions:
+  - name: my-cool-function
+    type: Container
+    container:
+      image: xkpg.io/my-cool-function:0.1.0
+```
+
+Under this proposal each function is the entrypoint of an OCI image, though the
+API is designed to support different function implementations (such as webhooks)
+in the future. The updated API would affect only the `Composition` type - no
+changes would be required to the schema of `CompositeResourceDefinitions`, XRs,
+etc.
+
+Notably the functions would not be responsible for interacting with the API
+server to create, update, or delete composed resources. Instead, they instruct
+Crossplane which resources should be created, updated, or deleted.
+
+Under the proposed design functions could also be used for purposes besides
+rendering composed resources, for example validating the results of the
+`resources` array or earlier functions in the `functions` array. Furthermore, a
+function could also be used to implement 'side effects' such as triggering a
+replication or backup.
+
+Below is a more detailed example of an entry in the `functions` array.
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v2alpha1
+kind: Composition
+metadata:
+  name: example
+spec:
+  compositeTypeRef:
+    apiVersion: database.example.org/v1alpha1
+    kind: XPostgreSQLInstance
+  functions:
+  - name: my-cool-function
+    type: Container
+    # Configuration specific to `type: Container` functions.
+    container:
+      # The OCI image to pull and run.
+      image: xkpg.io/my-cool-function:0.1.0
+      # Whether to pull the function Never, Always, or IfNotPresent.
+      imagePullPolicy: IfNotPresent
+      # Secrets used to pull from a private registry.
+      imagePullSecrets:
+      - namespace: crossplane-system
+        name: my-xpkg-io-creds
+      # Note that only resource limits are supported - not requests.
+      # The function will be run with the specified resource limits.
+      resources:
+        limits:
+          memory: 64Mi
+          cpu: 250m
+      # Defaults to 'Isolated' - i.e an isolated network namespace.
+      network: Accessible
+      # How long the function may run before it's killed. Defaults to 10s.
+      timeout: 30s
+      # Containers are run by an external process listening at the supplied
+      # endpoint. Specifying an endpoint is optional; the endpoint defaults to
+      # the below value.
+      runner:
+        endpoint: unix:///@crossplane/fn/default.sock
+    # An x-kubernetes-embedded-resource RawExtension (i.e. an unschemafied
+    # Kubernetes resource). Passed to the function as the config block of its
+    # FunctionIO.
+    config:
+      apiVersion: database.example.org/v1alpha1
+      kind: Config
+      metadata:
+        name: cloudsql
+      spec:
+        version: POSTGRES_9_6
+```
+
+### Function API
+
+This document proposes that each function uses a `FunctionIO` type as its input
+and output. In the case of `Container` functions this would correspond to stdin
+and stdout. Crossplane would be responsible for reading stdout from the final
+function and applying its changes to the relevant XR and composed resources.
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1alpha1
+kind: FunctionIO
+config:
+  apiVersion: database.example.org/v1alpha1
+  kind: Config
+  metadata:
+    name: cloudsql
+  spec:
+    version: POSTGRES_9_6
+observed:
+  composite:
+    resource:
+      apiVersion: database.example.org/v1alpha1
+      kind: XPostgreSQLInstance
+      metadata:
+        name: my-db
+      spec:
+        parameters:
+          storageGB: 20
+        compositionSelector:
+          matchLabels:
+            provider: gcp
+      status:
+        conditions:
+        - type: Ready
+          status: True
+    connectionDetails:
+    - name: uri
+      value: postgresql://db.example.org:5432
+```
+
+A `FunctionIO` resource consists of the following top-level fields:
+
+* The `apiVersion` and `kind` (required).
+* A `config` object (optional). This is a [Kubernetes resource][rawextension]
+  with an arbitrary schema that may be used to provide additional configuration
+  to a function. For example a `render-helm-chart` function might use its
+  `config` to specify which Helm chart to render. Functions need not return
+  their `config`, and any mutations will be ignored.
+* An `observed` object (required). This reflects the observed state of the XR,
+  any existing composed resources, and their connection details. Functions must
+  return the `observed` object unmodified.
+* A `desired` object (optional). This reflects the accumulated desired state of
+  the XR and any composed resources. Functions may mutate the `desired` object.
+* A `results` array (optional). Used to communicate information about the result
+  of a function, including warnings and errors. Functions may mutate the
+  `results` object.
+
+Each function takes its `config` (if any), `observed` state, and any previously
+accumulated `desired` state as input, and optionally mutates the `desired`
+state. This allows the output of one function to be the input to the next.
+
+The `observed` object consists of:
+
+* `observed.composite.resource`. The observed XR.
+* `observed.composite.connectionDetails`: The observed XR connection details.
+* `observed.resources[N].name`: The name of an observed composed resource.
+* `observed.resources[N].resource`: An observed composed resource.
+* `observed.resources[N].connectionDetails`: An observed composed resource's
+  current connection details.
+
+If an observed composed resource appears in the Composition's `spec.resources`
+array their `name` fields will match. Note that the `name` field is distinct
+from a composed resource's `metadata.name` - it is used to identify the resource
+within a Composition and/or its function pipeline.
+
+The `desired` object consists of:
+
+* `desired.composite.resource`. The desired XR.
+* `desired.composite.resource.connectionDetails`. Desired XR connection details.
+* `desired.resources[N].name`. The name of a desired composed resource.
+* `desired.resources[N].resource`. A desired composed resource.
+* `desired.resources[N].connectionDetails`. A desired composed resource's
+  connection details.
+* `desired.resources[N].readinessChecks`. A desired composed resource's
+  readiness checks.
+
+Note that the `desired.resources` array of the `FunctionIO` type is very
+similar to the `spec.resources` array of the `Composition` type. In comparison:
+
+* `name` works the same across both types, but is required by `FunctionIO`.
+* `connectionDetails` and `readinessChecks` work the same across both types.
+* `FunctionIO` does not support `base` and `patches`. Instead, a function should
+  configure the `resource` field accordingly.
+
+The `desired` state is _accumulated_ across the Composition and all of its
+functions. This means the first function may be passed desired state as
+specified by the `spec.resources` array of a Composite, if any, and each
+function must include the accumulated desired state in its output. Desired state
+is treated as an overlay on observed state, so a function pipeline need not
+specify the desired state of the XR (for example) unless a function wishes to
+mutate it.
+
+A full `FunctionIO` specification will accompany the implementation. Some
+example scenarios are illustrated below.
+
+A function that wanted to create (compose) a `CloudSQLInstance` would do so by
+returning the following `FunctionIO`:
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1alpha1
+kind: FunctionIO
+observed: {}  # Omitted for brevity.
+desired:
+  resources:
+  - name: cloudsqlinstance
+    resource:
+      apiVersion: database.gcp.crossplane.io/v1beta1
+      kind: CloudSQLInstance
+      spec:
+        forProvider:
+          databaseVersion: POSTGRES_9_6
+          region: us-central1
+          settings:
+            tier: db-custom-1-3840
+            dataDiskType: PD_SSD
+            dataDiskSizeGb: 20
+        writeConnectionSecretToRef:
+          namespace: crossplane-system
+          name: cloudsqlpostgresql-conn
+    connectionDetails:
+    - name: hostname
+      fromConnectionSecretKey: hostname
+    readinessChecks:
+    - type: None
+```
+
+A function that wanted to set only an XR connection detail could return:
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1alpha1
+kind: FunctionIO
+observed: {}  # Omitted for brevity.
+desired:
+  composite:
+    connectionDetails:
+    - type: FromValue
+      name: username
+      value: admin
+```
+
+A function wishing to delete a composed resource may do so by setting its
+`resource` to null, for example:
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1alpha1
+kind: FunctionIO
+observed: {}  # Omitted for brevity.
+desired:
+  resources:
+  - name: cloudsqlinstance
+    resource: null
+```
+
+A function that could not complete successfully could do so by returning the
+following `FunctionIO`:
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1alpha1
+kind: FunctionIO
+config:
+  apiVersion: database.example.org/v1alpha1
+  kind: Config
+  metadata:
+    name: cloudsql
+  spec:
+    version: POSTGRES_9_6
+observed: {}  # Omitted for brevity.
+results:
+- severity: Error
+  message: "Could not render Database.postgresql.crossplane.io/v1beta1`
+```
+
+### Running Container Function Pipelines
+
+While Crossplane typically runs in a Kubernetes cluster - a cluster designed to
+run containers - running an ordered _pipeline_ of short-lived containers via
+Kubernetes is much less straightforward than you might expect. Refer to
+[Alternatives Considered](#alternatives-considered) for details.
+
+In order to provide flexibility and choice of tradeoffs in running containers
+(e.g. speed, scalability, security) this document proposes Crossplane defer
+containerized functions to an external runner. Communication with the runner
+would occur via a gRPC API, with the runner expected to be listening at the
+`endpoint` specified via the function's `runner` configuration block. This
+endpoint would default to `unix:///@crossplane/fn/default.sock` - an abstract
+[Unix domain socket][unix-domain-sockets].
+
+Communication between Crossplane and a containerized function runner would use
+the following API:
+
+```protobuf
+syntax = "proto3";
+
+// This service defines the APIs for a containerized function runner.
+service ContainerizedFunctionRunner {
+    rpc RunFunction(RunFunctionRequest) returns (RunFunctionResponse) {}
+}
+
+// Corresponds to Kubernetes' image pull policy.
+enum ImagePullPolicy {
+  IF_NOT_PRESENT = 0;
+  ALWAYS = 1;
+  NEVER = 2;
+}
+
+// Corresponds to go-containerregistry's AuthConfig type.
+// https://pkg.go.dev/github.com/google/go-containerregistry@v0.11.0/pkg/authn#AuthConfig
+message ImagePullAuth {
+  string username = 1;
+  string password = 2;
+  string auth = 3;
+  string identity_token = 4;
+  string registry_token = 5;
+}
+
+message ImagePullConfig {
+  ImagePullPolicy pull_policy = 1;
+  ImagePullAuth auth = 2;
+}
+
+// Containers are run without network access (in an isolated network namespace)
+// by default.
+enum NetworkPolicy = {
+  ISOLATED = 0;
+  ACCESSIBLE = 1;
+}
+
+// Only resource limits are supported. Resource requests could be added in
+// future if a runner supported them (e.g. by running containers on Kubernetes).
+message Resources {
+  ResourceLimits limits = 1;
+}
+
+message ResourceLimits {
+  string memory = 1;
+  string cpu = 2;
+}
+
+message RunFunctionConfig {
+  Resources resources = 1;
+  NetworkPolicy network = 2;
+  Duration timeout = 3;
+}
+
+// The input FunctionIO is supplied as opaque bytes.
+message RunFunctionRequest {
+  string image = 1;
+  bytes input = 2;
+  ImagePullConfig = 3;
+  RunFunctionConfig = 4;
+}
+
+// The output FunctionIO is supplied as opaque bytes. Errors encountered while
+// running a function (as opposed to errors returned _by_ a function) will be
+// encapsulated as gRPC errors.
+message RunFunctionResponse {
+  bytes output = 1;
+}
+```
+
+### The Default Function Runner
+
+This document proposes that Crossplane include a default function runner. This
+runner would be implemented as a sidecar to the core Crossplane container that
+runs functions inside itself.
+
+The primary advantages of this approach are speed and control. There's no need
+to wait for another system (for example the Kubernetes control plane) to
+schedule each container, and the runner can easily pass stdout from one
+container to another's stdin. Speed of function runs is of particular importance
+given that each XR typically reconciles (i.e. invokes its function pipeline)
+once every 60 seconds.
+
+The disadvantages of running the pipeline inside a sidecar container are scale
+and reinvention of the wheel. The resources available to the sidecar container
+will bound how many functions it can run at any one time, and it will need to
+handle features that the Kubelet already offers such as pull secrets, caching
+etc.
+
+[Rootless containers][rootless] appear to be the most promising way to run
+functions as containers inside a container:
+
+> Rootless containers uses `user_namespaces(7)` (UserNS) for emulating fake
+> privileges that are enough to create containers. The pseudo-root user gains
+> capabilities such as `CAP_SYS_ADMIN` and `CAP_NET_ADMIN` inside UserNS to
+> perform fake-privileged operations such as creating mount namespaces, network
+> namespaces, and creating TAP devices.
+
+Using user namespaces allows the runner to use the other kinds of namespaces
+listed above to ensure an extra layer of isolation for the functions it runs.
+For example a network namespace could be configured to prevent a function having
+network access.
+
+User namespaces are well supported by modern Linux Kernels, having been
+introduced in Linux 3.8. Many OCI runtimes (including `runc`, `crun`, and
+`runsc`) support rootless mode. `crun` appears to be the most promising choice
+because:
+
+* It is more self-contained than `runc` (the reference and most commonly used
+  OCI runtime), which relies on setuid binaries to setup user namespaces.
+* `runsc` (aka gVisor) uses extra defense in depth features which are not
+  allowed inside most containers due to their seccomp policies.
+
+Of course, "a container" is in fact many technologies working together and some
+parts of rootless containers are less well supported than others; for example
+cgroups v2 is required in order to limit resources like CPU and memory available
+to a particular function. cgroups v2 has been available in Linux since 4.15, but
+was not enabled by many distributions until 2021. In practice this means
+Crossplane users must use a [sufficiently modern][cgroups-v2-distros]
+distribution on their Kubernetes nodes in order to constrain the resources of a
+Composition function.
+
+Similarly, [overlayfs] was not allowed inside user namespaces until Linux 5.11.
+Overlayfs is typically used to create a root filesystem for a container that is
+backed by a read-write 'upper' directory overlaid on a read-only 'lower'
+directory. This allows the root OCI image filesystem to persist as a cache of
+sorts, while changes made during the lifetime of a container can be easily
+discarded. It's possible to replicate these benefits (at the expense of disk
+usage and start-up time) by falling back to making a throwaway copy of the root
+filesystem for each container run where overlayfs is not available.
+
+Under the approach proposed by this document each function run would involve the
+following steps:
+
+1. Use [go-containerregistry] to pull the function's OCI image.
+1. Extract (untar) the OCI image's flattened filesystem to disk.
+1. Create a filesystem for the container - either an overlay or a copy of the
+   filesystem extracted in step 2.
+1. Derive an [OCI runtime configuration][oci-rt-cfg] from the
+   [OCI image configuration][oci-img-cfg] supplied by go-containerregistry.
+1. Execute `crun run` to invoke the function in a rootless container.
+
+Executing `crun` directly as opposed to using a higher level tool like `docker`
+or `podman` allows the default function runner to avoid new dependencies apart
+from a single static binary (i.e. `crun`). It keeps most functionality (pulling
+images etc) inside the runner's codebase, delegating only container creation to
+an external tool. Composition Functions are always short-lived and should always
+have their stdin and stdout attached to the runner, so wrappers like
+`containerd-shim` or `conmon` should not be required. The short-lived, "one
+shot" nature of Composition Functions means it should also be acceptable to
+`crun run` the container rather than using `crun create`, `crun start`, etc.
+
+At the time of writing rootless containers appear to be supported by Kubernetes,
+including Amazon's Elastic Kubernetes Service (EKS) and Google Kubernetes Engine
+(GKE).
+
+Testing using GKE 1.21.10-gke.2000 with Container Optimized OS (with containerd)
+cos-89-16108-604-19 nodes (Kernel COS-5.4.170) found that it was possible to run
+`unshare -rUm` (i.e. to create a new user and mount namespace) inside an Alpine
+Linux container as long as AppArmor was disabled by applying the annotation
+`container.apparmor.security.beta.kubernetes.io/${CONTAINER_NAME}=unconfined`.
+It's possible to create user namespaces with AppArmor enabled, but not to create
+mount namespaces with different mount propagation from their parent.
+
+It is not possible to use rootless containers with gVisor enabled, as gVisor
+does not yet [support mount namespaces][gvisor-mountns]. This means that it is
+not possible to use rootless containers with GKE Autopilot, which requires that
+gVisor be used.
+
+Testing using EKS v1.21.5-eks-9017834 with Amazon Linux 2 nodes (Kernel
+5.4.188-104.359.amzn2.x86_64) found that it was possible to run `unshare -rUm`
+inside an Alpine Linux container 'out of the box'.
+
+The `unshare` syscall used to create containers is rejected by the default
+Docker and containerd seccomp profiles. seccomp is disabled ("Unconstrained") by
+default in Kubernetes, but that will soon change per [this KEP][kep-seccomp]
+which proposes that Kubernetes use the seccomp profiles of its container engine
+(i.e. containerd) by default. Once this happens Crossplane will either need to
+run with the "Unconstrained" seccomp profile, or a variant of the default
+containerd seccomp profile that allows a few extra syscalls (i.e. at least
+`unshare` and `mount`). This can be done by setting a Pod's
+`spec.securityContext.seccompProfile.type` field to `Unconstrained`.
+
+### Packaging Containerized Functions
+
+This document proposes that containerized functions support Crossplane [package
+metadata][package-meta] in the form of a `package.yaml` file at the root of the
+flattened filesystem and/or the OCI layer annotated as `io.crossplane.xpkg:
+base` per the [xpkg spec][xpkg-spec]. This `package.yaml` file would contain a
+custom-resource-like YAML document of type `Function.meta.pkg.crossplane.io`.
+
+Unlike `Configuration` and `Provider` packages, `Function` packages would not
+actually be processed by the Crossplane package manager but rather by the
+Composition (`apiextensions`) machinery. In practice Crossplane would be
+ignorant of the `package.yaml` file; it would exist purely as a way to attach
+"package-like" metadata to containerized Crossplane functions. Therefore, unlike
+the existing package types the `package.yaml` would contain no `spec` section.
+
+An example `package.yaml` might look like:
+
+```yaml
+# Required. Must be as below.
+apiVersion: meta.pkg.crossplane.io/v1alpha1
+# Required. Must be as below.
+kind: Function
+# Required.
+metadata:
+  # Required. Must comply with Kubernetes API conventions.
+  name: function-example
+  # Optional. Must comply with Kubernetes API conventions.
+  annotations:
+    meta.crossplane.io/source: https://github.com/negz/example-fn
+    meta.crossplane.io/description: An example function
+```
+
+## Alternatives Considered
+
+Most of the alternatives considered in this design could also be thought of as
+future considerations. In most cases these alternatives don't make sense at the
+present time but likely will in the future.
+
+### Using Webhooks to Run Functions
+
+Crossplane could invoke functions by calling a webhook rather than running an
+OCI container. In this model function input and output would still take the form
+of a `FunctionIO`, but would be HTTP request and response bodies rather than a
+container's stdin and stdout.
+
+The primary detractor of this approach is the burden it puts on function authors
+and Crossplane operators. Rather than simply publishing an OCI image the author
+and/or Crossplane operator must deploy and operate a web server, ensuring secure
+communication between Crossplane and the webhook endpoint.
+
+Support for `type: Webhook` functions will likely be added shortly after initial
+support for `type: Container` functions is released.
+
+### Using chroots to Run Functions
+
+Crossplane could invoke functions packaged as OCI images by unarchiving them and
+then running them inside a simple `chroot`. This offers more compatibility than
+rootless containers at the expense of isolation - it's not possible to constrain
+a chrooted function's compute resources, network access, etc. `type: Chroot`
+functions would use the same artifacts as `type: Container` functions but invoke
+them differently.
+
+Support for `type: Chroot` functions could be added shortly after initial
+support for `type: Container` functions are released if `type: Container` proves
+to be insufficiently compatible (e.g. for clusters running gVisor, or that
+require seccomp be enabled).
+
+### Using Kubernetes to Run Containerized Functions
+
+Asking Kubernetes to run a container pipeline is less straightforward than you
+might think. Crossplane could schedule a `Pod` for each XR reconcile, or create
+a `CronJob` to do so regularly. Another option could be to connect directly to a
+Kubelet. This approach would enjoy all the advantages of the existing Kubelet
+machinery (pull secrets, caching, etc) but incurs overhead in other areas, for
+example:
+
+* Every reconcile requires a pod to be scheduled, which may potentially block on
+  node scale-up, etc.
+* stdin and stdout must be streamed via the API server, for example by using the
+  [`/attach` subresource][attach].
+* Running containers in order requires either (ab)using init containers or
+  injecting a middleware binary that blocks container starts to ensure they run
+  in order (similar to Argo Workflow's '[emissary]' executor):
+
+> The emissary works by replacing the container's command with its own command.
+> This allows that command to capture stdout, the exit code, and easily
+> terminate your process. The emissary can also delay the start of your process.
+
+You can see some of the many options Argo Workflows explored to address these
+issues before landing on `emissary` in their list of
+[deprecated executors][argo-deprecated-executors].
+
+### Using KRM Function Spec Compliant Functions
+
+While the design proposed by this document is heavily inspired by KRM Functions,
+the [KRM function specification][krm-fn-spec] as it currently exists is not an
+ideal fit. This is because:
+
+1. It is built around the needs of CLI tooling - including several references to
+   (client-side) 'files' that don't exist in the Crossplane context. 
+1. Crossplane needs additional metadata to distinguish which resource in the
+   `ResourceList` is the composite resource and which are the composed
+   resources.
+
+### gVisor
+
+[gVisor][gvisor] supports rootless mode, but requires too many privileges to run
+in a container. A proof-of-concept [exists][gvisor-unpriv] to add an
+`--unprivileged` flag to gVisor, allowing it to run inside a container. It's
+unlikely that gVisor will work in all situations in the near future - for
+example gVisor cannot currently run inside gVisor and support for anything other
+than x86 architectures is experimental.
+
+[term-composition]: https://crossplane.io/docs/v1.9/concepts/terminology.html#composition
+[v0.10.0]: https://github.com/crossplane/crossplane/releases/tag/v0.10.0
+[composition-design]: https://github.com/crossplane/crossplane/blob/e02c7a3/design/design-doc-composition.md#goals
+[declarative-app-management]: https://docs.google.com/document/d/1cLPGweVEYrVqQvBLJg6sxV-TrE5Rm2MNOBA_cxZP2WU/edit
+[bcl]: https://twitter.com/bgrant0607/status/1123620689930358786?lang=en
+[terraform-count]: https://www.terraform.io/language/meta-arguments/count
+[turing-complete]: https://en.wikipedia.org/wiki/Turing_completeness#Unintentional_Turing_completeness  
+[pitfalls-dsl]: https://github.com/kubernetes/community/blob/8956bcd54dc6f99bcb681c79a7e5399289e15630/contributors/design-proposals/architecture/declarative-application-management.md#pitfalls-of-configuration-domain-specific-languages-dsls
+[controller-runtime]: https://github.com/kubernetes-sigs/controller-runtime
+[krm-fn-spec]: https://github.com/kubernetes-sigs/kustomize/blob/9d5491/cmd/config/docs/api-conventions/functions-spec.md
+[rawextension]: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#rawextension
+[unix-domain-sockets]: https://man7.org/linux/man-pages/man7/unix.7.html
+[rootless]: https://rootlesscontaine.rs/how-it-works/userns/
+[cgroups-v2-distros]: https://rootlesscontaine.rs/getting-started/common/cgroup2/#checking-whether-cgroup-v2-is-already-enabled
+[overlayfs]: https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html
+[go-containerregistry]: https://github.com/google/go-containerregistry
+[oci-rt-cfg]: https://github.com/opencontainers/runtime-spec/blob/v1.0.2/config.md
+[oci-img-cfg]: https://github.com/opencontainers/image-spec/blob/v1.0.2/config.md
+[gvisor-mountns]: https://github.com/google/gvisor/issues/221
+[kep-seccomp]: https://github.com/kubernetes/enhancements/issues/2413
+[package-meta]: https://github.com/crossplane/crossplane/blob/035e77b/design/one-pager-package-format-v2.md
+[xpkg-spec]: https://github.com/crossplane/crossplane/blob/035e77b/docs/reference/xpkg.md
+[attach]: https://github.com/kubernetes/kubectl/blob/18a531/pkg/cmd/attach/attach.go
+[emissary]: https://github.com/argoproj/argo-workflows/blob/702b293/workflow/executor/emissary/emissary.go#L25
+[argo-deprecated-executors]: https://github.com/argoproj/argo-workflows/blob/v3.4.1/docs/workflow-executors.md
+[krm-fn-spec]: https://github.com/kubernetes-sigs/kustomize/blob/9d5491/cmd/config/docs/api-conventions/functions-spec.md
+[krm-fn-runtimes]: https://github.com/GoogleContainerTools/kpt/issues/2567
+[krm-fn-catalog]: https://catalog.kpt.dev
+[gvisor]: https://gvisor.dev
+[gvisor-unpriv]: https://github.com/google/gvisor/issues/4371#issuecomment-700917549
\ No newline at end of file
diff --git a/design/design-doc-composition-functions.md b/design/design-doc-composition-functions.md
index 107ccef51..e6a84e6ce 100644
--- a/design/design-doc-composition-functions.md
+++ b/design/design-doc-composition-functions.md
@@ -1,8 +1,8 @@
- # Composition Functions
+ # Composition Functions: Beta
 
-* Owners: Nic Cope (@negz), Sergen Yalçın (@sergenyalcin)
-* Reviewers: Crossplane Maintainers
-* Status: Draft
+* Owners: Nic Cope (@negz)
+* Reviewers: Hasan Turken (@turkenh), Jared Watts (@jbw976)
+* Status: Accepted
 
 ## Background
 
@@ -99,74 +99,99 @@ experiences designing Kubernetes, its precursor Borg, and many generations of
 configuration languages at Google, including [BCL/GCL][bcl]. In particular, we
 wanted to:
 
-* Avoid organically growing a new DSL. These languages tend to devolve to
-  incoherency as stakeholders push to “bolt on” new functionality to solve
-  pressing problems at the expense of measured design. Terraform’s DSL
-  unintuitively [supporting the count argument in some places but not
+* Avoid organically growing a new configuration Domain Specific Language (DSL).
+  These languages tend to devolve to incoherency as stakeholders push to “bolt
+  on” new functionality to solve pressing problems at the expense of measured
+  design. Terraform’s DSL [supporting the count argument in some places but not
   others][terraform-count] is a great example of this. Inventing a new DSL also
   comes with the cost of inventing new tooling to test, lint, generate, etc,
   your DSL.
 * Stick to configuration that could be modeled as a REST API. Modeling
-  Composition logic as a schemafied API resource makes it possible for
-  Crossplane to validate that logic and provide feedback to the platform team at
-  configuration time. It also greatly increases interoperability thanks to broad
-  support across tools and languages for interacting with REST APIs.
-
-It was also important to avoid the “worst of both worlds” - i.e. growing a fully
-featured ([Turing-complete][turing-complete]) DSL modeled as a REST API. To this
-end we omitted common language features such as conditionals and iteration. Our
-rationale being that these features were better deferred to a General Purpose
-Programming Language (GPL) designed by language experts, and with extensive
-existing tooling.
+  Composition logic as a schemafied API resource makes it possible to validate
+  that logic and provide feedback to the platform team at configuration time. It
+  also greatly increases interoperability thanks to broad support across tools
+  and languages for interacting with REST APIs.
+
+It was also important to avoid the “worst of both worlds” - i.e. growing a new,
+fully featured DSL modeled as a REST API. To this end we omitted common language
+features such as conditionals and iteration. Our rationale being that these
+features were better deferred to a General Purpose Programming Language (GPL) or
+a mature, existing configuration DSL, with an established ecosystem of tooling
+and documentation.
 
 Since its inception the Crossplane maintainers’ vision has been that there
 should essentially be two variants of Composition:
 
 * For simple cases, use contemporary "Patch and Transform" (P&T) Composition.
-* For advanced cases, bring your tool or programming language of choice.
-
-In this context a “simple case” might involve composing fewer than ten resources
-without the need for logic such as conditionals and iteration. Note that the
-Composition logic, whether P&T or deferred to a tool or programming language, is
-always behind the API line (behind an XR). This means the distinction is only
-important to the people authoring the Compositions, never to the people
-consuming them.
-
-Offering two variants of Composition allows Crossplane users to pick the one
-that is best aligned with their situation, preferences, and experience level.
-For simple cases you don’t need to learn a new programming language or tool, and
-there are no external dependencies - just write familiar, Kubernetes-style YAML.
-For advanced cases leverage proven tools and languages with existing ecosystems
-and documentation. Either way, Crossplane has no religion - if you prefer not to
-“write YAML”, pick another tool and vice versa.
+* For advanced cases, use your DSL or GPL of choice.
+
+Note that the Composition logic (whether P&T, DSL, or GPL) is always "behind the
+API line" (behind an XR). This means the distinction is only important to the
+people _authoring_ the Compositions, never to the people consuming them.
+
+In the time since P&T Composition became available we've seen that:
+
+* Folks want to use Composition for more complex cases than we anticipated. Many
+  XRs fan out into tens of composed resources, sometimes nested.
+* Many Compositions call for a high level of expressiveness - conditionals,
+  iteration, merging data from multiple fields, etc.
+* The lack of a more expressive alternative to P&T Composition _has_ set us down
+  the path of organically growing a new DSL. (e.g. [#1972], [#2352], [#4051],
+  [#3917], [#3919], [#3989], [#3498], [#3458], [#3316], [#4036], [#4065],
+  [#4026])
+* Organically growing a new DSL is not only undesirable, but _slow_. Because
+  each addition to P&T Composition changes Crossplane's core API we must be
+  careful about what we accept. Changes take a long time to reach consensus.
+  They're coupled to Crossplane's release cycle, which means changes can take a
+  long time to become available, and aren't backported to older versions.
+* How folks would ideally configure Composition varies. Some folks prefer a
+  particular configuration DSL, others look for webhooks written in a GPL, etc.
+
+To address these issues, we added support for 'Composition Functions' in
+Crossplane v1.11. Functions are an alpha feature and are off by default. In the
+[alpha design][alpha-design] we proposed that how a Function was implemented
+could vary in at least two ways:
+
+* By `type` - i.e. `type: Container`, `type: Webhook`.
+* By "runner" implementation - i.e. how a `type: Container` Function is run.
+
+We implemented a "default" or "reference" `type: Container` Function runner with
+known scalability and compatibility constraints, but didn't lay out a clear path
+for others to build alternative Function runners. Since alpha, no alternative
+Function runners have appeared.
+
+The aforementioned limitations of the reference Function runner lead to a demand
+to support webhook Functions as an alternative. Ultimately we were reluctant to
+support this because it fragments the Function ecosystem. When a Function can be
+an OCI container or a webhook (or more in future?) there is no one artifact that
+constitutes "a Function". This makes discovery and distribution of Functions
+more complicated than we would like.
+
+This document is an iteration on (and supersedes) the [previous design
+document][alpha-design] based on what we've learned since the feature was
+launched. In particular, it is motivated by the desire to identify a single kind
+of Function artifact suitable for most people - similar to Providers.
 
 ## Goals
 
 The proposal put forward by this document should:
 
-* Let folks use their composition tool and/or programming language of choice.
 * Support 'advanced' composition logic such as loops and conditionals.
-* Balance safety (e.g. sandboxing) with speed and simplicity.
-* Be possible to introduce behind a feature flag that is off by default.
-
-While not an explicit goal, it would also be ideal if the solution put forth by
-this document could serve as a test bed for new features in the contemporary
-'resources array' based form of Composition.
-
-The user experience around authoring and maintaining Composition Functions is
-out of scope for this proposal, which focuses only on adding foundational
-support for the feature to Crossplane. 
+* Let folks specify composition logic in their DSL or GPL of choice.
+* Make it easy to extend Crossplane with new ways to 'do composition'.
+* Decouple adding new ways to 'do composition' from the core release cycle.
+* Make it easy to discover and share new ways to 'do composition'.
+* Be possible to keep behind a feature flag until it is generally available.
 
 ## Proposal
 
-### Overview
-
-This document proposes that a new `functions` array be added to the existing
-`Composition` type. This array of functions would be called either instead of or
-in addition to the existing `resources` array in order to determine how an XR
-should be composed. The array of functions acts as a pipeline; the output of
-each function is passed as the input to the next. The output of the final
-function tells Crossplane what must be done to reconcile the XR.
+This document proposes that a new `pipeline` array of Function calls be added to
+the existing `Composition` type. This array of Functions would be called either
+instead of or in addition to the existing `resources` array in order to
+determine how an XR should be composed. The array of Functions acts as a
+pipeline; the output of each Function is passed as the input to the next. The
+output of the final Function call tells Crossplane what must be done to
+reconcile the XR.
 
 ```yaml
 apiVersion: apiextensions.crossplane.io/v2alpha1
@@ -175,32 +200,609 @@ metadata:
   name: example
 spec:
   compositeTypeRef:
-    apiVersion: database.example.org/v1alpha1
+    apiVersion: database.example.org/v1
     kind: XPostgreSQLInstance
-  functions:
-  - name: my-cool-function
-    type: Container
-    container:
-      image: xkpg.io/my-cool-function:0.1.0
+  # This Composition uses a pipeline of Functions instead of (P&T) resources.
+  pipeline:
+    # Each step in the pipeline calls one Composition Function.
+  - step: compose-xr-using-go-templates
+    # The functionRef tells the Composition which Function to call. Crossplane
+    # passes the desired and observed state of the XR and any existing composed
+    # resources as 'arguments' to the Function call.
+    functionRef:
+      name: go-templates
+    # A Function call may optionally accept input. Think of this like an
+    # additional, optional argument to the Function call. The input is a nested
+    # KRM resource - i.e. it has an apiVersion and kind.
+    input:
+      apiVersion: example.org/v1
+      kind: GoTemplate
+      source: Remote
+      remote: git://github.com/example/my-xpostgresql-go-templates
+    # A pipeline can have multiple steps. Each step is processed in order. This
+    # validation step is passed the desired state accumulated by the prior step.
+  - step: validate-composed-resources
+    functionRef:
+      name: cel-validation
 ```
 
-Under this proposal each function is the entrypoint of an OCI image, though the
-API is designed to support different function implementations (such as webhooks)
-in the future. The updated API would affect only the `Composition` type - no
-changes would be required to the schema of `CompositeResourceDefinitions`, XRs,
-etc.
+The updated API would affect only the `Composition` type - no changes would be
+required to the schema of `CompositeResourceDefinitions`, XRs, etc.
 
-Notably the functions would not be responsible for interacting with the API
-server to create, update, or delete composed resources. Instead, they instruct
-Crossplane which resources should be created, updated, or deleted.
-
-Under the proposed design functions could also be used for purposes besides
+Notably the Functions would not need to be responsible for interacting with the
+API server to create, update, or delete composed resources. Instead, they
+instruct Crossplane which resources should be created, updated, or deleted.
+Under the proposed design Functions could also be used for purposes besides
 rendering composed resources, for example validating the results of the
-`resources` array or earlier functions in the `functions` array. Furthermore, a
-function could also be used to implement 'side effects' such as triggering a
-replication or backup.
+`resources` array or earlier Functions in the `pipeline`.
 
-Below is a more detailed example of an entry in the `functions` array.
+Before you can use a Function, you must install it. Installing a Function works
+just like installing a Provider:
+
+```yaml
+apiVersion: pkg.crossplane.io/v1beta1
+kind: Function
+metadata:
+  name: go-templates
+spec:
+  package: xpkg.upbound.io/negz/go-templates:v0.1.0
+```
+
+The Function package's `metadata.name` corresponds to the `functionRef` in the
+previous example Composition. A Composition specifies which Function to run by
+referencing the package's name.
+
+### Calling a Function
+
+Despite the name, a 'Function' is actually more like a 'function server'. Under
+this proposal, Functions are long-running processes. When you install one, the
+package manager deploys it using a Kubernetes Deployment - the same way it would
+deploy a Provider.
+
+![Crossplane calling three Functions via gRPC](assets/design-doc-composition-functions/functions.png)
+
+Crossplane makes a gRPC `RunFunctionRequest` to the Function it wishes to
+invoke. The Function should respond with a `RunFunctionResponse`. These RPCs are
+defined as follows:
+
+```protobuf
+syntax = "proto3";
+
+import "google/protobuf/struct.proto";
+import "google/protobuf/duration.proto";
+
+// A FunctionRunnerService is a Composition Function.
+service FunctionRunnerService {
+  // RunFunction runs the Composition Function.
+  rpc RunFunction(RunFunctionRequest) returns (RunFunctionResponse) {}
+}
+
+// A RunFunctionRequest requests that the Composition Function be run.
+message RunFunctionRequest {
+  // Metadata pertaining to this request.
+  RequestMeta meta = 1;
+
+  // The observed state prior to invocation of a Function pipeline. State passed
+  // to each Function is fresh as of the time the pipeline was invoked, not as
+  // of the time each Function was invoked.
+  State observed = 2;
+
+  // Desired state according to a Function pipeline. The state passed to a
+  // particular Function may have been accumulated by processing a Composition's
+  // patch-and-transform resources array. It may also have been accumulated by
+  // previous Functions in the pipeline.
+  State desired = 3;
+
+  // Optional input specific to this Function invocation. A JSON representation
+  // of the 'input' block of the relevant entry in a Composition's pipeline.
+  optional google.protobuf.Struct input = 4;
+}
+
+// A RunFunctionResponse contains the result of a Composition Function run.
+message RunFunctionResponse {
+  // Metadata pertaining to this response.
+  ResponseMeta meta = 1;
+
+  // Desired state according to a Function pipeline. Functions may add desired
+  // state, and may mutate or delete any part of the desired state they are
+  // concerned with. A Function must pass through any part of the desired state
+  // that it is not concerned with.
+  State desired = 2;
+
+  // Results of the Function run. Results are used for observability purposes.
+  repeated Result results = 3;
+}
+
+// RequestMeta contains metadata pertaining to a RunFunctionRequest.
+message RequestMeta {
+  // An opaque string identifying the content of the request. Two identical
+  // requests should have the same tag.
+  string tag = 1;
+}
+
+// ResponseMeta contains metadata pertaining to a RunFunctionResponse.
+message ResponseMeta {
+  // An opaque string identifying the content of the request. Must match the
+  // meta.tag of the corresponding RunFunctionRequest.
+  string tag = 1;
+
+  // Time-to-live of this response. Deterministic Functions with no side-effects
+  // (e.g. simple templating Functions) may specify a TTL. Crossplane may choose
+  // to cache responses until the TTL expires.
+  optional google.protobuf.Duration ttl = 2;
+}
+
+// State of the composite resource (XR) and any composed resources.
+message State {
+  // The state of the composite resource (XR).
+  Resource composite = 1;
+
+  // The state of any composed resources.
+  map<string, Resource> resources = 2;
+}
+
+// A Resource represents the state of a resource.
+message Resource {
+  // The JSON representation of the resource.
+  google.protobuf.Struct resource = 1;
+
+  // The resource's connection details.
+  map<string, bytes> connection_details = 2;
+}
+
+// A Result of running a Function.
+message Result {
+  // Severity of this result.
+  Severity severity = 1;
+
+  // Human-readable details about the result.
+  string message = 2;
+}
+
+// Severity of Function results.
+enum Severity {
+  SEVERITY_UNSPECIFIED = 0;
+
+  // Fatal results are fatal; subsequent Composition Functions may run, but
+  // the Composition Function pipeline run will be considered a failure and
+  // the first fatal result will be returned as an error.
+  SEVERITY_FATAL = 1;
+
+  // Warning results are non-fatal; the entire Composition will run to
+  // completion but warning events and debug logs associated with the
+  // composite resource will be emitted.
+  SEVERITY_WARNING = 2;
+
+  // Normal results are emitted as normal events and debug logs associated
+  // with the composite resource.
+  SEVERITY_NORMAL = 3;
+}
+```
+
+This RPC is essentially the `RunFunctionRequest` from the [alpha Functions
+design][alpha-design] with the [`FunctionIO`][functionio-schema] elevated from
+opaque YAML-encoded bytes to 'native' RPC code. Kubernetes resources are
+represented using the [`google.protobuf.Struct` well-known
+type][google-protobuf-struct], which can encode arbitrary JSON.
+
+Some key differences between the alpha `FunctionIO` and the proposed beta
+`RunFunctionRequest`:
+
+* `observed.resources` and `desired.resources` are a map keyed by resource name,
+  not an array of objects with name fields. The previous pattern was a result of
+  `FunctionIO` attempting to be an idiomatic KRM object, which is not a
+  constraint for `RunFunctionRequest`. This is should make it easier to lookup
+  resources by name when developing Functions.
+* Entries in `desired.resources` can no longer return 'derived' connection
+  secrets or readiness checks, similar to an entry in the P&T resources array.
+  Instead a Function is intended to set XR connection details and/or readiness
+  by mutating them directly.
+
+The package manager is responsible for creating a headless Kubernetes Service
+where each Function's Deployment can be reached. The address of the Service will
+be exposed as the `status.endpoint` of the Function resource. The Service must
+be headless in order for Crossplane's gRPC client to load-balance connections
+when there are multiple Function replicas.
+
+Note that the fact this endpoint is powered by a Service is an implementation
+detail; it may be possible for Functions to be reached (and indeed deployed) by
+other means in future (see [Runtime Configuration](#runtime-configuration)).
+
+```yaml
+apiVersion: pkg.crossplane.io/v1beta1
+kind: Function
+metadata:
+  name: go-templates
+spec:
+  package: xpkg.upbound.io/negz/go-templates:v0.1.0
+status:
+  # The gRPC endpoint where Crossplane will send RunFunctionRequests.
+  endpoint: https://go-templates-9sdfn2
+```
+
+gRPC communication between Crossplane and a Function will be secured by mutual
+transport layer security (mTLS). Crossplane has an established pattern for
+setting up mTLS certificate bundles - we do this today to secure communication
+between the API server and our validation webhooks, and between Crossplane and
+External Secret Stores. We might want to [rethink this][cert-per-entity] in
+future, but to begin with we'll stick with the established pattern, i.e.:
+
+* The Helm chart creates an empty Secret for the client bundle.
+* The core init container bootstraps the client bundle.
+* The package manager creates a Secret for each Function's server bundle.
+* The package manager bootstraps each Function's server bundle.
+
+See [#3884] for an example of this pattern.
+
+The frequency with which Crossplane will call each Function is relative to how
+frequently each XR is reconciled. There are two main triggers to reconcile an XR:
+
+1. A watch-triggered reconcile, i.e. because the XR changed.
+2. A poll-triggered reconcile, by default once per 60 seconds.
+
+XRs rely on poll-triggered reconciliation to promptly correct drift of their
+composed resources. The poll interval can be set using the `--poll-interval`
+flag. The XR controller does not know what kinds of resources it will compose
+when started, so it cannot start a watch for them.
+
+Note that there is no watch on the Composition type. XRs consume a Composition
+via a CompositionRevision. Each time the Composition changes a new revision is
+created. The XR is pinned to a specific revision of a Composition until it is
+updated to consume a new one. This update would trigger a reconcile due to the
+watch on the XR.
+
+Assume:
+
+* There are 100 XRs, all of the same type, and all using the same Composition.
+* The XRs have all reached a steady state, polling at the default 60s frequency.
+* The Composition includes 1 Function call.
+
+In this case the Function would be called __100 * 1 = 100 times per 60s__.
+
+Most gRPC server implementations can serve multiple requests concurrently. For
+example in Go [each RPC is invoked in its own goroutine][go-grpc-concurrency].
+The gRPC project maintains a [suite of benchmarks][grpc-benchmarks]. At the time
+of writing these benchmarks show that an 8-core Python server can serve ~3,500
+trivial requests per second, while an 8-core Go server can serve ~180,000 per
+second. This should be considered a ceiling rather than indicative of real-world
+Function performance. In practice the throughput of a particular Function will
+be determined by the language it's written in, and what it does.
+
+Inevitably some Functions will be called frequently enough to exhaust the
+processing capacity of a single pod. A few potential approaches are possible to
+scale Functions include:
+
+* Tune the `--poll-interval`, e.g. to every 5m or 10m, to reduce calls.
+* Scale heavily used Functions horizontally, by increasing Deployment replicas.
+* Cache the responses of deterministic Function calls (see [Caching](#caching)).
+* Eliminate poll-triggered XR reconciliation to reduce calls (see [Dynamic
+  Composed Resource Watches](#dynamic-composed-resource-watches)).
+
+### Developing a Function
+
+I believe it's critical that Composition Functions are easy to develop - much
+easier than developing a Crossplane Provider.
+
+Not everyone will want or need to develop Functions. Some Functions will be
+_generic_. A generic Function would be able to work with any kind of XR.
+Consider for example a `go-templates` Function that effectively "outsourced" the
+entire Composition logic of an XR to a set of Go templates - a familiar
+authoring experience to anyone who has used Helm.
+
+Despite the potential for generic Functions, I feel it must be easy to "roll
+your own" because:
+
+* We want to make it really easy to make new generic Functions! The easier
+  Functions are to build, the more community members can contribute to the
+  ecosystem.
+* Sometimes there won't be a perfect Function for your unique use-case. The
+  easier Functions are to build, the more likely you are to be able to help
+  yourself.
+* Some folks just _prefer_ to configure how to take an XR and produce a set of
+  composed resources using the expressiveness and familiarity of a
+  general-purpose programming language. The more complex your Composition logic
+  becomes, the more appealing this approach is.
+
+I propose we do three things to make Functions easy to develop:
+
+1. Provide SDKs for popular languages - e.g. Go, Python, TypeScript.
+2. Provide tooling to scaffold, build, push, and test Functions.
+3. Support providing Function logic as an OCI container that runs to completion.
+
+Consider this section a medium-resolution sketch of the Function development
+experience. Likely each of these components (tooling, SDKs, etc) warrant at
+least a one-pager of their own, and they may not look exactly as proposed here.
+This section intends only to paint a picture of what Functions enable.
+
+#### Use Language SDKs
+
+Consider for example the following experience for writing a Go Function. Keep in
+mind a similar experience would also exist for other languages.
+
+```shell
+# Initialize a new Function that uses the Go SDK.
+$ kubectl crossplane init function --template="github.com/crossplane/fn-template-go"
+
+# This generates boilerplate that uses the Go SDK.
+$ ls
+crossplane.yaml  fn.go  fn_test.go  go.mod  main.go
+
+# At a minimum you need to add your Function logic to fn.go
+vim fn.go
+```
+
+Note that Functions have package metadata (in `crossplane.yaml` by convention)
+just like a Provider or Configuration. A Function may declare dependencies - for
+example a Function could depend on a Provider to indicate that it composes that
+Provider's MRs.
+
+After you initialize a Function using the Go SDK you must update fn.go to
+provide your composition logic. When you first open fn.go you’ll see that
+`kubectl crossplane init function` added boilerplate like this:
+
+```go
+package main
+
+import (
+    "context"
+
+    function "github.com/crossplane/fn-sdk-go"
+    "github.com/crossplane/fn-sdk-go/pb"
+    "github.com/crossplane/fn-sdk-go/errors"
+)
+
+// Function defines the logic of your Function.
+func Function(ctx context.Context, req *pb.RunFunctionRequest) (*pb.RunFunctionResponse, error) {
+  // Get the desired XR from the request.
+  xr := function.NewEmptyCompositeResource()
+  if err := function.GetCompositeResource(req, xr); err != nil {
+      return errors.Wrap(err, "could not get composite resource")
+  }
+
+  // TODO: Replace this with your function logic. :)
+  rsp := function.NewResponseTo(req)
+  function.Normalf(rsp, "successfully read desired composite resource %q", xr.GetName())
+
+  return rsp, nil
+}
+```
+
+To write a Function, replace the boilerplate with your own logic. Here’s an
+example that uses a mockup of a Go SDK to achieve the same goal as lab 4 from
+[the Kubecon EU ContribFest][kubecon-eu-contribfest].
+
+```yaml
+apiVersion: contribfest.crossplane.io/v1alpha1
+kind: XRobotGroup
+metadata:
+  name: somename
+spec:
+  count: 5
+```
+
+This example takes an `XRobotGroup` XR and uses its `spec.count` to compose the
+desired number of Robot resources:
+
+```go
+package main
+
+import (
+    "context"
+    "fmt"
+    "math/rand"
+
+    "github.com/upbound/provider-dummy/apis/iam/v1alpha1"
+
+    function "github.com/crossplane/fn-sdk-go"
+    "github.com/crossplane/fn-sdk-go/pb"
+    "github.com/crossplane/fn-sdk-go/errors"
+)
+
+var colors = []string{"red", "green", "blue", "yellow", "orange"}
+
+// Function defines the logic of your Function.
+func Function(ctx context.Context, req *pb.RunFunctionRequest) (*pb.RunFunctionResponse, error) {
+  // Get the desired XR from the request.
+  xr := function.NewEmptyCompositeResource()
+  function.GetCompositeResource(req, xr)
+
+  // Get the desired resource count from the XR.
+  count, err := xr.GetInteger("spec.count")
+  if err != nil {
+      return errors.Wrap(err, "could not get desired resource count")
+  }
+
+  // Create a response to return. This deep-copies any existing desired state.
+  rsp := function.NewResponseTo(req)
+
+  // Ensure the desired number of robots exist.
+  for i := 0; i < int(count); i++ {
+      name := fmt.Sprintf("robot-%d")
+
+      // Get the desired composed resource (if any) from the request.
+      // Since we're using Go we can import and work with API types if we want.
+      robot := &v1alpha1.Robot{}
+      function.GetComposedResource(req, name, robot)
+
+      // The robot's external name should be derived from the XR's.
+      function.SetExternalName(robot, fmt.Sprintf("%s-%s", function.GetExternalName(xr), name))
+
+      // Give this robot a random color!
+      if robot.Spec.ForProvider.Color != "" {
+          c := colors[rant.Intn(len(colors))]
+          robot.Spec.ForProvider.Color = c
+
+          // Add a result indicating that we set the color. Crossplane will
+          // surface this as a Kubernetes event and a debug log.
+          function.Normalf(rsp, "set robot %q color to %q", name, c)
+      }
+
+      // Set our new desired robot state. This will be a no-op if our robot
+      // already existed.
+      function.SetComposedResource(rsp, name, robot)
+  }
+
+  return rsp, nil
+}
+```
+
+The goals of a Composition Function SDK are to:
+
+* Eliminate boilerplate - make it feel like writing a function, not a server.
+* Expose an API that makes writing Functions intuitive and self-documenting.
+* Steer Function authors toward best practices, and away from anti-patterns.
+
+This mockup only touches on very basic functionality. A fully featured SDK would
+cover more use cases - for example reading Function-specific input and working
+with connection details.
+
+Once you’ve written your Function code you can test it using the tooling for
+your chosen language - in this case `go test`. `kubectl crossplane` would create
+boilerplate for unit tests.
+
+You build and push a Function just like you would a Provider or Configuration.
+There are already GitHub Actions that allow this to be done as part of a CI/CD
+pipeline.
+
+```shell
+# Building the Function produces an installable Function package.
+$ kubectl crossplane function build
+
+# Push the package to any OCI registry.
+$ kubectl crossplane function push xpkg.upbound.io/negz/function-many-robots:v0.1.0
+```
+
+Note that leveraging gRPC makes it possible to generate server stubs for [many
+languages][grpc-supported-languages]. I would not consider these server stubs to
+be 'an SDK' - they don't make writing Functions intuitive and self-documenting,
+or steer Function authors toward best practices. They do however go a long way
+toward eliminating boilerplate, including generating language-specific
+representations of a `RunFunctionRequest` and `RunFunctionResponse`.
+
+#### Use an OCI Container
+
+In the [alpha iteration of Composition Functions][alpha-design] all Functions
+are binaries that run to completion. These processes are packaged as an OCI
+container. A sidecar container in the Crossplane pod - `xfn` - runs each of them
+in a rootless container.
+
+This doesn't scale very well, but it does offer a way to build a Function for
+folks who don't want to write code; you can get a long way just using a tool
+like https://github.com/mikefarah/yq. It also offers a way to build Functions in
+languages that don't have a 'first class' SDK available. Think of this like AWS
+Lambda, which supports 9-10 first class languages, and OCI containers as a
+catch-all for everything else.
+
+I propose we move the `xfn` Function runner out of crossplane/crossplane,
+simplify it, and offer it as an alternative way to build a Function. You could
+think of this as a special kind of Composition Function SDK that builds a
+Composition Function from an OCI container, rather than building a Function a
+literal _function_ (e.g. written in Go).
+
+![Crossplane calling two containerized Functions via gRPC](assets/design-doc-composition-functions/containerized-functions.png)
+
+Under this design each containerized Function is "wrapped" in an `xfn`-like
+Composition Function. This means each Function exists as its own Deployment.
+
+This design merges the Protobuf `RunFunctionRequest` with the YAML `FunctionIO`
+types from the alpha design, so instead of reading a `FunctionIO` from stdin a
+containerized Function would read a JSON-encoded `RunFunctionRequest`, and write
+a `RunFunctionResponse` to stdout.
+
+Building a containerized Function would work as follows.
+
+Start with a simple container that reads a JSON `RunFunctionRequest` on stdin
+and writes a JSON `RunFunctionResponse` to stdout. This one simply adds a label
+(credit to @pedjak in https://github.com/crossplane/crossplane/issues/4293) for
+the idea).
+
+```Dockerfile
+FROM alpine:3.18
+RUN apk add --no-cache jq
+ENTRYPOINT ["/bin/sh", "-c", "jq '(.desired.resources[] | .resource.metadata.labels) |= {\"labelizer.xfn.crossplane.io/processed\": \"true\"} + .' | .desired"]
+```
+
+Then build a Function from the image. We want the image to run in its own
+container, not the Function container. This allows the OCI container to run
+within its own namespaces without interfering with (or being interfered with by)
+the `xfn`-like 'adaptor'. To do this we bake the container into the Function
+artifact:
+
+```shell
+# Build your OCI image.
+$ docker build .
+Successfully built 202dc6e5df4c
+
+# Save your OCI image to a tarball.
+$ docker save | gzip > function.tgz
+
+# Add some package metadata for your Function.
+$ vim crossplane.yaml
+
+# Build a Function from the tarball.
+$ kubectl crossplane function build --from-oci-tarball=function.tgz
+
+# Push the package to any OCI registry.
+$ kubectl crossplane function push xpkg.upbound.io/negz/function-labelizer:v0.1.0
+```
+
+The `function build --from-oci-tarball` command creates a Function powered by an
+`xfn`-like adaptor that:
+
+1. Prepares an OCI runtime bundle from the supplied tarball at startup.
+2. Listens for `RunFunctionRequest` RPCs.
+3. Prepares `RunFunctionResponse` RPCs by using an OCI runtime (e.g. `crun`) to
+   invoke the embedded Function in a rootless container.
+
+Note that most of this functionality already exists in `xfn`. This makes it
+cheap enough to implement this Function variant that I believe it's worth doing,
+even if it ends up covering a relatively niche case.
+
+### Installing a Function
+
+Before you can use a Function, you must install it. Installing a Function works
+just like installing a Provider:
+
+```yaml
+apiVersion: pkg.crossplane.io/v1beta1
+kind: Function
+metadata:
+  name: go-templates
+spec:
+  package: xpkg.upbound.io/negz/go-templates:v0.1.0
+```
+
+Like any other package, the Function type will have a corresponding
+FunctionRevision type. The package manager will reconcile the active revision by
+creating a Deployment - similar to how it installs a Provider.
+
+Functions can have and be dependencies, just like a Provider or Configuration.
+For example a Configuration could depend on the Functions its Compositions use,
+and a Function could depend on the Providers and Configurations whose resources
+it will compose.
+
+### Calling a Function
+
+Composition Functions are very flexible, so the user experience could vary a
+lot. However the general pattern is:
+
+1. Find (or build!) a Function you want to use.
+2. Install the Function using the package manager.
+3. Reference the Function in a Composition.
+4. Provide Function input in the Composition, where necessary.
+
+Again, consider this section a medium-resolution sketch of the Function
+experience. The generic Functions demonstrates here don't actually exist (yet!).
+They may not look exactly as proposed here. This section intends only to paint a
+picture of what Functions enable.
+
+Consider the hypothetical generic "Go templates" Function put forward at the
+[beginning of this proposal](#proposal). Such a Function would make it possible
+to write a Composition that used familiar, Helm-like moustache templates with
+support for iteration and conditionals.
+
+If we take the [contribfest][kubecon-eu-contribfest] example again, this might
+look something like this:
 
 ```yaml
 apiVersion: apiextensions.crossplane.io/v2alpha1
@@ -209,542 +811,327 @@ metadata:
   name: example
 spec:
   compositeTypeRef:
-    apiVersion: database.example.org/v1alpha1
+    apiVersion: database.example.org/v1
     kind: XPostgreSQLInstance
-  functions:
-  - name: my-cool-function
-    type: Container
-    # Configuration specific to `type: Container` functions.
-    container:
-      # The OCI image to pull and run.
-      image: xkpg.io/my-cool-function:0.1.0
-      # Whether to pull the function Never, Always, or IfNotPresent.
-      imagePullPolicy: IfNotPresent
-      # Secrets used to pull from a private registry.
-      imagePullSecrets:
-      - namespace: crossplane-system
-        name: my-xpkg-io-creds
-      # Note that only resource limits are supported - not requests.
-      # The function will be run with the specified resource limits.
-      resources:
-        limits:
-          memory: 64Mi
-          cpu: 250m
-      # Defaults to 'Isolated' - i.e an isolated network namespace.
-      network: Accessible
-      # How long the function may run before it's killed. Defaults to 10s.
-      timeout: 30s
-      # Containers are run by an external process listening at the supplied
-      # endpoint. Specifying an endpoint is optional; the endpoint defaults to
-      # the below value.
-      runner:
-        endpoint: unix:///@crossplane/fn/default.sock
-    # An x-kubernetes-embedded-resource RawExtension (i.e. an unschemafied
-    # Kubernetes resource). Passed to the function as the config block of its
-    # FunctionIO.
-    config:
-      apiVersion: database.example.org/v1alpha1
-      kind: Config
-      metadata:
-        name: cloudsql
-      spec:
-        version: POSTGRES_9_6
+  pipeline:
+  - step: compose-xr-using-go-templates
+    functionRef:
+      name: go-templates
+    input:
+      apiVersion: example.org/v1
+      kind: GoTemplate
+      source: Inline  # Or Remote, if you want to pull templates from git.
+      inline: |
+        {{- range $i := until ( . desired.composite.resource.spec.count ) }}
+        ---
+        apiVersion: iam.dummy.upbound.io/v1alpha1
+        kind: Robot
+        metadata:
+          annotations:
+            crossplane.io/external-name: {{ .desired.composite.resource | getExternalName }}-robot-{{ $i }}
+        spec:
+          forProvider:
+          {{- if $i | printf "robot-%d" | getComposedResource }}
+            # This resource already exists - use the same color.
+            color: {{ .spec.forProvider.color }}
+          {{ else }}
+            # The robot doesn't exist. Pick a new color.
+            color: {{ randomChoice "red" "green" "blue }}
+          {{- end }}
+        {{- end }}
 ```
 
-### Function API
+This example is functionally identical to [the Go example](#use-language-sdks),
+but uses a DSL - Go templates with http://masterminds.github.io/sprig/ and a
+few Crossplane-specific Functions.
 
-This document proposes that each function uses a `FunctionIO` type as its input
-and output. In the case of `Container` functions this would correspond to stdin
-and stdout. Crossplane would be responsible for reading stdout from the final
-function and applying its changes to the relevant XR and composed resources.
+Here's another example of a generic Function. This one lets Composition authors
+express their Composition logic in arbitrary, inline [Starlark][starlark].
 
 ```yaml
-apiVersion: apiextensions.crossplane.io/v1alpha1
-kind: FunctionIO
-config:
-  apiVersion: database.example.org/v1alpha1
-  kind: Config
-  metadata:
-    name: cloudsql
-  spec:
-    version: POSTGRES_9_6
-observed:
-  composite:
-    resource:
-      apiVersion: database.example.org/v1alpha1
-      kind: XPostgreSQLInstance
-      metadata:
-        name: my-db
-      spec:
-        parameters:
-          storageGB: 20
-        compositionSelector:
-          matchLabels:
-            provider: gcp
-      status:
-        conditions:
-        - type: Ready
-          status: True
-    connectionDetails:
-    - name: uri
-      value: postgresql://db.example.org:5432
+apiVersion: apiextensions.crossplane.io/v2alpha1
+kind: Composition
+metadata:
+  name: example
+spec:
+  compositeTypeRef:
+    apiVersion: database.example.org/v1
+    kind: XPostgreSQLInstance
+  pipeline:
+  - step: compose-xr-using-go-templates
+    functionRef:
+      name: starlark
+    input:
+      apiVersion: example.org/v1
+      kind: StarlarkScript
+      script: |
+        # main is passed a RunFunctionRequest as a (Python-like) dictionary.
+        # It must return a RunFunctionResponse-shaped dictionary.
+        def main(req):
+          rsp = {desired=req["desired"]}
+          xr = req["observed"]["composite"]["resource"]
+
+          for i in range(int(xr["spec"]["count"])):
+            name = "robot-{}".format(i)
+
+            # We already created this one during a previous reconcile.
+            if name in req["observed"]["resources"]:
+              rsp["desired"]["resources"][name] = req["observed"]["resources"][name]
+              continue
+
+            # This robot should exist, but doesn't.
+            rsp["desired"]["resources"][name] = {
+              "resource": {
+                "apiVersion": "iam.dummy.upbound.io/v1alpha1",
+                "kind": "Robot",
+                "metadata": {
+                  "annotations": {
+                    "crossplane.io/external-name": "{parent}-robot-{number}".format(
+                      parent=xr["metadata"]["annotations"]["crossplane.io/external-name"],
+                      number=i,
+                    )
+                  },
+                },
+                "spec": {
+                  "forProvider": {
+                    # Starlark is deterministic so these Robots are always purple. :)
+                    "color": "purple",
+                  },
+                },
+              }
+            }
+
+          return rsp
 ```
 
-A `FunctionIO` resource consists of the following top-level fields:
-
-* The `apiVersion` and `kind` (required).
-* A `config` object (optional). This is a [Kubernetes resource][rawextension]
-  with an arbitrary schema that may be used to provide additional configuration
-  to a function. For example a `render-helm-chart` function might use its
-  `config` to specify which Helm chart to render. Functions need not return
-  their `config`, and any mutations will be ignored.
-* An `observed` object (required). This reflects the observed state of the XR,
-  any existing composed resources, and their connection details. Functions must
-  return the `observed` object unmodified.
-* A `desired` object (optional). This reflects the accumulated desired state of
-  the XR and any composed resources. Functions may mutate the `desired` object.
-* A `results` array (optional). Used to communicate information about the result
-  of a function, including warnings and errors. Functions may mutate the
-  `results` object.
-
-Each function takes its `config` (if any), `observed` state, and any previously
-accumulated `desired` state as input, and optionally mutates the `desired`
-state. This allows the output of one function to be the input to the next.
-
-The `observed` object consists of:
-
-* `observed.composite.resource`. The observed XR.
-* `observed.composite.connectionDetails`: The observed XR connection details.
-* `observed.resources[N].name`: The name of an observed composed resource.
-* `observed.resources[N].resource`: An observed composed resource.
-* `observed.resources[N].connectionDetails`: An observed composed resource's
-  current connection details.
-
-If an observed composed resource appears in the Composition's `spec.resources`
-array their `name` fields will match. Note that the `name` field is distinct
-from a composed resource's `metadata.name` - it is used to identify the resource
-within a Composition and/or its function pipeline.
-
-The `desired` object consists of:
-
-* `desired.composite.resource`. The desired XR.
-* `desired.composite.resource.connectionDetails`. Desired XR connection details.
-* `desired.resources[N].name`. The name of a desired composed resource.
-* `desired.resources[N].resource`. A desired composed resource.
-* `desired.resources[N].connectionDetails`. A desired composed resource's
-  connection details.
-* `desired.resources[N].readinessChecks`. A desired composed resource's
-  readiness checks.
-
-Note that the `desired.resources` array of the `FunctionIO` type is very
-similar to the `spec.resources` array of the `Composition` type. In comparison:
-
-* `name` works the same across both types, but is required by `FunctionIO`.
-* `connectionDetails` and `readinessChecks` work the same across both types.
-* `FunctionIO` does not support `base` and `patches`. Instead, a function should
-  configure the `resource` field accordingly.
-
-The `desired` state is _accumulated_ across the Composition and all of its
-functions. This means the first function may be passed desired state as
-specified by the `spec.resources` array of a Composite, if any, and each
-function must include the accumulated desired state in its output. Desired state
-is treated as an overlay on observed state, so a function pipeline need not
-specify the desired state of the XR (for example) unless a function wishes to
-mutate it.
-
-A full `FunctionIO` specification will accompany the implementation. Some
-example scenarios are illustrated below.
-
-A function that wanted to create (compose) a `CloudSQLInstance` would do so by
-returning the following `FunctionIO`:
-
-```yaml
-apiVersion: apiextensions.crossplane.io/v1alpha1
-kind: FunctionIO
-observed: {}  # Omitted for brevity.
-desired:
-  resources:
-  - name: cloudsqlinstance
-    resource:
-      apiVersion: database.gcp.crossplane.io/v1beta1
-      kind: CloudSQLInstance
-      spec:
-        forProvider:
-          databaseVersion: POSTGRES_9_6
-          region: us-central1
-          settings:
-            tier: db-custom-1-3840
-            dataDiskType: PD_SSD
-            dataDiskSizeGb: 20
-        writeConnectionSecretToRef:
-          namespace: crossplane-system
-          name: cloudsqlpostgresql-conn
-    connectionDetails:
-    - name: hostname
-      fromConnectionSecretKey: hostname
-    readinessChecks:
-    - type: None
+Starlark is a (very) limited dialect of Python designed for configuration. It
+embeds into many languages, including Go. This makes it a great candidate to
+build a generic Function that allows Composition authors to provide _inline_
+logic in a general-purpose-ish programming language. This is similar to the
+["GitHub Script"][github-script] GitHub Action that lets you plug some arbitrary
+logic into a GitHub Action pipeline when no existing Action does quite what you
+need.
+
+#### Iterating on Compositions
+
+Getting fast feedback on a Crossplane Composition has historically been a pain
+because the Composition logic has been encapsulated in a Kubernetes controller.
+In order to know whether your Composition works as intended you need to:
+
+1. Deploy (the right version of!) Crossplane to a Kubernetes cluster.
+2. Install Providers and Configurations for any resources you want to compose.
+3. Apply your XRD and Composition (potentially via a Configuration).
+4. Create an XR (or claim).
+5. See whether it works.
+
+Moving Composition logic out of Crossplane and into versioned Functions makes it
+a lot easier to test and iterate on, client-side. For example a `function test`
+command could test a single Function in isolation:
+
+```shell
+# Test a single Function by passing it a JSON RunFunctionRequest. This pulls and
+# starts the Function, makes the request, stops it, then returns the result.
+$ kubectl crossplane function test xpkg.upbound.io/negz/go-templates:v0.1.0 run-function-request.json
+{
+  # JSON encoded RunFunctionResponse omitted for brevity.
+}
 ```
 
-A function that wanted to set only an XR connection detail could return:
+You could similarly imagine a more holistic, `helm template`-like experience
+that locally rendered an entire Composition by pulling and running its templates
+locally:
 
-```yaml
-apiVersion: apiextensions.crossplane.io/v1alpha1
-kind: FunctionIO
-observed: {}  # Omitted for brevity.
-desired:
-  composite:
-    connectionDetails:
-    - type: FromValue
-      name: username
-      value: admin
+```shell
+$ kubectl crossplane composition render composition.yaml xr.yaml
+---
+# YAML stream omitted for brevity.
 ```
 
-A function wishing to delete a composed resource may do so by setting its
-`resource` to null, for example:
+Assuming a 'pure' Composition consisting only of Functions with no P&T
+resources, this command would need only to iterate through the Composition's
+`pipeline`, and for each Function:
+
+1. Pull and start it.
+2. Form a `RunFunctionRequest` from the XR and any prior `RunFunctionResponse`.
+3. Print the `desired` block of final `RunFunctionResponse` as a YAML stream.
+
+This isn't quite an end-to-end test of Composition. The composed resources are
+not actually created and reconciled with an external system, so its not possible
+for example to derive XR status from composed resource status. (Though that
+could perhaps be simulated by providing fake observed resources.) It's also not
+possible in isolation to determine whether the rendered composed resources are
+schematically valid. It does however give you a good idea of whether your
+Composition will produce the set of composed resources that you'd expect - just
+like `helm template`.
+
+The advantage of Functions in this regard is that the `composition render`
+command would need to duplicate much less code from the Crossplane Composition
+controller than if it were to try to recreate Crossplane's hardcoded P&T logic.
+
+## Future Improvements
+
+The following functionality is out-of-scope for the beta implementation, but may
+be added in future.
+
+### Function Input Custom Resources
+
+In the current alpha implementation of Functions, and this design, Function
+input is a custom-resource-like inline resource (i.e. an
+`x-kubernetes-embedded-resource`):
 
 ```yaml
-apiVersion: apiextensions.crossplane.io/v1alpha1
-kind: FunctionIO
-observed: {}  # Omitted for brevity.
-desired:
-  resources:
-  - name: cloudsqlinstance
-    resource: null
+  pipeline:
+  - step: compose-xr-using-go-templates
+    functionRef:
+      name: starlark
+    input:
+      apiVersion: example.org/v1
+      kind: StarlarkScript
+      script: ...
 ```
 
-A function that could not complete successfully could do so by returning the
-following `FunctionIO`:
+In future it may be useful for a Function to be able to deliver this type as a
+custom resource definition (CRD). This would allow a single input to be more
+easily shared by multiple Compositions. A Composition could reference a Function
+input custom resource:
 
 ```yaml
-apiVersion: apiextensions.crossplane.io/v1alpha1
-kind: FunctionIO
-config:
-  apiVersion: database.example.org/v1alpha1
-  kind: Config
-  metadata:
-    name: cloudsql
-  spec:
-    version: POSTGRES_9_6
-observed: {}  # Omitted for brevity.
-results:
-- severity: Error
-  message: "Could not render Database.postgresql.crossplane.io/v1beta1`
+  pipeline:
+  - step: compose-xr-using-go-templates
+    functionRef:
+      name: starlark
+    inputRef:
+      apiVersion: example.org/v1
+      kind: StarlarkScript
+      name: make-some-purple-robots
 ```
 
-### Running Container Function Pipelines
+At this stage I suggest holding off on building this functionality until there
+is clear demand. 
 
-While Crossplane typically runs in a Kubernetes cluster - a cluster designed to
-run containers - running an ordered _pipeline_ of short-lived containers via
-Kubernetes is much less straightforward than you might expect. Refer to
-[Alternatives Considered](#alternatives-considered) for details.
+### Metrics and Tracing
 
-In order to provide flexibility and choice of tradeoffs in running containers
-(e.g. speed, scalability, security) this document proposes Crossplane defer
-containerized functions to an external runner. Communication with the runner
-would occur via a gRPC API, with the runner expected to be listening at the
-`endpoint` specified via the function's `runner` configuration block. This
-endpoint would default to `unix:///@crossplane/fn/default.sock` - an abstract
-[Unix domain socket][unix-domain-sockets].
+Crossplane does not currently expose its own metrics. It relies on [the set
+it gets from controller-runtime][controller-runtime-metrics], and metrics that
+may be [derived from the Kubernetes events it emits][event-exporter-metrics].
 
-Communication between Crossplane and a containerized function runner would use
-the following API:
+These metrics pertain to Kubernetes controllers, and will be insufficient when a
+significant portion of Composition logic is 'outsourced' to Functions. It will
+be important to establish a pattern for instrumenting Functions (e.g. as part of
+the Function SDKs). Request tracing in particular is likely to be useful in
+order to debug slow or failing Functions in a pipeline.
 
-```protobuf
-syntax = "proto3";
+Metrics and tracing for Functions must be implemented before the feature becomes
+generally available (GA).
 
-// This service defines the APIs for a containerized function runner.
-service ContainerizedFunctionRunner {
-    rpc RunFunction(RunFunctionRequest) returns (RunFunctionResponse) {}
-}
+### Caching
 
-// Corresponds to Kubernetes' image pull policy.
-enum ImagePullPolicy {
-  IF_NOT_PRESENT = 0;
-  ALWAYS = 1;
-  NEVER = 2;
-}
+The API proposed by this design accommodates caching. The `RunFunctionRequest`
+includes a `meta.tag` field. The tag identifies a unique Function input. How the
+tag is generated is up to the caller (i.e. Crossplane), but two functionally
+identical Function inputs should have the same tag. A Function can optionally
+signal that its response is valid for the same input for a period of time by
+returning a non-zero `meta.ttl`. This allows Crossplane (or an intermediary such
+as a reverse proxy) to cache the responses of deterministic and side-effect-less
+Functions.
 
-// Corresponds to go-containerregistry's AuthConfig type.
-// https://pkg.go.dev/github.com/google/go-containerregistry@v0.11.0/pkg/authn#AuthConfig
-message ImagePullAuth {
-  string username = 1;
-  string password = 2;
-  string auth = 3;
-  string identity_token = 4;
-  string registry_token = 5;
-}
+Building caching support is not in scope for beta. This will likely prove to be
+a requirement for GA.
 
-message ImagePullConfig {
-  ImagePullPolicy pull_policy = 1;
-  ImagePullAuth auth = 2;
-}
+### Dynamic Composed Resource Watches
 
-// Containers are run without network access (in an isolated network namespace)
-// by default.
-enum NetworkPolicy = {
-  ISOLATED = 0;
-  ACCESSIBLE = 1;
-}
+Ideally XR reconciliation would be purely watch-triggered - this would result in
+less work for the XR controller, and far fewer Function calls.
 
-// Only resource limits are supported. Resource requests could be added in
-// future if a runner supported them (e.g. by running containers on Kubernetes).
-message Resources {
-  ResourceLimits limits = 1;
-}
+The XR controller currently watches the XR type, but is also poll-triggered, by
+default polling desired state every 60 seconds. This interval can be changed by
+the `--poll-interval` flag. The XR reconciler is poll-triggered because it wants
+to know when composed resources change, in order to correct drift.
 
-message ResourceLimits {
-  string memory = 1;
-  string cpu = 2;
-}
+An XR controller doesn't know what kinds of resources it will compose at start
+time, which is typically when a controller's watches are configured.
+Furthermore, two different XRs of the same kind might compose completely
+different types of resources due to using different Compositions.
 
-message RunFunctionConfig {
-  Resources resources = 1;
-  NetworkPolicy network = 2;
-  Duration timeout = 3;
-}
+This is something we should revisit regardless of Functions. For example it may
+be possible to:
 
-// The input FunctionIO is supplied as opaque bytes.
-message RunFunctionRequest {
-  string image = 1;
-  bytes input = 2;
-  ImagePullConfig = 3;
-  RunFunctionConfig = 4;
-}
+1. Make one controller responsible (only) for selecting a Composition for an XR.
+2. Start another controller for every unique (XR, Composition) tuple.
+3. Restart the (XR, Composition) controller whenever the XR's
+   `spec.resourceRefs` changes, watching all referenced types.
 
-// The output FunctionIO is supplied as opaque bytes. Errors encountered while
-// running a function (as opposed to errors returned _by_ a function) will be
-// encapsulated as gRPC errors.
-message RunFunctionResponse {
-  bytes output = 1;
-}
-```
+### Runtime Configuration
+
+This design proposes Functions be long-running processes, installed by the
+package manager. Deploying Functions as Kubernetes Deployments (with Services,
+Service Accounts, etc) will no doubt necessitate something like a Provider's
+ControllerConfig type.
+
+We've identified that we [don't want to proceed][controllerconfig-deprecation]
+with ControllerConfig, but don't yet have a suitable alternative. Rather than
+propagating a ControllerConfig-like pattern I propose we prioritize finding an
+alternative. I intend to open a separate, simultaneous design to address this
+since it will affect Provider packages as well as Functions.
+
+### Patch-and-Transform as a Function
+
+A key benefit of Functions is that composition logic is decoupled from the core
+Crossplane release cycle. Moving composition logic out-of-tree and versioning it
+separately from Crossplane allows us to iterate faster, and experiment more
+freely.
+
+P&T style Composition could enjoy these benefits if it were not a special case,
+and were just another Function. Imagine for example wanting a new type of patch
+or a new type of transform and being able to simply fork the implementation to
+experiment without affecting everyone who uses Crossplane.
 
-### The Default Function Runner
-
-This document proposes that Crossplane include a default function runner. This
-runner would be implemented as a sidecar to the core Crossplane container that
-runs functions inside itself.
-
-The primary advantages of this approach are speed and control. There's no need
-to wait for another system (for example the Kubernetes control plane) to
-schedule each container, and the runner can easily pass stdout from one
-container to another's stdin. Speed of function runs is of particular importance
-given that each XR typically reconciles (i.e. invokes its function pipeline)
-once every 60 seconds.
-
-The disadvantages of running the pipeline inside a sidecar container are scale
-and reinvention of the wheel. The resources available to the sidecar container
-will bound how many functions it can run at any one time, and it will need to
-handle features that the Kubelet already offers such as pull secrets, caching
-etc.
-
-[Rootless containers][rootless] appear to be the most promising way to run
-functions as containers inside a container:
-
-> Rootless containers uses `user_namespaces(7)` (UserNS) for emulating fake
-> privileges that are enough to create containers. The pseudo-root user gains
-> capabilities such as `CAP_SYS_ADMIN` and `CAP_NET_ADMIN` inside UserNS to
-> perform fake-privileged operations such as creating mount namespaces, network
-> namespaces, and creating TAP devices.
-
-Using user namespaces allows the runner to use the other kinds of namespaces
-listed above to ensure an extra layer of isolation for the functions it runs.
-For example a network namespace could be configured to prevent a function having
-network access.
-
-User namespaces are well supported by modern Linux Kernels, having been
-introduced in Linux 3.8. Many OCI runtimes (including `runc`, `crun`, and
-`runsc`) support rootless mode. `crun` appears to be the most promising choice
-because:
-
-* It is more self-contained than `runc` (the reference and most commonly used
-  OCI runtime), which relies on setuid binaries to setup user namespaces.
-* `runsc` (aka gVisor) uses extra defense in depth features which are not
-  allowed inside most containers due to their seccomp policies.
-
-Of course, "a container" is in fact many technologies working together and some
-parts of rootless containers are less well supported than others; for example
-cgroups v2 is required in order to limit resources like CPU and memory available
-to a particular function. cgroups v2 has been available in Linux since 4.15, but
-was not enabled by many distributions until 2021. In practice this means
-Crossplane users must use a [sufficiently modern][cgroups-v2-distros]
-distribution on their Kubernetes nodes in order to constrain the resources of a
-Composition function.
-
-Similarly, [overlayfs] was not allowed inside user namespaces until Linux 5.11.
-Overlayfs is typically used to create a root filesystem for a container that is
-backed by a read-write 'upper' directory overlaid on a read-only 'lower'
-directory. This allows the root OCI image filesystem to persist as a cache of
-sorts, while changes made during the lifetime of a container can be easily
-discarded. It's possible to replicate these benefits (at the expense of disk
-usage and start-up time) by falling back to making a throwaway copy of the root
-filesystem for each container run where overlayfs is not available.
-
-Under the approach proposed by this document each function run would involve the
-following steps:
-
-1. Use [go-containerregistry] to pull the function's OCI image.
-1. Extract (untar) the OCI image's flattened filesystem to disk.
-1. Create a filesystem for the container - either an overlay or a copy of the
-   filesystem extracted in step 2.
-1. Derive an [OCI runtime configuration][oci-rt-cfg] from the
-   [OCI image configuration][oci-img-cfg] supplied by go-containerregistry.
-1. Execute `crun run` to invoke the function in a rootless container.
-
-Executing `crun` directly as opposed to using a higher level tool like `docker`
-or `podman` allows the default function runner to avoid new dependencies apart
-from a single static binary (i.e. `crun`). It keeps most functionality (pulling
-images etc) inside the runner's codebase, delegating only container creation to
-an external tool. Composition Functions are always short-lived and should always
-have their stdin and stdout attached to the runner, so wrappers like
-`containerd-shim` or `conmon` should not be required. The short-lived, "one
-shot" nature of Composition Functions means it should also be acceptable to
-`crun run` the container rather than using `crun create`, `crun start`, etc.
-
-At the time of writing rootless containers appear to be supported by Kubernetes,
-including Amazon's Elastic Kubernetes Service (EKS) and Google Kubernetes Engine
-(GKE).
-
-Testing using GKE 1.21.10-gke.2000 with Container Optimized OS (with containerd)
-cos-89-16108-604-19 nodes (Kernel COS-5.4.170) found that it was possible to run
-`unshare -rUm` (i.e. to create a new user and mount namespace) inside an Alpine
-Linux container as long as AppArmor was disabled by applying the annotation
-`container.apparmor.security.beta.kubernetes.io/${CONTAINER_NAME}=unconfined`.
-It's possible to create user namespaces with AppArmor enabled, but not to create
-mount namespaces with different mount propagation from their parent.
-
-It is not possible to use rootless containers with gVisor enabled, as gVisor
-does not yet [support mount namespaces][gvisor-mountns]. This means that it is
-not possible to use rootless containers with GKE Autopilot, which requires that
-gVisor be used.
-
-Testing using EKS v1.21.5-eks-9017834 with Amazon Linux 2 nodes (Kernel
-5.4.188-104.359.amzn2.x86_64) found that it was possible to run `unshare -rUm`
-inside an Alpine Linux container 'out of the box'.
-
-The `unshare` syscall used to create containers is rejected by the default
-Docker and containerd seccomp profiles. seccomp is disabled ("Unconstrained") by
-default in Kubernetes, but that will soon change per [this KEP][kep-seccomp]
-which proposes that Kubernetes use the seccomp profiles of its container engine
-(i.e. containerd) by default. Once this happens Crossplane will either need to
-run with the "Unconstrained" seccomp profile, or a variant of the default
-containerd seccomp profile that allows a few extra syscalls (i.e. at least
-`unshare` and `mount`). This can be done by setting a Pod's
-`spec.securityContext.seccompProfile.type` field to `Unconstrained`.
-
-### Packaging Containerized Functions
-
-This document proposes that containerized functions support Crossplane [package
-metadata][package-meta] in the form of a `package.yaml` file at the root of the
-flattened filesystem and/or the OCI layer annotated as `io.crossplane.xpkg:
-base` per the [xpkg spec][xpkg-spec]. This `package.yaml` file would contain a
-custom-resource-like YAML document of type `Function.meta.pkg.crossplane.io`.
-
-Unlike `Configuration` and `Provider` packages, `Function` packages would not
-actually be processed by the Crossplane package manager but rather by the
-Composition (`apiextensions`) machinery. In practice Crossplane would be
-ignorant of the `package.yaml` file; it would exist purely as a way to attach
-"package-like" metadata to containerized Crossplane functions. Therefore, unlike
-the existing package types the `package.yaml` would contain no `spec` section.
-
-An example `package.yaml` might look like:
+P&T Composition as a Function might look like this:
 
 ```yaml
-# Required. Must be as below.
-apiVersion: meta.pkg.crossplane.io/v1alpha1
-# Required. Must be as below.
-kind: Function
-# Required.
+apiVersion: apiextensions.crossplane.io/v1
+kind: Composition
 metadata:
-  # Required. Must comply with Kubernetes API conventions.
-  name: function-example
-  # Optional. Must comply with Kubernetes API conventions.
-  annotations:
-    meta.crossplane.io/source: https://github.com/negz/example-fn
-    meta.crossplane.io/description: An example function
+  name: example
+spec:
+  compositeTypeRef:
+    apiVersion: database.example.org/v1alpha1
+    kind: AcmeCoDatabase
+  pipeline:
+  - step: patch-and-transform
+    functionRef:
+      name: patch-and-transform
+    input:
+      apiVersion: apiextensions.crossplane.io/v1
+      kind: Resources
+      resources:
+      - name: cloudsqlinstance
+        base:
+          apiVersion: database.gcp.crossplane.io/v1beta1
+          kind: CloudSQLInstance
+          spec:
+            forProvider:
+              databaseVersion: POSTGRES_9_6
+              region: us-central1
+              settings:
+                tier: db-custom-1-3840
+                dataDiskType: PD_SSD
+        patches:
+        - type: FromCompositeFieldPath
+          fromFieldPath: spec.parameters.storageGB
+          toFieldPath: spec.forProvider.settings.dataDiskSizeGb
 ```
 
+Removing P&T support from the Composition type would be a breaking API change -
+not something we could do easily. One approach could be to deprecate and
+'freeze' the API, recommending folks who prefer P&T style use the generic P&T
+Function instead.
+
 ## Alternatives Considered
 
-Most of the alternatives considered in this design could also be thought of as
-future considerations. In most cases these alternatives don't make sense at the
-present time but likely will in the future.
-
-### Using Webhooks to Run Functions
-
-Crossplane could invoke functions by calling a webhook rather than running an
-OCI container. In this model function input and output would still take the form
-of a `FunctionIO`, but would be HTTP request and response bodies rather than a
-container's stdin and stdout.
-
-The primary detractor of this approach is the burden it puts on function authors
-and Crossplane operators. Rather than simply publishing an OCI image the author
-and/or Crossplane operator must deploy and operate a web server, ensuring secure
-communication between Crossplane and the webhook endpoint.
-
-Support for `type: Webhook` functions will likely be added shortly after initial
-support for `type: Container` functions is released.
-
-### Using chroots to Run Functions
-
-Crossplane could invoke functions packaged as OCI images by unarchiving them and
-then running them inside a simple `chroot`. This offers more compatibility than
-rootless containers at the expense of isolation - it's not possible to constrain
-a chrooted function's compute resources, network access, etc. `type: Chroot`
-functions would use the same artifacts as `type: Container` functions but invoke
-them differently.
-
-Support for `type: Chroot` functions could be added shortly after initial
-support for `type: Container` functions are released if `type: Container` proves
-to be insufficiently compatible (e.g. for clusters running gVisor, or that
-require seccomp be enabled).
-
-### Using Kubernetes to Run Containerized Functions
-
-Asking Kubernetes to run a container pipeline is less straightforward than you
-might think. Crossplane could schedule a `Pod` for each XR reconcile, or create
-a `CronJob` to do so regularly. Another option could be to connect directly to a
-Kubelet. This approach would enjoy all the advantages of the existing Kubelet
-machinery (pull secrets, caching, etc) but incurs overhead in other areas, for
-example:
-
-* Every reconcile requires a pod to be scheduled, which may potentially block on
-  node scale-up, etc.
-* stdin and stdout must be streamed via the API server, for example by using the
-  [`/attach` subresource][attach].
-* Running containers in order requires either (ab)using init containers or
-  injecting a middleware binary that blocks container starts to ensure they run
-  in order (similar to Argo Workflow's '[emissary]' executor):
-
-> The emissary works by replacing the container's command with its own command.
-> This allows that command to capture stdout, the exit code, and easily
-> terminate your process. The emissary can also delay the start of your process.
-
-You can see some of the many options Argo Workflows explored to address these
-issues before landing on `emissary` in their list of
-[deprecated executors][argo-deprecated-executors].
-
-### Using KRM Function Spec Compliant Functions
-
-While the design proposed by this document is heavily inspired by KRM Functions,
-the [KRM function specification][krm-fn-spec] as it currently exists is not an
-ideal fit. This is because:
-
-1. It is built around the needs of CLI tooling - including several references to
-   (client-side) 'files' that don't exist in the Crossplane context. 
-1. Crossplane needs additional metadata to distinguish which resource in the
-   `ResourceList` is the composite resource and which are the composed
-   resources.
-
-### gVisor
-
-[gVisor][gvisor] supports rootless mode, but requires too many privileges to run
-in a container. A proof-of-concept [exists][gvisor-unpriv] to add an
-`--unprivileged` flag to gVisor, allowing it to run inside a container. It's
-unlikely that gVisor will work in all situations in the near future - for
-example gVisor cannot currently run inside gVisor and support for anything other
-than x86 architectures is experimental.
+See the [alpha design document][alpha-design].
+
+
 
 [term-composition]: https://crossplane.io/docs/v1.9/concepts/terminology.html#composition
 [v0.10.0]: https://github.com/crossplane/crossplane/releases/tag/v0.10.0
@@ -752,27 +1139,29 @@ than x86 architectures is experimental.
 [declarative-app-management]: https://docs.google.com/document/d/1cLPGweVEYrVqQvBLJg6sxV-TrE5Rm2MNOBA_cxZP2WU/edit
 [bcl]: https://twitter.com/bgrant0607/status/1123620689930358786?lang=en
 [terraform-count]: https://www.terraform.io/language/meta-arguments/count
-[turing-complete]: https://en.wikipedia.org/wiki/Turing_completeness#Unintentional_Turing_completeness  
-[pitfalls-dsl]: https://github.com/kubernetes/community/blob/8956bcd54dc6f99bcb681c79a7e5399289e15630/contributors/design-proposals/architecture/declarative-application-management.md#pitfalls-of-configuration-domain-specific-languages-dsls
-[controller-runtime]: https://github.com/kubernetes-sigs/controller-runtime
-[krm-fn-spec]: https://github.com/kubernetes-sigs/kustomize/blob/9d5491/cmd/config/docs/api-conventions/functions-spec.md
-[rawextension]: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#rawextension
-[unix-domain-sockets]: https://man7.org/linux/man-pages/man7/unix.7.html
-[rootless]: https://rootlesscontaine.rs/how-it-works/userns/
-[cgroups-v2-distros]: https://rootlesscontaine.rs/getting-started/common/cgroup2/#checking-whether-cgroup-v2-is-already-enabled
-[overlayfs]: https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html
-[go-containerregistry]: https://github.com/google/go-containerregistry
-[oci-rt-cfg]: https://github.com/opencontainers/runtime-spec/blob/v1.0.2/config.md
-[oci-img-cfg]: https://github.com/opencontainers/image-spec/blob/v1.0.2/config.md
-[gvisor-mountns]: https://github.com/google/gvisor/issues/221
-[kep-seccomp]: https://github.com/kubernetes/enhancements/issues/2413
-[package-meta]: https://github.com/crossplane/crossplane/blob/035e77b/design/one-pager-package-format-v2.md
-[xpkg-spec]: https://github.com/crossplane/crossplane/blob/035e77b/docs/reference/xpkg.md
-[attach]: https://github.com/kubernetes/kubectl/blob/18a531/pkg/cmd/attach/attach.go
-[emissary]: https://github.com/argoproj/argo-workflows/blob/702b293/workflow/executor/emissary/emissary.go#L25
-[argo-deprecated-executors]: https://github.com/argoproj/argo-workflows/blob/v3.4.1/docs/workflow-executors.md
-[krm-fn-spec]: https://github.com/kubernetes-sigs/kustomize/blob/9d5491/cmd/config/docs/api-conventions/functions-spec.md
-[krm-fn-runtimes]: https://github.com/GoogleContainerTools/kpt/issues/2567
-[krm-fn-catalog]: https://catalog.kpt.dev
-[gvisor]: https://gvisor.dev
-[gvisor-unpriv]: https://github.com/google/gvisor/issues/4371#issuecomment-700917549
\ No newline at end of file
+[alpha-design]: defunct/design-doc-composition-functions.md
+[#1972]: https://github.com/crossplane/crossplane/pull/1972
+[#2352]: https://github.com/crossplane/crossplane/pull/2352
+[#4051]: https://github.com/crossplane/crossplane/pull/4051
+[#3917]: https://github.com/crossplane/crossplane/pull/3917
+[#3919]: https://github.com/crossplane/crossplane/pull/3919
+[#3989]: https://github.com/crossplane/crossplane/pull/3989
+[#3498]: https://github.com/crossplane/crossplane/pull/3498
+[#3458]: https://github.com/crossplane/crossplane/pull/3458
+[#3316]: https://github.com/crossplane/crossplane/pull/3316
+[#4036]: https://github.com/crossplane/crossplane/issues/4036
+[#4065]: https://github.com/crossplane/crossplane/issues/4065
+[#4026]: https://github.com/crossplane/crossplane/issues/4026
+[google-protobuf-struct]: https://protobuf.dev/reference/protobuf/google.protobuf/#struct
+[cert-per-entity]: https://github.com/crossplane/crossplane/issues/4305
+[#3884]: https://github.com/crossplane/crossplane/pull/3884
+[go-grpc-concurrency]: https://github.com/grpc/grpc-go/blob/master/Documentation/concurrency.md#servers
+[grpc-benchmarks]: https://grpc.io/docs/guides/benchmarking/
+[functionio-schema]: https://github.com/crossplane/crossplane/blob/v1.12.2/apis/apiextensions/fn/io/v1alpha1/functionio_types.go#L28
+[kubecon-eu-contribfest]: https://github.com/crossplane-contrib/contribfest/blob/main/lab-composition-functions/xfn-many/main.go
+[grpc-supported-languages]: https://grpc.io/docs/languages/
+[starlark]: https://github.com/bazelbuild/starlark/blob/master/spec.md
+[github-script]: https://github.com/actions/github-script
+[controller-runtime-metrics]: https://book.kubebuilder.io/reference/metrics-reference.html
+[event-exporter-metrics]: https://github.com/caicloud/event_exporter
+[controllerconfig-deprecation]: https://github.com/crossplane/crossplane/issues/2468
\ No newline at end of file
diff --git a/design/images/usage-owner-ref.png b/design/images/usage-owner-ref.png
new file mode 100644
index 000000000..14d2be2e3
Binary files /dev/null and b/design/images/usage-owner-ref.png differ
diff --git a/design/one-pager-generic-usage-type.md b/design/one-pager-generic-usage-type.md
new file mode 100644
index 000000000..19d6f2680
--- /dev/null
+++ b/design/one-pager-generic-usage-type.md
@@ -0,0 +1,498 @@
+# Generic `Usage` Type for Deletion Ordering
+
+* Owner: Hasan Türken (@turkenh)
+* Reviewers: @bobh66, @negz
+* Status: Draft
+
+## Background
+
+Crossplane is built on Kubernetes, which leans into eventual consistency. When
+several things need to happen in a particular order the "Kubernetes way" is to
+specify the desired end state and rely on the relevant controllers to ensure the
+system eventually arrives at that state, largely through the use of constant
+reconciliation (aka retries).
+
+Take for example the creation of a VPC Network and a Subnet. The VPC must be
+created and ready before a Subnet can be created within it. In Kubernetes, it’s
+typical to request both resources be created at once (thereby declaring the
+desired state). The creation of the Subnet will simply fail until its
+dependency - the VPC - becomes available.
+
+This loosely coupled, eventually consistent approach is simple and resilient,
+but it can appear chaotic. Operations must be attempted more times than would be
+necessary in a "smarter" system, and often the temporarily failing operations
+will emit warnings that may be confusing or concerning to the uninitiated. Some
+other systems - notably Terraform - avoid this by computing a dependency graph
+before attempting to enact the desired state. This allows the system to avoid
+trying operations it knows will fail due to missing dependencies until those
+dependencies are expected to be satisfied.
+
+A frequent negative side-effect of the eventually consistent approach that we
+see in Crossplane is resources becoming orphaned at delete time because their
+successful deletion depends on the existence of other resources which are often
+deleted before their dependents. It’s possible to author a set of interdependent
+resources that will eventually create successfully, but may not eventually
+delete successfully.
+
+Two variants of this problem are:
+
+- **Orphaned Managed Resources (MRs).** Consider a Helm Release MR that is
+deployed to an EKS Cluster MR. The Helm Release has a dependency on the EKS
+Cluster’s connection details - it uses them as Provider credentials. If the EKS
+Cluster is deleted before the Helm Release the Helm Release controller becomes
+unable to connect to its Provider in order to delete the Release. In practice
+the Release External Resource (ER) was implicitly deleted along with the EKS
+Cluster - there’s actually no work left for the Helm release controller to do -
+but it cannot know that. This results in the Helm Release MR being ‘orphaned’
+in a pending deletion that can never be satisfied.
+
+- **Orphaned ‘Side-Effect’ Resources.** Consider the same example as above, and
+assume the Helm Release includes a Kubernetes Service resource of type:
+LoadBalancer. A side-effect of the creation of this Service will be to create an
+ELB in the EKS cluster’s VPC. This ELB is not managed by or even known to
+Crossplane, and is not deleted along with the EKS cluster. This means that the
+Helm Release (and therefore its Service) must be deleted before the EKS cluster
+in order to ensure the EKS cluster’s controllers trigger deletion of the ELB.
+If it is not, the ELB will be orphaned, costing money and blocking the deletion
+of the VPC it was created in.
+
+These problems are exacerbated by the fact that Kubernetes uses "background
+cascading deletion" by default. Put otherwise, the successful deletion of a
+parent resource is not dependent on the successful deletion of its children.
+Instead, the parent resource will be deleted successfully and the children
+(i.e. resources that declare the parent as an owner reference) will later be
+deleted by the Kubernetes garbage collector. This makes it difficult to diagnose
+or even notice resources that become orphaned when their parent XR is deleted.
+
+## Goals
+
+- Support "cross-provider" dependencies, for example an MR from provider-helm
+depending on a ProviderConfig from provider-aws.
+- Don’t require Crossplane users to manage fine-grained RBAC access.
+- Don’t grant Crossplane providers broad RBAC access (e.g. to all other providers).
+
+## Proposal
+
+We propose introducing a new `Usage` type that can be used to declare usage
+relationships between Crossplane resources. This type will be defined by the
+Core Crossplane repository, and also be available for use as part of a
+composition.
+
+The relations defined by this type will be enforced by an admission webhook that
+will be running as part of the Crossplane core. Similar to the upstreams [liens
+proposal], the webhook will reject deletions of resources that are in use by
+other resources with a ["409 Conflict" error code] as `metav1.Status` as part of
+the admission response (not the actual http code of the admission request).
+
+The RBAC manager already grants Crossplane RBAC access to all types in order for
+it to do Composition. Therefore, there's no need for extra RBAC permissions to
+interact with resources across providers.
+
+### API
+
+We will introduce a new `Usage` type as follows:
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1alpha1
+kind: Usage
+metadata:
+  name: release-uses-cluster
+spec:
+  # Reason is optional when this Usage defines a "Dependency" type relationship,
+  # i.e. when the `spec.by` field is defined.
+  # It is required when the Usage is meant to be used for "Protection" purposes,
+  # i.e. when the `spec.by` is NOT defined.
+  reason: "Release uses Cluster"
+  # Reference to the resource that is being used.
+  of:
+    apiVersion: eks.upbound.io/v1beta1
+    kind: Cluster
+    resourceRef:
+      name: my-cluster
+  # Reference to the resource that is using the other resource.
+  # This field is optional and can be omitted when the Usage is meant to be used
+  # for "Protection" purposes.
+  by:
+    apiVersion: helm.crossplane.io/v1beta1
+    kind: Release
+    resourceRef:
+      name: my-prometheus-chart
+```
+
+The `spec.by` field will define the resource that will be using the resource
+defined in `spec.of`. Both will support only cluster-scoped resources, namely
+`Composites` and `Managed Resources`.
+
+In this example, we define a usage relationship between a Helm `Release` and an
+EKS `Cluster`. The deletion of the `Cluster` resource will be rejected as long
+as this `Release` resource exists.
+
+In addition to direct referencing with names, we will also support selectors to
+match labels or controller reference. This will allow us to define usage
+relationships between resources created by the same composition instance.
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1alpha1
+kind: Usage
+metadata:
+  name: release-uses-cluster
+spec:
+  of:
+    apiVersion: eks.upbound.io/v1beta1
+    kind: Cluster
+    resourceSelector:
+      matchControllerRef: true
+      matchLabels:
+        foo: bar
+  by:
+    apiVersion: helm.crossplane.io/v1beta1
+    kind: Release
+    resourceSelector:
+      matchControllerRef: true
+      matchLabels:
+        baz: qux
+```
+
+Another use case for `Usage` is to protect a resource from being deleted without
+necessarily being used by another resource. For example, a `Usage` that will
+prevent the deletion of a database instance could be defined as follows:
+
+```yaml
+apiVersion: crossplane.io/v1
+kind: Usage
+spec:
+  reason: "Production Database - should never be deleted"
+  of:
+    apiVersion: rds.aws.upbound.io/v1beta1
+    kind: Instance
+    resourceRef:
+      name: my-cluster
+```
+
+### Implementation
+
+The implementation of this feature will be done in two parts:
+
+1. The `Usage` type and its controller, which will manage the lifecycle of the
+   resource.
+2. The admission webhook that will enforce the usage relationships.
+
+Both will be implemented in the Crossplane core repository and would not require
+any changes in the providers.
+
+#### Usage Controller
+
+The usage controller will be responsible for managing the lifecycle of the usage
+relationship by:
+
+- Resolving selectors, if any.
+- Adding owner reference from the using resource (i.e., `spec.by`) to the `Usage`.
+- Adding/removing owner reference from the `Usage` resource to the used resource (i.e., `spec.of`).
+- Preventing deletion of `Usage` before the using resource is deleted.
+- Adding/removing `crossplane.io/in-use: true` label to ensure the `DELETE`
+  request intercepted by matching the selector in admission webhook rule.
+
+**Resolving selectors:**
+
+The API will support defining selectors for both the using and used resource.
+Initially, we will only support resolving references with the following rules:
+
+- Resolution will be made once when the `Usage` is created.
+- If multiple resources match the selector, a random one will be
+  selected.
+
+Once selectors are resolved, the controller will fill `resourceRef.name`.
+
+Example:
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1alpha1
+kind: Usage
+metadata:
+  name: release-uses-clusters
+spec:
+  of:
+    apiVersion: eks.upbound.io/v1beta1
+    kind: Cluster
+    resourceSelector:
+      matchControllerRef: true
+      matchLabels:
+         foo: bar
+  by:
+    apiVersion: helm.crossplane.io/v1beta1
+    kind: Release
+    resourceRef:
+      name: my-prometheus-chart
+```
+
+to:
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1alpha1
+kind: Usage
+metadata:
+  name: release-uses-clusters
+spec:
+  of:
+    apiVersion: eks.upbound.io/v1beta1
+    kind: Cluster
+    resourceRef: # added by the controller based on the selectors
+      name: my-cluster # added by the controller based on the selectors
+    resourceSelector:
+      matchControllerRef: true
+      matchLabels:
+         foo: bar
+  by:
+    apiVersion: helm.crossplane.io/v1beta1
+    kind: Release
+    resourceRef:
+      name: my-prometheus-chart
+```
+
+The [existing policies] for resolving Managed Resource references does not
+necessarily apply to the `Usage`. For example, supporting an `optional`
+reference while defining a usage relationship does not make sense. Also, the
+[reasoning behind] supporting different policies for MRs does not apply to
+the `Usage`. However, depending on the feedback, we may consider supporting
+resolve policies to configure whether the resolution should be made once or
+continuously in a future iteration.
+
+**Owner references:**
+
+We want the `Usage` to be deleted when the using resource is deleted. This will
+be achieved by adding an owner reference from the using resource to the `Usage`.
+
+Another owner reference will be added from the `Usage` to the used resource to
+prevent Garbage Collector from attempting to attempt to delete the used resource
+before the `Usage` is deleted. This is mostly a mitigation for the case where
+deletion process of composites, which rely on GC, could take too long because of
+the exponential backoff in the Garbage Collector as a result of failed DELETE
+API calls.
+
+<div style="text-align: center;"><img src="images/usage-owner-ref.png" height="256"></div>
+
+**Preventing early deletion of the `Usage`:**
+
+Typically, we expect a `Usage` resource to be defined as part of the same
+composition as the using and used resources. This means the `Usage` will get
+the delete request when the composite resource is deleted. In this case, the
+controller will prevent the deletion of the `Usage` until the using resource is
+deleted with the help of a finalizer.
+
+**Adding/removing `crossplane.io/in-use: true` label:**
+
+The admission webhook will intercept the `DELETE` requests for the used
+resources with the following admission webhook rule:
+
+```yaml
+webhooks:
+- name: nousages.apiextensions.crossplane.io
+  objectSelector:
+    matchLabels:
+      crossplane.io/in-use: true
+  rules:
+  - operations: ["DELETE"]
+    apiGroups: ["*"]
+    apiVersions: ["*"]
+    resources: ["*"]
+    scope: "*"
+```
+
+To ensure any delete request is intercepted by our webhook for a resource that
+is in use, the controller will add the `crossplane.io/in-use: true` label to the
+used resource when a `Usage` is created. It is also the controller's job to
+remove this labels when the `Usage` is deleted, and no other `Usage` resources
+reference the same used resource.
+
+#### Admission Webhook
+
+The admission webhook will be responsible for intercepting the `DELETE` requests
+for the used resource(s) and rejecting them with a ["409 Conflict" error code].
+We will fail open in case of any errors from the webhook with
+[`failurePolicy: Fail`].
+
+The webhook will be implemented as a Kubernetes validating admission webhook
+leveraging the [existing webhook machinery] in Crossplane and will be running
+as part of the Crossplane core.
+
+We will use the [Field Indexers] to efficiently query the used resource(s) when
+a `DELETE` request is received.
+
+Implementation of the core logic for the webhook handler:
+
+```go
+func (h *handler) validateNoUsages(ctx context.Context, u *unstructured.Unstructured) admission.Response {
+	usageList := &v1alpha1.UsageList{}
+	if err := h.reader.List(ctx, usageList, client.MatchingFields{inUseIndexKey: getIndexValueForObject(u)}); err != nil {
+		return admission.Errored(http.StatusInternalServerError, err)
+	}
+	if len(usageList.Items) > 0 {
+		return admission.Response{
+			AdmissionResponse: admissionv1.AdmissionResponse{
+				Allowed: false,
+				Result: &metav1.Status{
+					Code:   int32(http.StatusConflict),
+					Reason: metav1.StatusReason(fmt.Sprintf("The resource is used by %d resource(s), including %s/%s", len(usageList.Items), usageList.Items[0].Spec.By.Kind, usageList.Items[0].Spec.By.Name)),
+				},
+			},
+		}
+	}
+	return admission.Allowed("")
+}
+```
+
+### User Experience
+
+#### Defining a Usage Resource as part of a Composition
+
+We expect a typical usage relationship to be defined as part of the same
+composition as the using and used resources. In this case, the user will
+define the `Usage` resource as part of the composition template.
+
+Let's take [upbound/platform-ref-aws] as an example. In this composition, we
+have a [composition for `XCluster`] which composes `XEKS` composing an EKS cluster,
+and `XServices` composing a set of helm releases running on the cluster. In this
+case, we will add the following resource to the same composition to define a
+usage relationship between the `XServices` and the `XEKS` Composites:
+
+```yaml
+    - name: xservices-uses-xeks
+      base:
+        apiVersion: apiextensions.crossplane.io/v1alpha1
+        kind: Usage
+        spec:
+          of:
+            - apiVersion: aws.platformref.upbound.io/v1alpha1
+              kind: XEKS
+              selector:
+                matchControllerRef: true
+          by:
+            apiVersion: aws.platformref.upbound.io/v1alpha1
+            kind: XServices
+            selector:
+              matchControllerRef: true
+```
+
+#### Deleting a Claim or Composite which contains a Usage Relationship
+
+**With the (default) background propagation policy**, we expect no different in user
+experience. Users will be able to delete the Claim or Composite without any error. 
+The Claim or Composite will be deleted immediately, and the Kubernetes garbage
+collector will delete the Composed Resources together with the `Usage` resources
+in the background. It will take longer until all Composed Resources are garbage
+collected, however this will not be visible to the user.
+
+**With the foreground propagation policy**, the only difference we expect is
+that the deletion is taking longer than before. The user will be able to delete
+the Claim or Composite without any error. The deletion of the Claim or Composite
+will be blocked until all Composed Resources are deleted as before. It will take
+longer because deletion of the used resource will be blocked until the used
+resource hence `Usage` is deleted.
+
+#### Directly Deleting a Composite or Managed Resource that is in Use
+
+When trying to delete a Composite or managed resource directly, users will get
+an error as follows:
+
+```bash
+Error from server (The resource is used by Release/my-release): error when deleting "my-cluster.yaml": admission webhook "nousages.apiextensions.crossplane.io" denied the request: The resource is used by Release/my-release
+```
+
+From the error message, it is clear that the resource is in use by a `Release`
+resource. The user can then delete the `Release` resource and then delete the
+Composite or managed resource again.
+
+## Alternatives Considered
+
+### Prior Art
+
+Some alternatives considered in this [previous document], namely:
+
+- Assume Implicit Deletion
+- In-use Liens
+- Composition-internal Dependencies
+- Composition-ordered Dependencies
+- Introduce a Global Dependency Object
+- Introduce a Scoped Dependency Object
+- Model Dependencies in Spec
+- Dependent-managed Finalizers
+
+Please see that document for further details on each of these alternatives.
+
+Among these alternatives, this proposal focuses on the 
+"Introduce a Global Dependency Object" option but with some implementation
+differences which are explained in detail in the following section.
+
+### Alternative Implementations with a Global `Usage` Object 
+
+There are three distinct types of resources that we want to block the deletion
+of when used by other resources:
+
+1. A Managed Resource.
+  - We control the deletion logic; we have the most power.
+  - We can use an annotation or finalizer (or whatever we want) and handle the
+    logic in managed reconciler for all managed resources, e.g., do not delete
+    the external resource if the managed resource is in use.
+2. A Composite Resource with the (default) background deletion policy.
+  - We rely on the garbage collector for deletion. Composed Resources got the
+    deletion request only after the Composite disappeared from the API.
+  - So, we can still rely on a finalizer-based approach to block the deletion
+    of the whole tree of owned (Composed) Resources.
+3. A Composite Resource but with the foreground deletion policy.
+  - We still rely on the garbage collector for deletion; however, Composed
+    Resources got the deletion request immediately after the Composite got the
+    deletion request, no matter whether it has additional finalizers or not.
+  - Finalizer-based approaches wouldn't work out of the box.
+
+So, we have three possible solutions to cover all the scenarios:
+
+**Option A:** Block the delete request with the help of a webhook in any case.
+
+**Option B:** When there is a dependency on a Composite Resource, resolve that
+dependency down to all Composed Resources and only implement deletion prevention
+logic for the Managed Resources.
+
+**Option C:** For Composite resources, do not rely on the garbage collector by
+removing owner references to delete Composed Resources and implement our custom
+deletion logic.
+
+Option C is not ideal because it requires us to implement our custom deletion
+logic for Composites, and will break the tools using owner references
+(e.g. ArgoCD) to track the relationship between Composites and Composed
+Resources.
+
+If we try to make a comparison between the other two approaches:
+
+|  | Option A: Webhook & Reject Delete Requests | Option B: Delay Deletion Until No Usage with Resolved `Usages` |
+|-------------------|--------------------------------------------|----------------------------------------|
+| Implementation Complexity | Easier to implement, fewer complexities to handle during development. | More complex to implement due to the need to _continuously_ resolve dependencies to XRs down to nested XRs and MRs .  |
+| User Experience | Provides immediate feedback, but users need to manually retry deletion after all usages are gone.  | Users do not need to manually retry deletions, which may be smoother, but the delay in deletion might be confusing if not properly communicated. |
+| Alignment with the Kubernetes Philosophy | Aligned with Kubernetes' Liens KEP (still open), which advocates for proactive protection against deletion. | Aligned with Kubernetes' philosophy of eventual consistency and the existing behavior of PV/PVC protection and Crossplane ProviderConfigUsage. |
+| Scalability | Scales better since it will require one `Usage` per XR. | Ends up with multiple `Usages` per XR and can grow quickly when there are nested XRs or multiple dependencies on the same parent. |
+| User Error Protection | The upfront rejection of deletion requests can prevent accidental deletion when there are still usages. | While deletion will eventually occur when all usages are gone, there could be a risk of unintended consequences during the delay period. |
+| Debugging/Troubleshooting | Easier to debug since only `Usage`s created by the user exists, and resources won't have deletion timestamp. | More difficult to debug since further `Usage`s created by the controllers, and it is not easy to identify whether a resource is being deleted or waiting for dependencies to be gone at a given moment.  |
+| Ease of Rollout | Easier, only requires changes in the XP repo. | Requires changes in XP, runtime, and providers needs to be updated. |
+
+Especially considering that there is no clear agreement in upstream, we
+believe _it could be good to choose a practical approach and start with the
+simpler solution, namely the webhook-based solution_. We think it's better to
+start with the simple option, and then, if we find out that it's not enough, we
+can think about changing to a more complex solution. If we check the above table,
+it is clear that the webhook-based solution is the simpler one.
+
+If we decide to change the underlying implementation later, we don't expect
+an API change except getting rejected deletes vs. not. This also provides us a
+good opportunity to test the webhook-based solution in the field and gather
+feedback from the community before committing to a more complex solution.
+
+[liens proposal]: https://github.com/kubernetes/enhancements/pull/2840
+["409 Conflict" error code]: https://datatracker.ietf.org/doc/html/rfc2616#section-10.4.10
+[existing policies]: https://github.com/crossplane/crossplane-runtime/blob/23eaff94e7385121bca832955c8885f925f55ae6/apis/common/v1/resource.go#L80
+[reasoning behind]: https://github.com/crossplane/crossplane-runtime/pull/328
+[webhook rules]: https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-rules
+[`failurePolicy: Fail`]: https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#failure-policy
+[existing webhook machinery]: https://github.com/crossplane/crossplane/pull/2919
+[Field Indexers]: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/client#FieldIndexer
+[previous document]: https://docs.google.com/document/d/1Yu3GxNMJOOMWMjD6gEw5QtT8Fz5qQSyo18ZZFKqcqKw
+[upbound/platform-ref-aws]: https://github.com/upbound/platform-ref-aws/tree/v0.6.0
+[composition for `XCluster`]: https://github.com/upbound/platform-ref-aws/blob/v0.6.0/package/cluster/composition.yaml
\ No newline at end of file
diff --git a/go.mod b/go.mod
index ea2f3bb28..4a9dd46c9 100644
--- a/go.mod
+++ b/go.mod
@@ -7,7 +7,7 @@ require (
 	github.com/AdaLogics/go-fuzz-headers v0.0.0-20230106234847-43070de90fa1
 	github.com/Masterminds/semver v1.5.0
 	github.com/alecthomas/kong v0.8.0
-	github.com/bufbuild/buf v1.23.1
+	github.com/bufbuild/buf v1.24.0
 	github.com/crossplane/crossplane-runtime v0.20.0-rc.0.0.20230622044456-2dfb8bc6bfbc
 	github.com/cyphar/filepath-securejoin v0.2.3
 	github.com/google/go-cmp v0.5.9
@@ -34,6 +34,7 @@ require (
 	sigs.k8s.io/controller-runtime v0.15.0
 	sigs.k8s.io/controller-tools v0.12.0
 	sigs.k8s.io/e2e-framework v0.2.0
+	sigs.k8s.io/kind v0.20.0
 	sigs.k8s.io/yaml v1.3.0
 )
 
@@ -67,8 +68,8 @@ require (
 	github.com/aws/smithy-go v1.13.5 // indirect
 	github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20230510185313-f5e39e5f34c7 // indirect
 	github.com/beorn7/perks v1.0.1 // indirect
-	github.com/bufbuild/connect-go v1.8.0 // indirect
-	github.com/bufbuild/connect-opentelemetry-go v0.3.0 // indirect
+	github.com/bufbuild/connect-go v1.9.0 // indirect
+	github.com/bufbuild/connect-opentelemetry-go v0.4.0 // indirect
 	github.com/bufbuild/protocompile v0.5.1 // indirect
 	github.com/cenkalti/backoff/v3 v3.0.0 // indirect
 	github.com/cespare/xxhash/v2 v2.2.0 // indirect
@@ -78,9 +79,9 @@ require (
 	github.com/dave/jennifer v1.6.0 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/dimchansky/utfbom v1.1.1 // indirect
-	github.com/docker/cli v24.0.2+incompatible // indirect
+	github.com/docker/cli v24.0.4+incompatible // indirect
 	github.com/docker/distribution v2.8.2+incompatible // indirect
-	github.com/docker/docker v24.0.2+incompatible // indirect
+	github.com/docker/docker v24.0.4+incompatible // indirect
 	github.com/docker/docker-credential-helpers v0.7.0 // indirect
 	github.com/docker/go-connections v0.4.0 // indirect
 	github.com/docker/go-units v0.5.0 // indirect
@@ -107,7 +108,7 @@ require (
 	github.com/google/gnostic v0.6.9 // indirect
 	github.com/google/go-containerregistry/pkg/authn/kubernetes v0.0.0-20230516205744-dbecb1de8cfa // indirect
 	github.com/google/gofuzz v1.2.0 // indirect
-	github.com/google/pprof v0.0.0-20230602150820-91b7bce49751 // indirect
+	github.com/google/pprof v0.0.0-20230705174524-200ffdc848b8 // indirect
 	github.com/hashicorp/errwrap v1.1.0 // indirect
 	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
 	github.com/hashicorp/go-hclog v1.0.0 // indirect
@@ -125,7 +126,7 @@ require (
 	github.com/jmespath/go-jmespath v0.4.0 // indirect
 	github.com/josharian/intern v1.0.0 // indirect
 	github.com/json-iterator/go v1.1.12 // indirect
-	github.com/klauspost/compress v1.16.6 // indirect
+	github.com/klauspost/compress v1.16.7 // indirect
 	github.com/klauspost/pgzip v1.2.6 // indirect
 	github.com/mailru/easyjson v0.7.7 // indirect
 	github.com/mattn/go-colorable v0.1.13 // indirect
@@ -140,7 +141,7 @@ require (
 	github.com/morikuni/aec v1.0.0 // indirect
 	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
-	github.com/opencontainers/image-spec v1.1.0-rc3 // indirect
+	github.com/opencontainers/image-spec v1.1.0-rc4 // indirect
 	github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // indirect
 	github.com/pkg/profile v1.7.0 // indirect
 	github.com/prometheus/client_golang v1.15.1 // indirect
@@ -162,14 +163,14 @@ require (
 	go.uber.org/atomic v1.11.0 // indirect
 	go.uber.org/multierr v1.11.0 // indirect
 	go.uber.org/zap v1.24.0 // indirect
-	golang.org/x/crypto v0.10.0 // indirect
-	golang.org/x/mod v0.11.0 // indirect
-	golang.org/x/net v0.11.0 // indirect; indirect // indirect
+	golang.org/x/crypto v0.11.0 // indirect
+	golang.org/x/mod v0.12.0 // indirect
+	golang.org/x/net v0.12.0 // indirect; indirect // indirect
 	golang.org/x/oauth2 v0.8.0 // indirect
-	golang.org/x/term v0.9.0 // indirect
-	golang.org/x/text v0.10.0 // indirect
+	golang.org/x/term v0.10.0 // indirect
+	golang.org/x/text v0.11.0 // indirect
 	golang.org/x/time v0.3.0 // indirect
-	golang.org/x/tools v0.10.0 // indirect
+	golang.org/x/tools v0.11.0 // indirect
 	gomodules.xyz/jsonpatch/v2 v2.3.0 // indirect
 	google.golang.org/appengine v1.6.7 // indirect
 	google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
diff --git a/go.sum b/go.sum
index 47a6cd29a..bc68d756d 100644
--- a/go.sum
+++ b/go.sum
@@ -72,6 +72,7 @@ github.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZ
 github.com/Azure/go-autorest/tracing v0.6.0 h1:TYi4+3m5t6K48TGI9AUdb+IzbnSxvnvUMfuitfgcfuo=
 github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU=
 github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
+github.com/BurntSushi/toml v1.0.0/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
 github.com/BurntSushi/toml v1.2.1/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
 github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
 github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
@@ -83,6 +84,7 @@ github.com/alecthomas/assert/v2 v2.1.0 h1:tbredtNcQnoSd3QBhQWI7QZ3XHOVkw1Moklp2o
 github.com/alecthomas/kong v0.8.0 h1:ryDCzutfIqJPnNn0omnrgHLbAggDQM2VWHikE1xqK7s=
 github.com/alecthomas/kong v0.8.0/go.mod h1:n1iCIO2xS46oE8ZfYCNDqdR0b0wZNrXAIAqro/2132U=
 github.com/alecthomas/repr v0.1.0 h1:ENn2e1+J3k09gyj2shc0dHr/yjaWSHRlrJ4DPMevDqE=
+github.com/alessio/shellescape v1.4.1/go.mod h1:PZAiSCk0LJaZkiCSkPv8qIobYglO3FPpyFjDCtHLS30=
 github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
 github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=
 github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
@@ -121,12 +123,12 @@ github.com/benbjohnson/clock v1.3.5 h1:VvXlSJBzZpA/zum6Sj74hxwYI2DIxRWuNIoXAzHZz
 github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
 github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
 github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=
-github.com/bufbuild/buf v1.23.1 h1:BeMkA3+e9XTFEZPDlAI7cydFKlq24wwYiOHp8CvsRzc=
-github.com/bufbuild/buf v1.23.1/go.mod h1:ERFRzJiIjAOzUSJ3vz1zoI7XfxlBnCwZEyL+NJm4pko=
-github.com/bufbuild/connect-go v1.8.0 h1:srluNkFkZBfSfg9Qb6DrO+5nMaxix//h2ctrHZhMGKc=
-github.com/bufbuild/connect-go v1.8.0/go.mod h1:GmMJYR6orFqD0Y6ZgX8pwQ8j9baizDrIQMm1/a6LnHk=
-github.com/bufbuild/connect-opentelemetry-go v0.3.0 h1:AuZi3asTDKmjGtd2aqpyP4p5QvBFG/YEaHopViLatnk=
-github.com/bufbuild/connect-opentelemetry-go v0.3.0/go.mod h1:r1ppyTtu1EWeRodk4Q/JbyQhIWtO7eR3GoRDzjeEcNU=
+github.com/bufbuild/buf v1.24.0 h1:36rVJMJX7BI9Z6nUPpirF+TUO9tVZ45u5VAfj5E7Bgw=
+github.com/bufbuild/buf v1.24.0/go.mod h1:cacBvncWbYnUcNX580lqllR3qlEetMX/KVm27pUc4Kc=
+github.com/bufbuild/connect-go v1.9.0 h1:JIgAeNuFpo+SUPfU19Yt5TcWlznsN5Bv10/gI/6Pjoc=
+github.com/bufbuild/connect-go v1.9.0/go.mod h1:CAIePUgkDR5pAFaylSMtNK45ANQjp9JvpluG20rhpV8=
+github.com/bufbuild/connect-opentelemetry-go v0.4.0 h1:6JAn10SNqlQ/URhvRNGrIlczKw1wEXknBUUtmWqOiak=
+github.com/bufbuild/connect-opentelemetry-go v0.4.0/go.mod h1:nwPXYoDOoc2DGyKE/6pT1Q9MPSi2Et2e6BieMD0l6WU=
 github.com/bufbuild/protocompile v0.5.1 h1:mixz5lJX4Hiz4FpqFREJHIXLfaLBntfaJv1h+/jS+Qg=
 github.com/bufbuild/protocompile v0.5.1/go.mod h1:G5iLmavmF4NsYtpZFvE3B/zFch2GIY8+wjsYLR/lc40=
 github.com/buger/jsonparser v1.1.1/go.mod h1:6RYKKt7H4d4+iWqouImQ9R2FZql3VbhNgx27UK13J/0=
@@ -148,6 +150,7 @@ github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnht
 github.com/cncf/xds/go v0.0.0-20210312221358-fbca930ec8ed/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
 github.com/containerd/stargz-snapshotter/estargz v0.14.3 h1:OqlDCK3ZVUO6C3B/5FSkDwbkEETK84kQgEeFwDC+62k=
 github.com/containerd/stargz-snapshotter/estargz v0.14.3/go.mod h1:KY//uOCIkSuNAHhJogcZtrNHdKrA99/FCCRjE3HD36o=
+github.com/cpuguy83/go-md2man/v2 v2.0.1/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/cpuguy83/go-md2man/v2 v2.0.2 h1:p1EgwI/C7NhT0JmVkwCD2ZBK8j4aeHQX2pMHHBfMQ6w=
 github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
@@ -171,12 +174,12 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/dimchansky/utfbom v1.1.1 h1:vV6w1AhK4VMnhBno/TPVCoK9U/LP0PkLCS9tbxHdi/U=
 github.com/dimchansky/utfbom v1.1.1/go.mod h1:SxdoEBH5qIqFocHMyGOXVAybYJdr71b1Q/j0mACtrfE=
-github.com/docker/cli v24.0.2+incompatible h1:QdqR7znue1mtkXIJ+ruQMGQhpw2JzMJLRXp6zpzF6tM=
-github.com/docker/cli v24.0.2+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
+github.com/docker/cli v24.0.4+incompatible h1:Y3bYF9ekNTm2VFz5U/0BlMdJy73D+Y1iAAZ8l63Ydzw=
+github.com/docker/cli v24.0.4+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
 github.com/docker/distribution v2.8.2+incompatible h1:T3de5rq0dB1j30rp0sA2rER+m322EBzniBPB6ZIzuh8=
 github.com/docker/distribution v2.8.2+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
-github.com/docker/docker v24.0.2+incompatible h1:eATx+oLz9WdNVkQrr0qjQ8HvRJ4bOOxfzEo8R+dA3cg=
-github.com/docker/docker v24.0.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
+github.com/docker/docker v24.0.4+incompatible h1:s/LVDftw9hjblvqIeTiGYXBCD95nOEEl7qRsRrIOuQI=
+github.com/docker/docker v24.0.4+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
 github.com/docker/docker-credential-helpers v0.7.0 h1:xtCHsjxogADNZcdv1pKUHXryefjlVRqWqIhk/uXJp0A=
 github.com/docker/docker-credential-helpers v0.7.0/go.mod h1:rETQfLdHNT3foU5kuNkFR1R1V12OJRRO5lzt2D1b5X0=
 github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
@@ -313,9 +316,10 @@ github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLe
 github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20201218002935-b9804c9f04c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
 github.com/google/pprof v0.0.0-20211214055906-6f57359322fd/go.mod h1:KgnwoLYCZ8IQu3XUZ8Nc/bM9CCZFOyjUNOSygVozoDg=
-github.com/google/pprof v0.0.0-20230602150820-91b7bce49751 h1:hR7/MlvK23p6+lIw9SN1TigNLn9ZnF3W4SYRKq2gAHs=
-github.com/google/pprof v0.0.0-20230602150820-91b7bce49751/go.mod h1:Jh3hGz2jkYak8qXPD19ryItVnUgpgeqzdkY/D0EaeuA=
+github.com/google/pprof v0.0.0-20230705174524-200ffdc848b8 h1:n6vlPhxsA+BW/XsS5+uqi7GyzaLa5MH7qlSLBZtRdiA=
+github.com/google/pprof v0.0.0-20230705174524-200ffdc848b8/go.mod h1:Jh3hGz2jkYak8qXPD19ryItVnUgpgeqzdkY/D0EaeuA=
 github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
+github.com/google/safetext v0.0.0-20220905092116-b49f7bc46da2/go.mod h1:Tv1PlzqC9t8wNnpPdctvtSUOPUUg4SHeE6vR1Ir2hmg=
 github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
 github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
@@ -359,6 +363,7 @@ github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:
 github.com/ianlancetaylor/demangle v0.0.0-20210905161508-09a460cdf81d/go.mod h1:aYm2/VgdVmcIU8iMfdMvDMsRAQjcfZSKFby6HOFvi/w=
 github.com/imdario/mergo v0.3.16 h1:wwQJbIsHYGMUyLSPrEq1CT16AhnhNJQ51+4fdHUnCl4=
 github.com/imdario/mergo v0.3.16/go.mod h1:WBLT9ZmE3lPoWsEzCh9LPo3TiwVN+ZKEjmz+hD27ysY=
+github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
 github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
 github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
 github.com/jdxcode/netrc v0.0.0-20221124155335-4616370d1a84 h1:2uT3aivO7NVpUPGcQX7RbHijHMyWix/yCnIrCWc+5co=
@@ -379,8 +384,8 @@ github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1
 github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
 github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
-github.com/klauspost/compress v1.16.6 h1:91SKEy4K37vkp255cJ8QesJhjyRO0hn9i9G0GoUwLsk=
-github.com/klauspost/compress v1.16.6/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
+github.com/klauspost/compress v1.16.7 h1:2mk3MPGNzKyxErAw8YaohYh69+pa4sIQSC0fPGCFR9I=
+github.com/klauspost/compress v1.16.7/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
 github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU=
 github.com/klauspost/pgzip v1.2.6/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
 github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
@@ -401,6 +406,7 @@ github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovk
 github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
 github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
 github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84=
+github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
 github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
 github.com/mattn/go-isatty v0.0.17 h1:BTarxUcIeDqL27Mc+vyvdWYSL28zpIhv3RoTdsLMPng=
 github.com/mattn/go-isatty v0.0.17/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
@@ -433,10 +439,11 @@ github.com/onsi/ginkgo/v2 v2.9.5 h1:+6Hr4uxzP4XIUyAkg61dWBw8lb/gc4/X5luuxN/EC+Q=
 github.com/onsi/gomega v1.27.7 h1:fVih9JD6ogIiHUN6ePK7HJidyEDpWGVB5mzM7cWNXoU=
 github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
 github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
-github.com/opencontainers/image-spec v1.1.0-rc3 h1:fzg1mXZFj8YdPeNkRXMg+zb88BFV0Ys52cJydRwBkb8=
-github.com/opencontainers/image-spec v1.1.0-rc3/go.mod h1:X4pATf0uXsnn3g5aiGIsVnJBR4mxhKzfwmvK/B2NTm8=
+github.com/opencontainers/image-spec v1.1.0-rc4 h1:oOxKUJWnFC4YGHCCMNql1x4YaDfYBTS5Y4x/Cgeo1E0=
+github.com/opencontainers/image-spec v1.1.0-rc4/go.mod h1:X4pATf0uXsnn3g5aiGIsVnJBR4mxhKzfwmvK/B2NTm8=
 github.com/opencontainers/runtime-spec v1.1.0-rc.3.0.20230610073135-48415de180cf h1:AGnwZS8lmjGxN2/XlzORiYESAk7HOlE3XI37uhIP9Vw=
 github.com/opencontainers/runtime-spec v1.1.0-rc.3.0.20230610073135-48415de180cf/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
+github.com/pelletier/go-toml v1.9.4/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c=
 github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 h1:KoWmjvw+nsYOo29YJK9vDA65RGE3NrOnUtO7a+RF9HU=
 github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8/go.mod h1:HKlIX3XHQyzLZPlr7++PzdhaXEj94dEiJgZDTsxEqUI=
 github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -473,6 +480,7 @@ github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVs
 github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
 github.com/spf13/afero v1.9.5 h1:stMpOSZFs//0Lv29HduCmli3GUfpFoF3Y1Q/aXj/wVM=
 github.com/spf13/afero v1.9.5/go.mod h1:UBogFpq8E9Hx+xc5CNTTEpTnuHVmXDwZcZcE1eb/UhQ=
+github.com/spf13/cobra v1.4.0/go.mod h1:Wo4iy3BUC+X2Fybo0PDqwJIv3dNRiZLHQymsfxlB84g=
 github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I=
 github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0=
 github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
@@ -546,8 +554,8 @@ golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5y
 golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
 golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
 golang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58=
-golang.org/x/crypto v0.10.0 h1:LKqV2xt9+kDzSTfOhx4FrkEBcMrAgHSYgzywV9zcGmM=
-golang.org/x/crypto v0.10.0/go.mod h1:o4eNf7Ede1fv+hwOwZsTHl9EsPFO6q6ZvYR8vYfY45I=
+golang.org/x/crypto v0.11.0 h1:6Ewdq3tDic1mg5xRO4milcWCfMVQhI4NkqWWvqejpuA=
+golang.org/x/crypto v0.11.0/go.mod h1:xgJhtzW8F9jGdVFWZESrid1U1bjeNy4zgy5cRr/CIio=
 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
 golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -585,8 +593,8 @@ golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.5.1/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
 golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
 golang.org/x/mod v0.7.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
-golang.org/x/mod v0.11.0 h1:bUO06HqtnRcc/7l71XBe4WcqTZ+3AH1J59zWDDwLKgU=
-golang.org/x/mod v0.11.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
+golang.org/x/mod v0.12.0 h1:rmsUpXtvNzj340zd98LZ4KntptpfRHwpFOHG188oHXc=
+golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
 golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -625,8 +633,8 @@ golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qx
 golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
 golang.org/x/net v0.5.0/go.mod h1:DivGGAXEgPSlEBzxGzZI+ZLohi+xUj054jfeKui00ws=
 golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
-golang.org/x/net v0.11.0 h1:Gi2tvZIJyBtO9SDr1q9h5hEQCp/4L2RQ+ar0qjx2oNU=
-golang.org/x/net v0.11.0/go.mod h1:2L/ixqYpgIVXmeoSA/4Lu7BzTG4KIyPIryS4IsOd1oQ=
+golang.org/x/net v0.12.0 h1:cfawfvKITfUsFCeJIHJrbSxpeu/E81khclypR0GVT50=
+golang.org/x/net v0.12.0/go.mod h1:zEVYFnQC7m/vmpQFELhcD1EWkZlX69l4oqgmer6hfKA=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -695,6 +703,7 @@ golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBc
 golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210616045830-e2b7044e8c71/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210819135213-f52c844e1c1c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211007075335-d3039528d8ac/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -713,8 +722,8 @@ golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9sn
 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/term v0.4.0/go.mod h1:9P2UbLfCdcvo3p/nzKvsmas4TnlujnuoV9hGgYzW1lQ=
 golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
-golang.org/x/term v0.9.0 h1:GRRCnKYhdQrD8kfRAdQ6Zcw1P0OcELxGLKJvtjVMZ28=
-golang.org/x/term v0.9.0/go.mod h1:M6DEAAIenWoTxdKrOltXcmDY3rSplQUkrvaDU5FcQyo=
+golang.org/x/term v0.10.0 h1:3R7pNqamzBraeqj/Tj8qt1aQ2HpmlC+Cx/qL/7hn4/c=
+golang.org/x/term v0.10.0/go.mod h1:lpqdcUyK/oCiQxvxVrppt5ggO2KCZ5QblwqPnfZ6d5o=
 golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -726,8 +735,8 @@ golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
 golang.org/x/text v0.6.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
 golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
-golang.org/x/text v0.10.0 h1:UpjohKhiEgNc0CSauXmwYftY1+LlaC75SJwh0SgCX58=
-golang.org/x/text v0.10.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
+golang.org/x/text v0.11.0 h1:LAntKIrcmeSKERyiOh0XMV39LXS8IE9UL2yP7+f5ij4=
+golang.org/x/text v0.11.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
 golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -787,8 +796,8 @@ golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
 golang.org/x/tools v0.1.8/go.mod h1:nABZi5QlRsZVlzPpHl034qft6wpY4eDcsTt5AaioBiU=
 golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
 golang.org/x/tools v0.5.0/go.mod h1:N+Kgy78s5I24c24dU8OfWNEotWjutIs8SnJvn5IDq+k=
-golang.org/x/tools v0.10.0 h1:tvDr/iQoUqNdohiYm0LmmKcBk+q86lb9EprIUFhHHGg=
-golang.org/x/tools v0.10.0/go.mod h1:UJwyiVBsOA2uwvK/e5OY3GTpDUJriEd+/YlqAwLPmyM=
+golang.org/x/tools v0.11.0 h1:EMCa6U9S2LtZXLAMoWiR/R8dAQFRqbAitmbJ2UKhoi8=
+golang.org/x/tools v0.11.0/go.mod h1:anzJrxPjNtfgiYQYirP2CPGzGLxrH2u2QBhn6Bf3qY8=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -963,6 +972,8 @@ sigs.k8s.io/e2e-framework v0.2.0 h1:gD6AWWAHFcHibI69E9TgkNFhh0mVwWtRCHy2RU057jQ=
 sigs.k8s.io/e2e-framework v0.2.0/go.mod h1:E6JXj/V4PIlb95jsn2WrNKG+Shb45xaaI7C0+BH4PL8=
 sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo=
 sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0=
+sigs.k8s.io/kind v0.20.0 h1:f0sc3v9mQbGnjBUaqSFST1dwIuiikKVGgoTwpoP33a8=
+sigs.k8s.io/kind v0.20.0/go.mod h1:aBlbxg08cauDgZ612shr017/rZwqd7AS563FvpWKPVs=
 sigs.k8s.io/structured-merge-diff/v4 v4.2.3 h1:PRbqxJClWWYMNV1dhaG4NsibJbArud9kFxnAMREiWFE=
 sigs.k8s.io/structured-merge-diff/v4 v4.2.3/go.mod h1:qjx8mGObPmV2aSZepjQjbmb2ihdVs8cGKBraizNC69E=
 sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=



Test patch:
diff --git a/test/e2e/funcs/env.go b/test/e2e/funcs/env.go
index 78330a5a9..363f4199f 100644
--- a/test/e2e/funcs/env.go
+++ b/test/e2e/funcs/env.go
@@ -18,12 +18,19 @@ package funcs
 
 import (
 	"context"
+	"fmt"
+	"os"
+	"path/filepath"
 	"testing"
 
+	corev1 "k8s.io/api/core/v1"
 	"sigs.k8s.io/e2e-framework/pkg/env"
 	"sigs.k8s.io/e2e-framework/pkg/envconf"
+	"sigs.k8s.io/e2e-framework/pkg/envfuncs"
 	"sigs.k8s.io/e2e-framework/pkg/features"
 	"sigs.k8s.io/e2e-framework/third_party/helm"
+	"sigs.k8s.io/kind/pkg/apis/config/v1alpha4"
+	"sigs.k8s.io/yaml"
 
 	"github.com/crossplane/crossplane-runtime/pkg/errors"
 
@@ -32,6 +39,8 @@ import (
 	secretsv1alpha1 "github.com/crossplane/crossplane/apis/secrets/v1alpha1"
 )
 
+type kindConfigContextKey string
+
 // HelmRepo manages a Helm repo.
 func HelmRepo(o ...helm.Option) env.Func {
 	return func(ctx context.Context, c *envconf.Config) (context.Context, error) {
@@ -102,3 +111,104 @@ func EnvFuncs(fns ...env.Func) env.Func {
 		return ctx, nil
 	}
 }
+
+// CreateKindClusterWithConfig create kind cluster of the given name according to
+// configuration referred via configFilePath.
+// The configuration is placed in test context afterward
+func CreateKindClusterWithConfig(clusterName, configFilePath string) env.Func {
+	return EnvFuncs(
+		envfuncs.CreateKindClusterWithConfig(clusterName, "\"\"", configFilePath),
+		func(ctx context.Context, config *envconf.Config) (context.Context, error) {
+			b, err := os.ReadFile(filepath.Clean(configFilePath))
+			if err != nil {
+				return ctx, err
+			}
+			cfg := &v1alpha4.Cluster{}
+			err = yaml.Unmarshal(b, cfg)
+			if err != nil {
+				return ctx, err
+			}
+			return context.WithValue(ctx, kindConfigContextKey(clusterName), cfg), nil
+		},
+	)
+}
+
+// ServiceIngressEndPoint returns endpoint (addr:port) that can be used for accessing
+// the service in the cluster with the given name.
+func ServiceIngressEndPoint(ctx context.Context, cfg *envconf.Config, clusterName, namespace, serviceName string) (string, error) {
+	_, found := envfuncs.GetKindClusterFromContext(ctx, clusterName)
+	client := cfg.Client()
+	service := &corev1.Service{}
+	err := client.Resources().Get(ctx, serviceName, namespace, service)
+	if err != nil {
+		return "", err
+	}
+
+	var nodePort int32
+	for _, p := range service.Spec.Ports {
+		if p.NodePort != 0 {
+			nodePort = p.NodePort
+			break
+		}
+	}
+	if nodePort == 0 {
+		return "", errors.Errorf("No nodePort found for service %s/%s at cluster %s", namespace, serviceName, clusterName)
+	}
+	if found {
+		kindCfg, err := kindConfig(ctx, clusterName)
+		if err != nil {
+			return "", err
+		}
+		hostPort, err := findHostPort(kindCfg, nodePort)
+		if err != nil {
+			return "", err
+		}
+		return fmt.Sprintf("localhost:%v", hostPort), nil
+	}
+	nodes := &corev1.NodeList{}
+	if err := client.Resources().List(ctx, nodes); err != nil {
+		return "", errors.Wrap(err, "cannot list nodes")
+	}
+	addr, err := findAnyNodeIPAddress(nodes)
+	if err != nil {
+		return "", err
+	}
+	return fmt.Sprintf("%s:%v", addr, nodePort), nil
+}
+
+func kindConfig(ctx context.Context, clusterName string) (*v1alpha4.Cluster, error) {
+	v := ctx.Value(kindConfigContextKey(clusterName))
+	if v == nil {
+		return nil, errors.Errorf("No kind config found in context for cluster %s", clusterName)
+	}
+	kindCfg, ok := v.(*v1alpha4.Cluster)
+	if !ok {
+		return nil, errors.Errorf("kind config is not of type v1alpha4.Cluster for clustername %s", clusterName)
+	}
+	return kindCfg, nil
+}
+
+func findAnyNodeIPAddress(nodes *corev1.NodeList) (string, error) {
+	if len(nodes.Items) == 0 {
+		return "", errors.New("no nodes in the cluster")
+	}
+	for _, a := range nodes.Items[0].Status.Addresses {
+		if a.Type == corev1.NodeInternalIP {
+			return a.Address, nil
+		}
+	}
+	return "", errors.Errorf("no ip address found for nodes: %v", nodes)
+}
+
+func findHostPort(kindCfg *v1alpha4.Cluster, containerPort int32) (int32, error) {
+	for _, n := range kindCfg.Nodes {
+		if n.Role == v1alpha4.ControlPlaneRole {
+			for _, pm := range n.ExtraPortMappings {
+				if pm.ContainerPort == containerPort {
+					return pm.HostPort, nil
+				}
+			}
+		}
+	}
+	return 0, errors.Errorf("No host port found in kind config for container port: %v", containerPort)
+}
diff --git a/test/e2e/main_test.go b/test/e2e/main_test.go
index 6cb8a456a..a9464975f 100644
--- a/test/e2e/main_test.go
+++ b/test/e2e/main_test.go
@@ -19,6 +19,7 @@ package e2e
 import (
 	"flag"
 	"os"
+	"path/filepath"
 	"strings"
 	"testing"
 
@@ -116,8 +117,11 @@ func HelmOptions(extra ...helm.Option) []helm.Option {
 	return append(o, extra...)
 }
 
-// The test environment, shared by all E2E test functions.
-var environment env.Environment
+var (
+	// The test environment, shared by all E2E test functions.
+	environment env.Environment
+	clusterName string
+)
 
 func TestMain(m *testing.M) {
 	// TODO(negz): Global loggers are dumb and klog is dumb. Remove this when
@@ -136,7 +140,7 @@ func TestMain(m *testing.M) {
 
 	cfg, _ := envconf.NewFromFlags()
 
-	clusterName := envconf.RandomName("crossplane-e2e", 32)
+	clusterName = envconf.RandomName("crossplane-e2e", 32)
 	if *kindClusterName != "" {
 		clusterName = *kindClusterName
 	}
@@ -144,8 +148,13 @@ func TestMain(m *testing.M) {
 	// we want to create the cluster if it doesn't exist, but only if we're
 	isKindCluster := *create || *kindClusterName != ""
 	if isKindCluster {
+		kindCfg, err := filepath.Abs(filepath.Join("test", "e2e", "testdata", "kindConfig.yaml"))
+		if err != nil {
+			log.Log.Error(err, "error getting kind config file")
+			os.Exit(1)
+		}
 		setup = []env.Func{
-			envfuncs.CreateKindCluster(clusterName),
+			funcs.CreateKindClusterWithConfig(clusterName, kindCfg),
 		}
 	} else {
 		cfg.WithKubeconfigFile(conf.ResolveKubeConfigFile())
diff --git a/test/e2e/testdata/images/labelizer/Dockerfile b/test/e2e/testdata/images/labelizer/Dockerfile
index a399088f0..18ca61c70 100644
--- a/test/e2e/testdata/images/labelizer/Dockerfile
+++ b/test/e2e/testdata/images/labelizer/Dockerfile
@@ -1,4 +1,4 @@
-FROM mikefarah/yq:4.34.1
+FROM mikefarah/yq:4.34.2
 
 COPY labelizer.sh /bin
 USER root
diff --git a/test/e2e/testdata/kindConfig.yaml b/test/e2e/testdata/kindConfig.yaml
new file mode 100644
index 000000000..90ee5b5e5
--- /dev/null
+++ b/test/e2e/testdata/kindConfig.yaml
@@ -0,0 +1,8 @@
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+nodes:
+- role: control-plane
+  extraPortMappings:
+  # expose NodePort 32000 to the host
+  - containerPort: 32000
+    hostPort: 3000
\ No newline at end of file
diff --git a/test/e2e/xfn_test.go b/test/e2e/xfn_test.go
index b847a35c5..5f5984c53 100644
--- a/test/e2e/xfn_test.go
+++ b/test/e2e/xfn_test.go
@@ -106,24 +106,11 @@ func TestXfnRunnerImagePull(t *testing.T) {
 						))),
 			).
 			WithSetup("CopyFnImageToRegistry", func(ctx context.Context, t *testing.T, config *envconf.Config) context.Context {
-				nodes := &corev1.NodeList{}
-				if err := config.Client().Resources().List(ctx, nodes); err != nil {
-					t.Fatal("cannot list nodes", err)
-				}
-				if len(nodes.Items) == 0 {
-					t.Fatalf("no nodes in the cluster")
-				}
-				var addr string
-				for _, a := range nodes.Items[0].Status.Addresses {
-					if a.Type == corev1.NodeInternalIP {
-						addr = a.Address
-						break
-					}
-				}
-				if addr == "" {
-					t.Fatalf("no nodes with private address")
+				reg, err := funcs.ServiceIngressEndPoint(ctx, config, clusterName, "reg", "private-docker-registry")
+				if err != nil {
+					t.Fatal(err)
 				}
-
+				t.Logf("registry endpoint %s", reg)
 				srcRef, err := name.ParseReference("crossplane-e2e/fn-labelizer:latest")
 				if err != nil {
 					t.Fatal(err)
@@ -133,7 +120,7 @@ func TestXfnRunnerImagePull(t *testing.T) {
 					t.Fatal(err)
 				}
 				err = wait.For(func() (done bool, err error) {
-					err = crane.Push(src, fmt.Sprintf("%s:32000/fn-labelizer:latest", addr), crane.Insecure)
+					err = crane.Push(src, fmt.Sprintf("%s/fn-labelizer:latest", reg), crane.Insecure)
 					if err != nil {
 						return false, nil //nolint:nilerr // we want to retry and to throw error
 					}