Project: Rook (Runtime / Cloud Native Storage)
File: ceph-object-bucket-notifications.md

title: Object Bucket Notifications Rook supports the creation of bucket notifications via two custom resources: a `CephBucketNotification` is a custom resource the defines: topic, events and filters of a bucket notification, and is described by a Custom Resource Definition (CRD) shown below. Bucket notifications are associated with a bucket by setting labels on the Object Bucket claim (OBC). See the Ceph documentation for detailed information: . a `CephBucketTopic` is custom resource which represents a bucket notification topic and is described by a CRD shown below. A bucket notification topic represents an endpoint (or a "topic" inside this endpoint) to which bucket notifications could be sent. A CephBucketNotification defines what bucket actions trigger the notification and which topic to send notifications to. A CephBucketNotification may also define a filter, based on the object's name and other object attributes. Notifications can be associated with buckets created via ObjectBucketClaims by adding labels to an ObjectBucketClaim with the following format: ```yaml bucket-notification-<notification name>: <notification name> ``` The CephBucketTopic, CephBucketNotification and ObjectBucketClaim must all belong to the same namespace. If a bucket was created manually (not via an ObjectBucketClaim), notifications on this bucket should also be created manually. However, topics in these notifications may reference topics that were created via CephBucketTopic resources. A CephBucketTopic represents an endpoint (of types: Kafka, AMQP0.9.1 or HTTP), or a specific resource inside this endpoint (e.g a Kafka or an AMQP topic, or a specific URI in an HTTP server). The CephBucketTopic also holds any additional info needed for a CephObjectStore's RADOS Gateways (RGW) to connect to the endpoint. Topics don't belong to a specific bucket or notification. Notifications from multiple buckets may be sent to the same topic, and one bucket (via multiple CephBucketNotifications) may send notifications to multiple topics. Notifications may be sent synchronously, as part of the operation that triggered them. In this mode, the operation is acknowledged only after the notification is sent to the topics configured endpoint, which means that the round trip time of the notification is added to the latency of the operation itself. The original triggering operation will still be considered as successful even if the notification fail with an error, cannot be delivered or times out. Notifications may also be sent asynchronously. They will be committed into persistent storage and then asynchronously sent to the topics configured endpoint. In this case, the only latency added to the original operation is of committing the notification to persistent storage. If the notification fail with an error, cannot be delivered or times out, it will be retried until successfully acknowledged. ```yaml apiVersion:

kind: CephBucketTopic metadata: name: my-topic [1] namespace: my-app-space [2] spec: objectStoreName: my-store [3] objectStoreNamespace: rook-ceph [4] opaqueData: my@email.com [5] persistent: false [6] endpoint: [7] http: [8] uri: http://my-notification-endpoint:8080 disableVerifySSL: true [9] sendCloudEvents: false [10] ``` `name` of the `CephBucketTopic` In case of AMQP endpoint, the name is used for the AMQP topic (routing key for a topic exchange) In case of Kafka endpoint, the name is used as the Kafka topic `namespace`(optional) of the `CephBucketTopic`. Should match the namespace of the CephBucketNotification associated with this CephBucketTopic, and the OBC with the label referencing the CephBucketNotification `objectStoreName` is the name of the object store in which the topic should be created. This must be the same object store used for the buckets associated with the notifications referencing this topic. `objectStoreNamespace` is the namespace of the object store in which the topic should be created `opaqueData` (optional) is added to all notifications triggered by a notifications associated with the topic `persistent` (optional) indicates whether notifications to this endpoint are persistent (=asynchronous) or sent synchronously (false by default) `endpoint` to which to send the notifications to. Exactly one of the endpoints must be defined: `http`, `amqp`, `kafka` `http` (optional) hold the spec for an HTTP endpoint. The format of the URI would be: `http` port defaults to: 80/443 for HTTP/S accordingly `disableVerifySSL` indicates whether the RGW is going to verify the SSL certificate of the HTTP server in case HTTPS is used ("false" by default) `sendCloudEvents`: (optional) send the notifications with the . Supported for Ceph Quincy (v17) or newer ("false" by default) `amqp` (optional) hold the spec for an AMQP endpoint. The format of the URI would be: `amqp` port defaults to: 5672/5671 for AMQP/S accordingly user/password defaults to: guest/guest user/password may only be provided if HTTPS is used with the RGW. If not, topic creation request will be rejected vhost defaults to: / `disableVerifySSL` (optional) indicates whether the RGW is going to verify the SSL certificate of the AMQP server in case AMQPS is used ("false" by default) `ackLevel` (optional) indicates what kind of ack the RGW is waiting for after sending the notifications: none: message is considered delivered if sent to broker broker: message is considered delivered if acked by broker (default) routable: message is considered delivered if broker can route to a consumer `exchange` in the AMQP broker that would route the notifications. Different topics pointing to the same endpoint must use the same exchange `kafka` (optional) hold the spec for a Kafka endpoint. The format of the URI would be: `kafka://[<user>:<password>@]<fqdn>[:<port]` port defaults to: 9092 user/password may only be provided if HTTPS is used with the

If not, topic creation request will be rejected user/password may only be provided together with `useSSL`, if not, the connection to the broker would fail `disableVerifySSL` (optional) indicates whether the RGW is going to verify the SSL certificate of the Kafka server in case `useSSL` flag is used ("false" by default) `ackLevel` (optional) indicates what kind of ack the RGW is waiting for after sending the notifications: none: message is considered delivered if sent to broker broker: message is considered delivered if acked by broker (default) `useSSL` (optional) indicates that secure connection will be used for connecting with the broker (false by default) !!! note In case of Kafka and AMQP, the consumer of the notifications is not required to ack the notifications, since the broker persists the messages before delivering them to their final destinations. ```yaml apiVersion: ceph.rook.io/v1 kind: CephBucketNotification metadata: name: my-notification [1] namespace: my-app-space [2] spec: topic: my-topic [3] filter: [4] keyFilters: [5] name: prefix value: hello name: suffix value: .png name: regex value: "[a-z]\\." metadataFilters: [6] name: x-amz-meta-color value: blue name: x-amz-meta-user-type value: free tagFilters: [7] name: project value: brown events: [8] s3:ObjectCreated:Put s3:ObjectCreated:Copy ``` `name` of the `CephBucketNotification` `namespace`(optional) of the `CephBucketNotification`. Should match the namespace of the CephBucketTopic referenced in [3], and the OBC with the label referencing the CephBucketNotification `topic` to which the notifications should be sent `filter` (optional) holds a list of filtering rules of different types. Only objects that match all the filters will trigger notification sending `keyFilter` (optional) are filters based on the object key. There could be up to 3 key filters defined: `prefix`, `suffix` and `regex` `metadataFilters` (optional) are filters based on the object metadata. All metadata fields defined as filters must exists in the object, with the values defined in the filter. Other metadata fields may exist in the object `tagFilters` (optional) are filters based on object tags. All tags defined as filters must exists in the object, with the values defined in the filter. Other tags may exist in the object `events` (optional) is a list of events that should trigger the notifications. By default all events should trigger notifications. Valid Events are: s3:ObjectCreated: s3:ObjectCreated:Put s3:ObjectCreated:Post s3:ObjectCreated:Copy s3:ObjectCreated:CompleteMultipartUpload s3:ObjectRemoved: s3:ObjectRemoved:Delete s3:ObjectRemoved:DeleteMarkerCreated For a notifications to be associated with a bucket, a labels must be added to the OBC, indicating the name of the notification. To delete a notification from a bucket the matching label must be removed. When an OBC is deleted, all of the notifications associated with the bucket will be deleted as well. ```yaml apiVersion: objectbucket.io/v1alpha1 kind: ObjectBucketClaim metadata: name: ceph-notification-bucket labels: some-label: some-value bucket-notification-my-notification: my-notification bucket-notification-another-notification: another-notification spec: generateBucketName: ceph-bkt storageClassName: rook-ceph-delete-bucket ```