Project: Grafana Loki (Observability and Analysis / Observability)
File: object_storage.md

title: "Object Storage" description: "Setup for storing logs to Object Storage" lead: "" date: 2022-06-21T08:48:45+00:00 lastmod: 2022-06-21T08:48:45+00:00 draft: false images: [] menu: docs: parent: "lokistack" weight: 100 toc: true Loki Operator supports , , , , and for LokiStack object storage. Note: Upon setting up LokiStack for any object storage provider, you should configure a logging collector that references the LokiStack in order to view the logs. Create a on AWS. Deploy the Loki Operator to your cluster. Create an Object Storage secret with keys as follows: ```console kubectl create secret generic lokistack-dev-s3 \ --from-literal=bucketnames="<BUCKET_NAME>" \ --from-literal=endpoint="<AWSBUCKETENDPOINT>" \ --from-literal=accesskeyid="<AWSACCESSKEY_ID>" \ --from-literal=accesskeysecret="<AWSACCESSKEY_SECRET>" \ --from-literal=region="<AWSREGIONYOURBUCKETLIVES_IN>" ``` or with `SSE-KMS` encryption ```console kubectl create secret generic lokistack-dev-s3 \ --from-literal=bucketnames="<BUCKET_NAME>" \ --from-literal=endpoint="<AWSBUCKETENDPOINT>" \ --from-literal=accesskeyid="<AWSACCESSKEY_ID>" \ --from-literal=accesskeysecret="<AWSACCESSKEY_SECRET>" \ --from-literal=sse_type="SSE-KMS" \ --from-literal=ssekmskeyid="<AWSSSEKMSKEY_ID>" \ --from-literal=ssekmsencryptioncontext="<OPTIONALAWSSSEKMSENCRYPTIONCONTEXT_JSON>" ``` See also official docs on and (Note: Only content without newlines allowed, because it is exposed via environment variable to the containers). or with `SSE-S3` encryption ```console kubectl create secret generic lokistack-dev-s3 \ --from-literal=bucketnames="<BUCKET_NAME>" \ --from-literal=endpoint="<AWSBUCKETENDPOINT>" \ --from-literal=accesskeyid="<AWSACCESSKEY_ID>" \ --from-literal=accesskeysecret="<AWSACCESSKEY_SECRET>" \ --from-literal=sse_type="SSE-S3" ``` where `lokistack-dev-s3` is the secret name. Create an instance of by referencing the secret name and type as `s3`: ```yaml spec: storage: secret: name: lokistack-dev-s3 type: s3 ``` Create a on Azure. Deploy the Loki Operator to your cluster. Create an Object Storage secret with keys as follows: ```console kubectl create secret generic lokistack-dev-azure \ --from-literal=container="<AZURECONTAINERNAME>" \ --from-literal=environment="<AZURE_ENVIRONMENTs>" \ --from-literal=accountname="<AZUREACCOUNT_NAME>" \ --from-literal=accountkey="<AZUREACCOUNT_KEY>" \ --from-literal=endpointsuffix="<OPTIONALAZUREENDPOINTSUFFIX>" ``` where `lokistack-dev-azure` is the secret name. Create an instance of by referencing the secret name and type as `azure`: ```yaml spec: storage: secret: name: lokistack-dev-azure type: azure ``` Create a on Google Cloud Platform. Create a under same project. Create a under same project for GCP authentication. Deploy the Loki Operator to your cluster. Copy the service account credentials received from GCP into a file name `key.json`. Create an Object Storage secret with keys `bucketname` and `key.json` as follows: ```console kubectl create secret generic lokistack-dev-gcs \ --from-literal=bucketname="<BUCKET_NAME>" \ --from-file=key.json="<PATH/TO/KEY.JSON>" ``` where `lokistack-dev-gcs` is the secret name, `<BUCKET_NAME>` is the name of bucket created in requirements step and `<PATH/TO/KEY.JSON>` is the file path where the `key.json` was copied to. Create an instance of by referencing the secret name and type as `gcs`: ```yaml spec: storage: secret: name: lokistack-dev-gcs type: gcs ``` Deploy Minio on your Cluster, e.g. using the Create a on Minio via CLI. Deploy the Loki Operator to your

Create an Object Storage secret with keys as follows: ```console kubectl create secret generic lokistack-dev-minio \ --from-literal=bucketnames="<BUCKET_NAME>" \ --from-literal=endpoint="<MINIOBUCKETENDPOINT>" \ --from-literal=accesskeyid="<MINIOACCESSKEY_ID>" \ --from-literal=accesskeysecret="<MINIOACCESSKEY_SECRET>" ``` where `lokistack-dev-minio` is the secret name. Create an instance of by referencing the secret name and type as `s3`: ```yaml spec: storage: secret: name: lokistack-dev-minio type: s3 ``` Deploy the on your cluster. Deploy the Loki Operator to your cluster. Create an ObjectBucketClaim in `openshift-logging` namespace: ```yaml apiVersion: objectbucket.io/v1alpha1 kind: ObjectBucketClaim metadata: name: loki-bucket-odf namespace: openshift-logging spec: generateBucketName: loki-bucket-odf ``` Get bucket properties from the associated ConfigMap: ```console BUCKETHOST=$(kubectl get -n openshift-logging configmap loki-bucket-odf -o jsonpath='{.data.BUCKETHOST}') BUCKETNAME=$(kubectl get -n openshift-logging configmap loki-bucket-odf -o jsonpath='{.data.BUCKETNAME}') BUCKETPORT=$(kubectl get -n openshift-logging configmap loki-bucket-odf -o jsonpath='{.data.BUCKETPORT}') ``` Get bucket access key from the associated Secret: ```console ACCESSKEYID=$(kubectl get -n openshift-logging secret loki-bucket-odf -o jsonpath='{.data.AWSACCESSKEY_ID}' | base64 -d) SECRETACCESSKEY=$(kubectl get -n openshift-logging secret loki-bucket-odf -o jsonpath='{.data.AWSSECRETACCESS_KEY}' | base64 -d) ``` Create an Object Storage secret with keys as follows: ```console kubectl create -n openshift-logging secret generic lokistack-dev-odf \ --from-literal=accesskeyid="${ACCESSKEYID}" \ --from-literal=accesskeysecret="${SECRETACCESSKEY}" \ --from-literal=bucketnames="${BUCKET_NAME}" \ --from-literal=endpoint="https://${BUCKETHOST}:${BUCKETPORT}" ``` Where `lokistack-dev-odf` is the secret name. The values for `ACCESSKEYID`, `SECRETACCESSKEY`, `BUCKETNAME`, `BUCKETHOST` and `BUCKET_PORT` are taken from your ObjectBucketClaim's accompanied secret and ConfigMap. Create an instance of by referencing the secret name and type as `s3`: ```yaml apiVersion: loki.grafana.com/v1 kind: LokiStack metadata: name: logging-loki namespace: openshift-logging spec: storage: secret: name: lokistack-dev-odf type: s3 tls: caName: openshift-service-ca.crt tenants: mode: openshift-logging ``` Create a on Swift. Deploy the Loki Operator to your cluster. Create an Object Storage secret with keys as follows: ```console kubectl create secret generic lokistack-dev-swift \ --from-literal=authurl="<SWIFTAUTH_URL>" \ --from-literal=username="<SWIFT_USERNAMEClaim>" \ --from-literal=userdomainname="<SWIFTUSERDOMAIN_NAME>" \ --from-literal=userdomainid="<SWIFTUSERDOMAIN_ID>" \ --from-literal=userid="<SWIFTUSER_ID>" \ --from-literal=password="<SWIFT_PASSWORD>" \ --from-literal=domainid="<SWIFTDOMAIN_ID>" \ --from-literal=domainname="<SWIFTDOMAIN_NAME>" \ --from-literal=containername="<SWIFTCONTAINER_NAME>" \ ``` where `lokistack-dev-swift` is the secret name. Optionally you can provide project specific data and/or a region as follows: ```console kubectl create secret generic lokistack-dev-swift \ --from-literal=authurl="<SWIFTAUTH_URL>" \ --from-literal=username="<SWIFT_USERNAMEClaim>" \ --from-literal=userdomainname="<SWIFTUSERDOMAIN_NAME>" \ --from-literal=userdomainid="<SWIFTUSERDOMAIN_ID>" \ --from-literal=userid="<SWIFTUSER_ID>" \ --from-literal=password="<SWIFT_PASSWORD>" \ --from-literal=domainid="<SWIFTDOMAIN_ID>" \ --from-literal=domainname="<SWIFTDOMAIN_NAME>" \ --from-literal=containername="<SWIFTCONTAINER_NAME>" \ --from-literal=projectid="<SWIFTPROJECT_ID>" \ --from-literal=projectname="<SWIFTPROJECT_NAME>" \ --from-literal=projectdomainid="<SWIFTPROJECTDOMAIN_ID>" \ --from-literal=projectdomainname="<SWIFTPROJECTDOMAIN_name>" \ --from-literal=region="<SWIFT_REGION>" \ ``` Create an instance of by referencing the secret name and type as `swift`: ```yaml spec: storage: secret: name: lokistack-dev-swift type: swift ``` Create a on AlibabaCloud. Deploy the Loki Operator to your cluster. Create an Object Storage secret with keys as follows: ```console kubectl create secret generic lokistack-dev-alibabacloud \ --from-literal=bucket="<BUCKET_NAME>" \ --from-literal=endpoint="<OSSBUCKETENDPOINT>" \ --from-literal=accesskeyid="<OSSACCESSKEY_ID>" \ --from-literal=secretaccesskey="<OSSACCESSKEY_SECRET>" ``` where `lokistack-dev-alibabacloud` is the secret name. Create an instance of by referencing the secret name and type as `alibabacloud`: ```yaml spec: storage: secret: name: lokistack-dev-alibabacloud type: alibabacloud ```