Challenge scenario

You are just starting your junior cloud engineer role with Jooli inc. So far you have been helping teams create and manage Google Cloud resources.

You are expected to have the skills and knowledge for these tasks, so don’t expect step-by-step guides.

Your challenge

You are asked to help a newly formed development team with some of their initial work on a new project around storing and organizing photographs, called Memories. You have been asked to assist the Memories team with initial configuration for their application development environment.

You receive the following request to complete the following tasks:

Create a bucket for storing the photographs.
Create a Pub/Sub topic that will be used by a Cloud Run Function you create.
Create a Cloud Run Function.
Remove the previous cloud engineer’s access from the memories project.
Some Jooli Inc. standards you should follow:

Create all resources in the us-east1 region and us-east1-c zone, unless otherwise directed.
Use the project VPCs.
Naming is normally team-resource, e.g. an instance could be named kraken-webserver1
Allocate cost effective resource sizes. Projects are monitored and excessive resource use will result in the containing project's termination (and possibly yours), so beware. This is the guidance the monitoring team is willing to share; unless directed, use e2-micro for small Linux VMs and e2-medium for Windows or other applications such as Kubernetes nodes.
Each task is described in detail below, good luck!

Task 1. Create a bucket

You need to create a bucket called qwiklabs-gcp-02-80bae9d1ea28-bucket for the storage of the photographs. Ensure the resource is created in the us-east1 region and us-east1-c zone.

Click Check my progress to verify the objective.
Assessment Completed!
Create a bucket called qwiklabs-gcp-02-80bae9d1ea28-bucket

Assessment Completed!

Task 2. Create a Pub/Sub topic

Create a Pub/Sub topic called topic-memories-680 for the Cloud Run Function to send messages.

Click Check my progress to verify the objective.
Assessment Completed!
Create a Pub/Sub topic called topic-memories-680

Assessment Completed!

Task 3. Create the thumbnail Cloud Run Function

Create a Cloud Run Function memories-thumbnail-creator that will to create a thumbnail from an image added to the qwiklabs-gcp-02-80bae9d1ea28-bucket bucket. Ensure the Cloud Run Function is using the Cloud Run function environment (which is 2nd generation). Ensure the resource is created in the us-east1 region and us-east1-c zone.

Create a Cloud Run Function called memories-thumbnail-creator
Note: The Cloud Run Function is required to executes every time an object is created in the bucket created in Task 1. During the process Cloud Run Function may request permission to enable APIs. Please enable each of the required APIs as requested.
Make sure you set the Entry point (Function to execute) to memories-thumbnail-creator and Trigger to Cloud Storage.

Add the following code to the index.js:

const functions = require('@google-cloud/functions-framework');
const crc32 = require("fast-crc32c");
const { Storage } = require('@google-cloud/storage');
const gcs = new Storage();
const { PubSub } = require('@google-cloud/pubsub');
const imagemagick = require("imagemagick-stream");

functions.cloudEvent('memories-thumbnail-creator', cloudEvent => {
  const event = cloudEvent.data;

  console.log(`Event: ${event}`);
  console.log(`Hello ${event.bucket}`);

  const fileName = event.name;
  const bucketName = event.bucket;
  const size = "64x64"
  const bucket = gcs.bucket(bucketName);
  const topicName = "topic-memories-680";
  const pubsub = new PubSub();
  if ( fileName.search("64x64_thumbnail") == -1 ){
    // doesn't have a thumbnail, get the filename extension
    var filename_split = fileName.split('.');
    var filename_ext = filename_split[filename_split.length - 1];
    var filename_without_ext = fileName.substring(0, fileName.length - filename_ext.length );
    if (filename_ext.toLowerCase() == 'png' || filename_ext.toLowerCase() == 'jpg'){
      // only support png and jpg at this point
      console.log(`Processing Original: gs://${bucketName}/${fileName}`);
      const gcsObject = bucket.file(fileName);
      let newFilename = filename_without_ext + size + '_thumbnail.' + filename_ext;
      let gcsNewObject = bucket.file(newFilename);
      let srcStream = gcsObject.createReadStream();
      let dstStream = gcsNewObject.createWriteStream();
      let resize = imagemagick().resize(size).quality(90);
      srcStream.pipe(resize).pipe(dstStream);
      return new Promise((resolve, reject) => {
        dstStream
          .on("error", (err) => {
            console.log(`Error: ${err}`);
            reject(err);
          })
          .on("finish", () => {
            console.log(`Success: ${fileName} → ${newFilename}`);
              // set the content-type
              gcsNewObject.setMetadata(
              {
                contentType: 'image/'+ filename_ext.toLowerCase()
              }, function(err, apiResponse) {});
              pubsub
                .topic(topicName)
                .publisher()
                .publish(Buffer.from(newFilename))
                .then(messageId => {
                  console.log(`Message ${messageId} published.`);
                })
                .catch(err => {
                  console.error('ERROR:', err);
                });
          });
      });
    }
    else {
      console.log(`gs://${bucketName}/${fileName} is not an image I can handle`);
    }
  }
  else {
    console.log(`gs://${bucketName}/${fileName} already has a thumbnail`);
  }
});
Copied!
Add the following code to the package.json:
{
  "name": "thumbnails",
  "version": "1.0.0",
  "description": "Create Thumbnail of uploaded image",
  "scripts": {
    "start": "node index.js"
  },
  "dependencies": {
    "@google-cloud/functions-framework": "^3.0.0",
    "@google-cloud/pubsub": "^2.0.0",
    "@google-cloud/storage": "^5.0.0",
    "fast-crc32c": "1.0.4",
    "imagemagick-stream": "4.1.1"
  },
  "devDependencies": {},
  "engines": {
    "node": ">=4.3.2"
  }
}
Copied!
Verify the thumbnail was successfully created.

Note: If you get a permission denied error stating it may take a few minutes before all necessary permissions are propagated to the Service Agent, wait a few minutes and try again. Ensure you have the relevant roles (Eventarc Service Agent, Eventarc Event Receiver, Service Account Token Creator, and Pub/Sub Publisher) assigned to the correct service accounts.
Click Check my progress to verify the objective.
Verify the Cloud Run Function


Task 4. Test the Infrastructure

You must upload one JPG or PNG image into the bucket

Upload a PNG or JPG image to qwiklabs-gcp-02-80bae9d1ea28-bucket bucket.
Note: Alternatively Download this image https://storage.googleapis.com/cloud-training/gsp315/map.jpg to your machine. Then upload it to the bucket.
You will see a thumbnail image appear shortly afterwards (use REFRESH in the bucket details).
Task 5. Remove the previous cloud engineer

You will see that there are two users defined in the project.

One is your account (<PRESIDIO_ANONYMIZED_EMAIL_ADDRESS> with the role of Owner).
The other is the previous cloud engineer (<PRESIDIO_ANONYMIZED_EMAIL_ADDRESS> with the role of Viewer).
Remove the previous cloud engineer’s access from the project.
Click Check my progress to verify the objective.
Remove the previous cloud engineer

please help me I'm stuck 




Welcome to Cloud Shell! Type "help" to get started.
To set your Cloud Platform project in this session use “gcloud config set project [PROJECT_ID]”
student_04_d12b9cbe64c6@cloudshell:~$ gcloud config set project qwiklabs-gcp-02-80bae9d1ea28
Updated property [core/project].
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ gcloud storage buckets create qwiklabs-gcp-02-80bae9d1ea28-bucket \
  --location=us-east1 \
  --uniform-bucket-level-access
ERROR: (gcloud.storage.buckets.create) "gcloud storage buckets create" only accepts bucket URLs.
Example: "gs://bucket"
Received: "file://qwiklabs-gcp-02-80bae9d1ea28-bucket"
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ gcloud storage buckets create gs://qwiklabs-gcp-02-80bae9d1ea28-bucket \
  --location=us-east1 \
  --uniform-bucket-level-access
Creating gs://qwiklabs-gcp-02-80bae9d1ea28-bucket/...
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ gcloud pubsub topics create topic-memories-680
Created topic [projects/qwiklabs-gcp-02-80bae9d1ea28/topics/topic-memories-680].
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ mkdir thumbnail-function && cd thumbnail-function
student_04_d12b9cbe64c6@cloudshell:~/thumbnail-function (qwiklabs-gcp-02-80bae9d1ea28)$ cat > index.js << 'EOF'
student_04_d12b9cbe64c6@cloudshell:~/thumbnail-function (qwiklabs-gcp-02-80bae9d1ea28)$ mkdir thumbnail-function && cd thumbnail-fun
student_04_d12b9cbe64c6@cloudshell:~/thumbnail-function (qwiklabs-gcp-02-80bae9d1ea28)$ cat > index.js << 'EOF'
const functions = require('@google-cloud/functions-framework'); const crc32 = require("fast-crc32c"); const { Storage } = require('@google-cloud/storage'); const gcs = new Storage(); const { PubSub } = require('@google-cloud/pubsub'); const imagemagick = require("imagemagick-stream");

functions.cloudEvent('memories-thumbnail-creator', cloudEvent => { const event = cloudEvent.data;

console.log(Event: ${event}); console.log(Hello ${event.bucket});

const fileName = event.name; const bucketName = event.bucket; const size = "64x64" const bucket = gcs.bucket(bucketName); const topicName = "topic-memories-680"; const pubsub = new PubSub(); if ( fileName.search("64x64_thumbnail") == -1 ){ var filename_split = fileName.split('.'); var filename_ext = filename_split[filename_split.length - 1]; var filename_without_ext = fileName.substring(0, fileName.length - filename_ext.length ); if (filename_ext.toLowerCase() == 'png' || filename_ext.toLowerCase() == 'jpg'){ console.log(Processing Original: gs://${bucketName}/${fileName}); const gcsObject = bucket.file(fileName); let newFilename = filename_without_ext + size + '_thumbnail.' + filename_ext; let gcsNewObject = bucket.file(newFilename); let srcStream = gcsObject.createReadStream(); let dstStream = gcsNewObject.createWriteStream(); let resize = imagemagick().resize(size).quality(90); srcStream.pipe(resize).pipe(dstStream); return new Promise((resolve, reject) => { dstStream .on("error", (err) => { console.log(Error: ${err}); reject(err); }) .on("finish", () => { console.log(Success: ${fileName} → ${newFilename}); gcsNewObject.setMetadata( { contentType: 'image/'+ filename_ext.toLowerCase() }, function(err, apiResponse) {}); pubsub .topic(topicName) .publisher() .publish(Buffer.from(newFilename)) .then(messageId => { console.log(Message ${messageId} published.); }) .catch(err => { console.error('ERROR:', err); }); }); }); } else { console.log(gs://${bucketName}/${fileName} is not an image I can handle); } } else { console.log(gs://${bucketName}/${fileName} already has a thumbnail); } }); EOF
> ^C
student_04_d12b9cbe64c6@cloudshell:~/thumbnail-function (qwiklabs-gcp-02-80bae9d1ea28)$ ls
student_04_d12b9cbe64c6@cloudshell:~/thumbnail-function (qwiklabs-gcp-02-80bae9d1ea28)$ ls
student_04_d12b9cbe64c6@cloudshell:~/thumbnail-function (qwiklabs-gcp-02-80bae9d1ea28)$ cd ..
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ ls
README-cloudshell.txt  thumbnail-function
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ cd thumbnail-function
student_04_d12b9cbe64c6@cloudshell:~/thumbnail-function (qwiklabs-gcp-02-80bae9d1ea28)$ ls
student_04_d12b9cbe64c6@cloudshell:~/thumbnail-function (qwiklabs-gcp-02-80bae9d1ea28)$ cd ..
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ ls
README-cloudshell.txt  thumbnail-function
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ cat > index.js << 'EOF'
const functions = require('@google-cloud/functions-framework');
const crc32 = require("fast-crc32c");
const { Storage } = require('@google-cloud/storage');
const gcs = new Storage();
const { PubSub } = require('@google-cloud/pubsub');
const imagemagick = require("imagemagick-stream");

functions.cloudEvent('memories-thumbnail-creator', cloudEvent => {
  const event = cloudEvent.data;

  console.log(`Event: ${event}`);
  console.log(`Hello ${event.bucket}`);

  const fileName = event.name;
  const bucketName = event.bucket;
  const size = "64x64"
  const bucket = gcs.bucket(bucketName);
  const topicName = "topic-memories-680";
  const pubsub = new PubSub();
  if ( fileName.search("64x64_thumbnail") == -1 ){
    var filename_split = fileName.split('.');
    var filename_ext = filename_split[filename_split.length - 1];
    var filename_without_ext = fileName.substring(0, fileName.length - filename_ext.length );
    if (filename_ext.toLowerCase() == 'png' || filename_ext.toLowerCase() == 'jpg'){
      console.log(`Processing Original: gs://${bucketName}/${fileName}`);
      const gcsObject = bucket.file(fileName);
      let newFilename = filename_without_ext + size + '_thumbnail.' + filename_ext;
      let gcsNewObject = bucket.file(newFilename);
      let srcStream = gcsObject.createReadStream();
      let dstStream = gcsNewObject.createWriteStream();
      let resize = imagemagick().resize(size).quality(90);
      srcStream.pipe(resize).pipe(dstStream);
      return new Promise((resolve, reject) => {
        dstStream
          .on("error", (err) => {
            console.log(`Error: ${err}`);
            reject(err);
          })
EOF console.log(`gs://${bucketName}/${fileName} already has a thumbnail`);ndle`);
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ ls
index.js  README-cloudshell.txt  thumbnail-function
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ cat > package.json << 'EOF'
{
"name": "thumbnails",
"version": "1.0.0",
"description": "Create Thumbnail of uploaded image",
"scripts": {
 "start": "node index.js"
},
"dependencies": {
 "@google-cloud/functions-framework": "^3.0.0",
 "@google-cloud/pubsub": "^2.0.0",
 "@google-cloud/storage": "^5.0.0",
 "fast-crc32c": "1.0.4",
 "imagemagick-stream": "4.1.1"
},
"devDependencies": {},
"engines": {
 "node": ">=4.3.2"
}
}
EOF
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ ls
index.js  package.json  README-cloudshell.txt  thumbnail-function
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ gcloud functions deploy memories-thumbnail-creator \
  --gen2 \
  --runtime=nodejs20 \
  --region=us-east1 \
  --entry-point=memories-thumbnail-creator \
  --trigger-bucket=qwiklabs-gcp-02-80bae9d1ea28-bucket
API [run.googleapis.com] not enabled on project [qwiklabs-gcp-02-80bae9d1ea28]. Would you like to enable and retry (this will take a
 few minutes)? (y/N)?  y

Enabling service [run.googleapis.com] on project [qwiklabs-gcp-02-80bae9d1ea28]...
Operation "operations/acf.p2-649878662423-da461d54-a625-49b9-8bc4-17a761e9ed02" finished successfully.
API [eventarc.googleapis.com] not enabled on project [qwiklabs-gcp-02-80bae9d1ea28]. Would you like to enable and retry (this will 
take a few minutes)? (y/N)?  y

Enabling service [eventarc.googleapis.com] on project [qwiklabs-gcp-02-80bae9d1ea28]...
Operation "operations/acat.p2-649878662423-309f1d40-deb5-4af3-9e15-855d89905d63" finished successfully.
ERROR: (gcloud.functions.deploy) ResponseError: status=[400], code=[Ok], message=[Validation failed for trigger projects/qwiklabs-gcp-02-80bae9d1ea28/locations/us-east1/triggers/memories-thumbnail-creator-256190: Invalid resource state for "": Permission denied while using the Eventarc Service Agent. If you recently started to use Eventarc, it may take a few minutes before all necessary permissions are propagated to the Service Agent. Otherwise, verify that it has Eventarc Service Agent role.]
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$ gcloud functions deploy memories-thumbnail-creator \
  --entry-point memories-thumbnail-creator \
  --trigger-event google.storage.object.finalize \
  --trigger-resource gs://qwiklabs-gcp-02-80bae9d1ea28-bucket \
  --runtime nodejs18 \
  --region us-east1
As of this Cloud SDK release, new functions will be deployed as 2nd gen  functions by default. This is equivalent to currently deploying new  with the --gen2 flag. Existing 1st gen functions will not be impacted and will continue to deploy as 1st gen functions.
You can disable this behavior by explicitly specifying the --no-gen2 flag or by setting the functions/gen2 config property to 'off'.
To learn more about the differences between 1st gen and 2nd gen functions, visit:
https://cloud.google.com/functions/docs/concepts/version-comparison
Preparing function...done.                                                                                                         
X  Deploying function...                                                                                                           
     [Build] Logs are available at [https://console.cloud.google.com/cloud-build/builds;region=us-east1/8ed9021c-081f-4267-bbc2-b44
  1f9565de0?project=649878662423]                                                                                                  
  OK [Service]                                                                                                                     
  OK [Trigger]                                                                                                                     
  -  [ArtifactRegistry] Deleting function artifacts in Artifact Registry...                                                        
  .  [Healthcheck]                                                                                                                 
  .  [Triggercheck]                                                                                                                
Failed.                                                                                                                            
  [WARNING] *** Improve build performance by generating and committing package-lock.json.
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Creating trigger failed for projects/qwiklabs-gcp-02-80bae9d1ea28/locations/us-east1/triggers/memories-thumbnail-creator-732066: Failed to update storage bucket metadata. Possible causes include:
- Cloud Storage service agent not being able to read the Cloud Pub/Sub topic (projects/qwiklabs-gcp-02-80bae9d1ea28/topics/eventarc-us-east1-memories-thumbnail-creator-732066-351) created by Eventarc. (See https://cloud.google.com/eventarc/docs/run/quickstart-storage#before-you-begin)
- Notification quota exceeded for the GCS bucket (qwiklabs-gcp-02-80bae9d1ea28-bucket). (See https://cloud.google.com/storage/quotas#buckets): invalid argument
student_04_d12b9cbe64c6@cloudshell:~ (qwiklabs-gcp-02-80bae9d1ea28)$