How add files to S3 Bucket using Shell Script - onlinetutorialspoint

PROGRAMMINGJava ExamplesC Examples
Java Examples
C Examples
C Tutorials
aws
JAVAEXCEPTIONSCOLLECTIONSSWINGJDBC
EXCEPTIONS
COLLECTIONS
SWING
JDBC
JAVA 8
SPRING
SPRING BOOT
HIBERNATE
PYTHON
PHP
JQUERY
PROGRAMMINGJava ExamplesC Examples
Java Examples
C Examples
C Tutorials
aws
Here we will see how to add files to S3 Bucket using Shell Script.
Create S3 Bucket
Create an IAM user, get Access Key and Secret Key
The shell script is the most prominent solution to push the files into S3 bucket if we consider this task as an independent. As part of this tutorial, I am going to push all the files under /opt/s3files directory to s3 bucket.
bucket=my-files
files_location=/opt/s3files/
now_time=$(date +"%H%M%S")
contentType="application/x-compressed-tar"
dateValue=`date -R`
# your key goes here..
s3Key=AKIAWQEOCFWY224LOMFA
# your secrets goes here..
s3Secret=wu78rgdkjrhhdbcoUed+Xm1chCytUTiVeyOUDFJE

function pushToS3()
{
  files_path=$1
  for file in $files_path*
  do
    fname=$(basename $file)
    logInfo "Start sending $fname to S3"
    resource="/${bucket}/${now_date}/${fname}_${now_time}"
    stringToSign="PUT\n\n${contentType}\n${dateValue}\n${resource}"
    signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`
    curl -X PUT -T "${file}" \
     -H "Host: ${bucket}.s3.amazonaws.com" \
     -H "Date: ${dateValue}" \
     -H "Content-Type: ${contentType}" \
     -H "Authorization: AWS ${s3Key}:${signature}" \
      https://${bucket}.s3.amazonaws.com/${now_date}/${fname}_${now_time}
     logInfo "$fname has been sent to S3 successfully."
  done
}
pushToS3 $files_location
Provide execute permissions to the file:
$chmod 777 SendToS3.sh
$./SendToS3.sh
S3 Upload Objects
Happy Learning 🙂

Date yyyymmdd format in Shell Script
Shell Script How to write logs in a Separate File
How to Copy Local Files to AWS EC2 instance Manually ?
Java Reflection Get Field Information
Modes of Python Program
How to add dynamic files to JTree
Java 8 walk How to Read all files in a folder
Java 8 – Convert java.util.Date to java.time.LocalDate
php readfile Example Tutorials
How to convert Timestamp to Date in Java

Date yyyymmdd format in Shell Script
Shell Script How to write logs in a Separate File
How to Copy Local Files to AWS EC2 instance Manually ?
Java Reflection Get Field Information
Modes of Python Program
How to add dynamic files to JTree
Java 8 walk How to Read all files in a folder
Java 8 – Convert java.util.Date to java.time.LocalDate
php readfile Example Tutorials
How to convert Timestamp to Date in Java






Luis Felipe
December 20, 2021 at 4:11 am - Reply 

Hello!
I have a question: does S3 verify if a data already exists in it before upload? I mean, if I already have an archive in S3 and it’s still in the path of upload I don’t want to send it to S3 again.










Luis Felipe
December 20, 2021 at 4:11 am - Reply 

Hello!
I have a question: does S3 verify if a data already exists in it before upload? I mean, if I already have an archive in S3 and it’s still in the path of upload I don’t want to send it to S3 again.




Hello!
I have a question: does S3 verify if a data already exists in it before upload? I mean, if I already have an archive in S3 and it’s still in the path of upload I don’t want to send it to S3 again.
 


Δ
 Install Java on Mac OS
 Install AWS CLI on Windows
 Install Minikube on Windows
 Install Docker Toolbox on Windows
 Install SOAPUI on Windows
 Install Gradle on Windows
 Install RabbitMQ on Windows
 Install PuTTY on windows
 Install Mysql on Windows
 Install Hibernate Tools in Eclipse
 Install Elasticsearch on Windows
 Install Maven on Windows
 Install Maven on Ubuntu
 Install Maven on Windows Command
 Add OJDBC jar to Maven Repository
 Install Ant on Windows
 Install RabbitMQ on Windows
 Install Apache Kafka on Ubuntu
 Install Apache Kafka on Windows