How to use S3 services with python
Tags: python, aws, cloud
Published: 2022-09-06 23:12:28

## Introduction
##### Hi, in this article we will discuss how to connect vultr object storage with python.

> You can also use (digitalocean, aws or etc..)

##### In the beginning, What is Object Storage?
Object Storage is Vultr's S3-compatible, web-accessible cloud object storage service for storing and retrieving file objects. 
Objects can be accessed privately or publicly, over the web. Use cases for cloud object storage including:  

* Share Files like ( images, videos, pdf, ...etc)   
* Share software or business files
* Share static website hosting for assets such as images, JavaScript, CSS, or media files

Summary, When you need to handle vast numbers of file objects without having to manage or grow the underlying filesystem, object storage is suitable.

## Getting Started
#### How to create object storage
Select the Objects menu at the top of the [Vultr customer portal](https://my.vultr.com/) to navigate to the object storage subscription listing, then use the [Add Object Storage](https://my.vultr.com/objectstorage/) form to deploy. After deployment, view your access keys on the [subscription listing page](https://my.vultr.com/objectstorage/).
######  If you have any question about object storage see this [article](https://www.vultr.com/docs/vultr-object-storage).

### Installing requirements

1. Create new Object storage with Vultr.
2. Download and Install Python from the [python.org](https://www.python.org/) .
3. Installing boto3 with pip line:
	1.   Open cmd for Windows or terminal from MacOS.
	2.   And type  `pip install boto3`

### What is boto3?
Boto3 is the Software Development Kit (SDK) for Python, which allows Python developers to write software that makes use of services like Vultr Object Storage.

### Let's get started:

* Open your favorite code editor. 
* Create a new python file with any name.
* Let's code:

###### First thing first we must get the Acess Key & Secret Key from your object storage page from [subscription listing page](https://my.vultr.com/objectstorage/).

And let us save it in variable:

	object_storage_config = {
		'region_name': 'ewr1',
		'endpoint_url': 'https://ewr1.vultrobjects.com',
		'aws_access_key_id': 'Your_access_key',
		'aws_secret_access_key': 'Your_secret_key'
	}
	
#### How to upload a file to your file dir:

	import boto3
	client = boto3.client('s3', **object_storage_config)
  

**Now we have connected to vultr s3 object storage**

######  Let us upload a file to object storage but first we must create a new bucket:
Objects are stored in buckets. Before you can begin uploading objects, you need to create at least one bucket. Buckets are similar to folders on a file system.
Know more in this article [article](https://www.vultr.com/docs/vultr-object-storage).

Now we create a new bucket:

	client.create_bucket(Bucket="bucket_name")
	
Now upload file with boto3:

	client.upload_file("file_name", "bucket_name", "object_name.extention")
	
Example code for the above:

	import boto3
	
	object_storage_config = {
		'region_name': 'ewr1',
		'endpoint_url': 'https://ewr1.vultrobjects.com',
		'aws_access_key_id': 'Your_access_key',
		'aws_secret_access_key': 'Your_secret_key'
	}
	
	client = boto3.client('s3', **object_storage_config)
	client.create_bucket(Bucket="uploaded_images")
	client.upload_file("./images/logo.png", "uploaded_images", "logo.png")
	
#### What about Multipart files
We can upload a multipart file with function **upload_fileobj**.

	client.upload_fileobj(file, "bucket_name", "object_name.extention")
	
Example:

    import boto3
    object_storage_config = {
        'region_name': 'ewr1',
        'endpoint_url': 'https://ewr1.vultrobjects.com',
        'aws_access_key_id': 'Your_access_key',
        'aws_secret_access_key': 'Your_secret_key'
    }

    client = boto3.client('s3', **object_storage_config)
    # WE ALREADY CREATED BUCKET
    with open("FILE_NAME", "rb") as file:
        client.upload_fileobj(file, "uploaded_images", "logo.png")
				
Now we have uploaded a file to s3 storage but we have a problem. The problem is that the files we have uploaded are in private mode and there is no one can access them without an access key.
We have two options to solve this first to make the files in the public mode when we upload it and second we can create a Presigned URL.
### What is Presigned URLs?
Vultr users who can access the object will generate a pre-signed URL. The generated URL will be delivered to unauthorized users. The pre-signed URL can be entered in a browser or used in an HTML web page or application. Credentials that use a pre-signed URL are the credentials of the Vultr user who generated the URL.
pre-signed URLs are valid for the limited time specified when the URL was generated.

	response = client.generate_presigned_url(
		'get_object', 
		Params={'Bucket': "bucket_name", 'Key': "object_name"},
		ExpiresIn=expiration
	)
	
> Expiration: Time in seconds for the presigned URL to remain valid

Example:

    import boto3
    object_storage_config = {
        'region_name': 'ewr1',
        'endpoint_url': 'https://ewr1.vultrobjects.com',
        'aws_access_key_id': 'Your_access_key',
        'aws_secret_access_key': 'Your_secret_key'
    }

    client = boto3.client('s3', **object_storage_config)
    response = client.generate_presigned_url('get_object', Params={'Bucket': "uploaded_images", 'Key': "logo.png"}, ExpiresIn=3600)
    print(response)
		
###  Option two is to upload the file in public mode
We can do that by adding **ExtraArgs** to **upload_file** or **upload_fileobj** function.  
Example:

	import boto3

	object_storage_config = {
			'region_name': 'ewr1',
			'endpoint_url': 'https://ewr1.vultrobjects.com',
			'aws_access_key_id': 'Your_access_key',
			'aws_secret_access_key': 'Your_secret_key'
	}

	client = boto3.client('s3', **object_storage_config)
	client.create_bucket(Bucket="uploaded_images")
	client.upload_file("./images/logo.png", "uploaded_images", "logo2.png", ExtraArgs={"ACL": "public-read"})