Uploading Files To Amazon S3 With Flask Form – Part1 – Uploading Small Files

Uploading Files To Amazon S3 With Flask Form - Part1 - Uploading Small Files

Uploading Files To Amazon S3 With Flask Form – Part1 – Uploading Small Files

This article is aimed at developers who are interested to upload small files to Amazon S3 using Flask Forms. In the following tutorial, I will start with an overview of the Amazon S3 followed by the python Boto3 code to manage file operations on the S3 bucket and finally integrate the code with Flask Form.


1) Application Overview

As a web developer, it is a common requirement to have the functionality of uploading the files to a database or to a server for further processing. Cloud computing brings the concept of treating the infrastructure as not hardware rather as software enabling the web developers with limited knowledge of infrastructure/hardware to take full advantage of the services.

Amazon is the most popular choice of cloud computing and Python became the go-to programming language for any cloud computing.

Objectives of this tutorial:

By the end of this tutorial, you will be able to:

  1. Create an S3 bucket with Python SDK.
  2. Enable bucket versioning to keep a track of file versions.
  3. Upload small files to S3 with Python SDK.
  4. Create a Flask Form to allow a certain type of files to upload to S3.
Uploading Files To Amazon S3 With Flask Form Design

Uploading Files To Amazon S3 With Flask Form Design

Alright, let us start with an introduction to Amazon S3.


2) Introduction To Amazon S3

Amazon Simple Storage Service (in short S3)  provides secure and highly scalable object storage which is very easy to use as it has a very simple web service interface to store and retrieve any amount of data. The main advantage of using S3 to store objects (in our case small files) is to access them anytime anywhere from the web rather than logging into a database or an application server to access a file.

With AWS SDK we can integrate S3 with other AWS services and external Flask applications as well.

The term “files” and “objects” are pretty much the same when dealing with AWS S3 as it refers to all the files as objects. Let us understand the basic components of S3 before go any further.

AWS S3 Components

AWS S3 Components

The basic components of an S3 bucket are

  1. Bucket name
  2. Objects /files in a bucket
  3. Key.

The above diagram should help you understand how the components are classified in an S3 bucket.  The Key is a unique identifier mapped to each object in an S3. More details on S3 can be found on their official website.


3) Storage Solution With Python SDK

Next up we are going to get our back end code ready which takes the input object from the user through the flask form and loads it into the S3. First thing first let’s create the S3 bucket.

Install the python botot3 with python package manager i.e. pip

1) Define S3 Client

We need to import boto3 into our code and also define a function to define the s3 client. As S3 is a global service and not region-specific we need not specify the region while defining the client.

2) Create S3 Bucket

The following function will accept the bucket name as a parameter and uses the s3 client from the above function to create the bucket in your current region. If the region is not specified the bucket is created in the default “us-east” region. The bucket name must adhere to the below standards

  1. The bucket name must be 3 – 63 characters in length.
  2. Lower case, numbers, and hyphens are allowed to use.
  3. Do not use dot (.) and hyphen (-) characters in the bucket name.

3) Creating Bucket Policy

Now we have a bucket created, we will create a bucket policy to restrict who and from where the objects inside the buckets can be accessed. In short, bucket policy is the way to configure the access policies to your bucket like the IP Ranges, hosts, who, and what can be done to your bucket.

I will be using the JSON format (dictionary format) to specify the policy configuration.

4) Versioning Objects In S3

AWS S3 provides the versioning of the objects in S3 and by default, it is not enabled. Each object has a unique version ID in buckets with version enabled.

The version id for objects will be set to Null for S3 buckets that are not version enabled.

5) Validating Bucket and Bucket Policy

Now that we have the code for creating the bucket and the policy, we will execute the code and validate the bucket and its policy using the following code.


4) Uploading Small Files To S3 With Python SDK

Amazon S3 provides a couple of ways to upload the files, depending on the size of the file user can choose to upload a small file using the “put_object” method or use the multipart upload method. I will upload a separate tutorial on how to upload huge files to S3 with Flask.  In this tutorial, we will focus on uploading small files.

From the S3 API specification, to upload a file we need to pass the full file path, bucket name, and KEY. The KEY as you can remember from the introduction section identifies the location path of your file in an S3 bucket.  As S3 works as a key-value pair, it is mandatory to pass the KEY to the upload_file method.

The following two methods will show you how to upload a small file to S3 followed by listing all the files in a bucket.


5. Integration With Flask Web Application

Let us set up the app like below and then we will go into details.

Time to discuss the components in detail before we execute the code We will first discuss the design steps that are being implemented. The design is pretty simple and I will do my best to explain it in that way.

5a) Components: Templates/main,html 

I will be using a single template (main.html) simple enough for this demonstration purpose.

This is the home page for the user to upload a file to S3. The HTML template is quite simple with just the upload file option and submits button.

The template is embedded with flask messages while will be passed by the application code based on the validation results. If the user submits the button without choosing a file or uploads a file that is not in the allowed extensions the error message appears on the main page else a success message appears.

5b) Components: templates/includes/_flashmsg.html

5c) Components: templates/includes/_formhelpers.html

5d) Components: views/s3.py

The details inside s3.py are pretty much the same we discussed in the above section. I will show the code once again here.

5e) Components: app.py

The app.py is the main orchestrator of our program. We first set the allowed file extensions to Excel spreadsheets using the ALLOWED_EXTENSIONS variable. The function index or the app route ‘/’ just displays the main.html page.

The function upload_files_to_s3 will be triggered when the user clicks on the submit button on the main.html page and validates the following scenarios:

  1. If the file to upload is empty (i.e. missing) or the file extension is not in the allowed extensions variable the function throws the error message.
  2. If the file to upload is in the allowed extensions then the file will be uploaded to S3 using the s3_upload_small_files function in views/s3.py

This pretty much concludes the programming part.


6. Final Step – Executing The Code

Home Page:

Flask Application Home Page

Flask Application Home Page

 

SCENARIO 1:

No File selected: When the user clicks the submit button without selecting any file.

Flask Application No File Selected To Upload

Flask Application No File Selected To Upload

SCENARIO 2:

Wrong File Extension: When the user tries to upload a file that is not set in the extension.

Flask Application Wrong File Format

Flask Application Wrong File Format

SCENARIO 3:

File Upload: When the user tries to upload the right file extension.

Flask Application Successful File Uploads

Flask Application Successful File Uploads

You can validate the file details by running the function – s3_read_objects in views/s3.py


If you liked this article and if it helped you in any way, feel free to like it and subscribe to this website for more tutorials.

If you believe this article will be of big help to someone, feel free to share.

About the author

Stay Informed

It's important to keep up
with industry - subscribe!

Stay Informed

Looks good!
Please enter the correct name.
Please enter the correct email.
Looks good!

Related articles

Implementing Machine Learning in Web Applications with Python and TensorFlow

The Google-developed open-source software package TensorFlow is used to create and train machine learning models. Because it functions particularly ...

Flask Development Made Easy: A Comprehensive Guide to Test-Driven Development

In this comprehensive guide, I will share my extensive experience in implementing TDD in Flask development, providing you with practical tips, code ...

Implementing a BackgroundRunner with Flask-Executor

In today's fast-paced digital landscape, providing a responsive and user-friendly experience has become paramount for web applications. One common ...

No comments yet

Sign in

Forgot password?

Or use a social network account

 

By Signing In \ Signing Up, you agree to our privacy policy

Password recovery

You can also try to

Or use a social network account

 

By Signing In \ Signing Up, you agree to our privacy policy