How to Build a Web API
By: Kevin C Lee
Tags: Programming, Python Programming, Data Science, Python, Flask

How to Build a Web API

Guided Example to Set Up an API using Python and Flask to Make Data Accessible to Users

Note: The code for this post can be found here

Photo by geralt on Pixabay

API stands for Application Programming Interface. It is a software intermediary that allows systems to communicate with each other. Most businesses online have likely built APIs for customers and/or for internal use. For example, when a user enters a URL into their browser, e.g. www.medium.com, they are making a request to Medium’s server. Medium will then give back a response to be interpreted and displayed on the user’s browser. Modern client-to-server communications are mostly handled by APIs. The type and response will be dependent on a set of dedicated URLs (endpoints) and the request type. In this guided walkthrough, we’ll use Python and Flask to build a Web API.

Project Plan for Web API

Web APIs democratize data and incentivize scientific research. Below is a small sample of what one might expect from a Web API (that provides brewery data). As downstream users of APIs, knowing the format of the response, which is commonly in JSON, enables us to consume this data for other applications in real time. This type of information should be found in the accompanied documentation. By the same token, designing an API from the perspective of a user is important to ensure it is useful to others. A good API will have well-designed URLs that make it easy for users to intuitively find resources.

[

{

"state": "CA",

"city": "Napa",

"name": "Napa Palisades Beer Company"

}

]

When to Choose a Web API

A Web API isn’t the only approach to provide access to data. However, it is a better alternative to data dump when:

Data set is large, making downloads resource-intensive Data changes or is updated frequently Data needs to be accessed in real time, perhaps for further processing Data doesn’t need to be accessed in its entirety at once

RESTful API

REST (REpresentational State Transfer) is a terminology that is frequently used to describe APIs. It is a framework that describe some best practices for implementing APIs. APIs that satisfy the REST principles are recognized as RESTful APIs. More information on REST can be found here.

What the Flask

Flask is a web framework for Python, which means it provides functionality to building web applications, including managing HTTP requests. We will need this feature to establish communication with our application. For demonstration, we’re using Flask in a very limited capacity, although it can be used to fully develop a robust website. Instructions for the installation and more can be found here. In addition, we will need to use an extension to handle the database connection to complete the API.

Creating a Basic Flask Web Application

We’ll begin by using Flask to create a simple web application. After learning the basics of a Flask app and making sure the software is configured correctly, we’ll turn it into a functioning API. Below is the layout of the API project folder. Importantly, it will contain an app.py code file and a breweries.sqlite file as the data source.

Folder Structure

-====app.py====- from flask import Flask

app=Flask(__name__) @app.route('/')

def index():

return 'Welcome' if __name__ == '__main__':

app.run()

In this step, we create an instance of the Flask class and pass the name of the application (variable __name__) as the argument to the constructor. A Flask object is a WSGI (Web Server Gateway Interface) application which means that the web server passes all the requests it receives to this instance for further processing. Flask maps HTTP requests to Python functions. This is achieved through the .route() decorator in the code. A Route binds a URL to a function, which can be programmed to respond to a request. The designated URL, ‘/’, or technically when no additional path is provided, is mapped to our simple user-defined function, index(). This function returns a simple HTML markup welcome text to be displayed in the browser. The request is then completed. The condition __name__==’__main__’ ensures that the .run() method starts the server when the script is executed as the main program.

Run the application using Python and visit localhost:5000. Port 5000 on the local machine, or http://127.0.0.1:5000/ is the default port which Flask serves the application.

$ python app.py ======================================== Output

* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

To stop the server, hit CTRL+C in the terminal. Any URL that isn’t mapped to a view function will be met with a 404 Not Found error.

Sample Data

Building onto what we’ve created so far, we can implement a small API with sample data that is hardcoded within our web application.

We designate the route ‘/sample/’ to call the user-defined sample() function, which returns the sample data in the form of a list of a Python dictionary with key-value pairs. When URL in route ends with a trailing slash (/), Flask will redirect the request without the trailing slash to the URL with the trailing slash. This means a request to /breweries will be redirected to /breweries/. The query results have to be serialized through the jsonify() method and turned into a response object.

Creating a Database (Optional)

To optimize the data transfer process, we’ll load them into a database and allow the application to query as needed. We use the Python SQL Toolkit SQLAlchemy to accomplish much of what we need. It also has an extension for Flask. For completeness, this demonstration will begin with data in the form of a Pandas DataFrame. Continue to next steps if the data is already in a database.

-====csv_to_sqlite.py====- import sqlalchemy

from sqlalchemy import create_engine conn=create_engine('sqlite:///breweries.sqlite').connect()

df=pd.read_csv('breweries.csv', index_col='index')

df.to_sql('breweries', conn) # use PRAGMA table_info to get column names

# conn.execute('PRAGMA table_info(breweries); ').fetchall() sql_rm_query='ALTER TABLE breweries RENAME TO breweries_old; '

sql_crt_query='CREATE TABLE breweries (\

id bigint primary key,\

name text,\

website text,\

address text,\

city text,\

state text\

); '

sql_ins_query='INSERT INTO breweries SELECT * FROM breweries_old; '

sql_drp_query='DROP TABLE breweries_old' conn.execute(sql_rm_query)

conn.execute(sql_crt_query)

conn.execute(sql_ins_query)

conn.execute(sql_drp_query)

We’re also choosing to use SQLite for our application for its easy of use. Normally, loading a DataFrame into a table can be done quickly with to_sql(). However, it doesn’t not allow a column to be defined as the Primary Key, which would be useful for our database. Since SQLite doesn’t allow significant changes to existing tables, we use a series of sql commands to create a new table with a primary key and insert into it the data from the old table.

Connecting the Web Application to the Database

With the database set up, making the connection is easy with SQLAlchemy. More information on SQLAlchemy can be found here. Having established the connection, the web application can perform queries against the database. For what we designed the API to do, we want to perform SELECT query when a designated URL is accessed.

-====app.py====- from flask import jsonify

from flask_cors import CORS

from sqlalchemy import create_engine CORS(app)

engine=create_engine('sqlite:///assets/data/breweries.sqlite')

def fetch():

results=engine.execute('select state, city, name from breweries')

return jsonify([{'state': result[0], 'city': result[1], 'name': result[2]} for result in results]) @app .route('/breweries/')def fetch():results=engine.execute('select state, city, name from breweries')return jsonify([{'state': result[0], 'city': result[1], 'name': result[2]} for result in results])

We use the flask-CORS extension to make cross-origin resource sharing possible, since we are hosting the database and API in a separate domain. Without going into great detail, CORS enables controlled access to resources located outside of a given domain. The create_engine() function produces an Engine object based on the database URL, and is usually the starting point for any SQLAlchemy application. The database URL usually includes username, password, hostname, database name as well as optional keyword arguments for additional configuration. The typical form of a database URL is:

dialect+driver://username:password@host:port/database

We designate the route ‘/breweries/’ to call the user-defined fetch() function, which executes an SQL query against the database and returns the results.

P.S. For simplicity sake we did not use the Object-Relational Mapping (ORM) feature from SQLAlchemy, which it’s famous for. The idea is to allow queries, simple or complex, to be written using the object-oriented paradigm (Python).

About Routes and API Design

As it stands, the API returns all results to the user at one URL. This approach is inefficient and not a thoughtful design. We can improve our API by allowing users to narrow or filter the data they “request”. One way to solve this problem is to designate more URLs for the different potential use cases. We can create as many routes as needed for our application. This will be necessary when the application expands its functionality. Web applications use different HTTP methods when accessing URLs. For the purposes of this API, we will only use the default GET requests. That is, .route() defaults to .route(methods=[‘GET’])



def fetch_california():

# return filtered results @app .route('/breweries/california')def fetch_california():# return filtered results

def fetch_texas():

# return filtered results @app .route('/breweries/texas')def fetch_texas():# return filtered results .etc

While this might solve some problems from the user’s perspective, the code is cumbersome to write.

Adding Query Parameters

The routes that we have created so far were static. An alternative approach to our problem is to allow dynamic URLs by including variable(s) as part of the .route() decorator. This RESTful approach makes the API much more maintainable and usable. Variables can be added to a URL by marking sections with <variable_name>. The mapped function will receive the <variable_name> as an argument. The specificity passed through the URL are known as query parameters.

-====app.py====-

.route('/breweries/<query_string>')

def fetch(query_string=None):

query_param='where '

params=0

if query_string:

for each_param in query_string.split('&'):

key, value=each_param.split('=')

if params>0:

query_param+='and '

if key.lower()=='state':

query_param+=f'state="{value.upper()}"'

params=params+1

if key.lower()=='name':

query_param+=f'name like "%{value.capitalize()}%"'

params=params+1

if key.lower()=='city':

query_param+=f'city="{value.capitalize()}"'

params=params+1

results=engine.execute(f'select state, city, name from breweries {query_param}')

else:

results=engine.execute('select state, city, name from breweries')

return jsonify([{'state': result[0], 'city': result[1], 'name': result[2]} for result in results]) @app .route('/breweries/') @app .route('/breweries/ ')def fetch(query_string=None):query_param='where 'params=0if query_string:for each_param in query_string.split('&'):key, value=each_param.split('=')if params>0:query_param+='and 'if key.lower()=='state':query_param+=f'state="{value.upper()}"'params=params+1if key.lower()=='name':query_param+=f'name like "%{value.capitalize()}%"'params=params+1if key.lower()=='city':query_param+=f'city="{value.capitalize()}"'params=params+1results=engine.execute(f'select state, city, name from breweries {query_param}')else:results=engine.execute('select state, city, name from breweries')return jsonify([{'state': result[0], 'city': result[1], 'name': result[2]} for result in results])

The <query_string> placeholder will map anything that comes after the ‘/breweries/’ URL. For example, if the user visit ‘/breweries/state=tx&name=brewery’, the ‘state=tx&name=brewery’ will be passed as the argument to the function as query_string. We can also designate multiple URLs to the same function. In this case, the fetch() function is triggered regardless of whether a query string is provided. The variable has to be included as a parameter in the function definition. As part of the API design, we let users know that the web application expects query strings in a specific format:

/breweries/state=tx&name=brewery

Much of the rest of the code is a series of string manipulation of the input variable to parse and build the SQL query statement. If no query_string is passed, the web application will return all entries from our database as before.

API in Action

Using the API, users can create Analytical tools that quickly summarize the data. An advantage of going through an API is that downstream consumers of the data are automatically updated with the database. As it turns out California (not surprisingly) leads the country in the number of breweries.

Brewery Count by State (Plotly.js)

Summary

In this article, we used Flask to create an API. Once you’re satisfied with your web application, I hope you’ll consider deploying it online. I can’t wait to see what you create.