How to Build APIs with Flask and Create Boilerplates?

Great, we have the EDAs and the Model ready, even we have the .pkl file with the result, so we have to create the service to expose this to the real world. But before that we have to organize the structure of the project in Flask. This will help us to have a common pattern where the files live and where we can extend to create and add more models under the same server


However this is not an easy task, basically because Flask is not an opinionated framework, which means that each data scientist can structure a project according to their own necessity, or desires. There is not a common or suggested pattern to follow. Fortunately there are a couple of packages that help us structure a project in a more ‘formal’ way, so here we’ll going to explain how to do it more professionally

Content of the blog post
  1. What is Flask?
  2. What is an API?
  3. How to structure APIs projects?
  4. Using Flask Blueprints

Step #1 - What is Flask?


Flask is a micro framework for web development in the Python language. What micro framework means specifically is that it comes with very little in terms of available features and boilerplate code at the start of a project’s development. Now that’s not to say there aren’t libraries upon libraries of features and plugins available for the Flask framework, but by default, Flask will tend to not include a feature unless you as a developer have specified otherwise.

This differs largely from frameworks like Rails or Django (the latter being another highly popular Python web framework), where a large amount of features and conveniences are generated for you on project creation. For example, both of these frameworks provide a Database-Abstraction layer that allows developers to easily read and write to databases through object models (e.g. Rails through Active Model, Django through their Object-relational Mapper).

The downside of a micro framework is obviously the lack of these features on start-up. Many if not all of the features in the more extensive frameworks exist in the Flask library, but they have to be included explicitly. The major upside, however, is that if you aren’t going to use those features, it’s a lot easier to just make a Flask app and avoid needing to delete a lot of unnecessary files and dependencies. Micro framework applications come with a lot less code bloat, and can always be scaled up with additional tools as needed.

For my purposes, my backend simply needed to be an API for reading and writing data files, and returning formatted JSON objects. In fact, I didn’t even need a database. In my case, the Flask micro framework was much easier to implement, and provided me the flexibility to include further Python modules as needed.

Step #2 - What is an API?

API stands for “Application Programming Interface”. In the simplest terms an API is just a structured way for software applications to communicate with each other.

An API can give you access to the services or data available from another software application. When the application receives a specially formatted request - referred to as an API call - it responds by providing the requested service or data in a way that can be integrated into other applications or workflows.

Why use APIs?
APIs are often used by software developers to enhance their own projects. Instead of building everything from scratch developers can use APIs to take advantage of services offered by other applications. For example, the Google Maps API makes it relatively easy to incorporate an interactive map into your project - there’s no need to create your own.

The restaurant metaphor: API as waiter
Some people find it helpful to think of an API as a waiter taking and delivering orders in a restaurant. Andrew Park describes this metaphor in his blog:

Imagine you’re a customer at a restaurant. The waiter (the API) functions as an intermediary between customers like you (the user) and the kitchen (web server). You tell the waiter your order (API call), and the waiter requests it from the kitchen. Finally, the waiter will provide you with what you ordered.

As a customer, you don’t need to know how the kitchen or the restaurant operates in order to get what you want: the food. You just need to know how to order it.

That last sentence - you just need to know how to order it - is important to remember. Though APIs for different services have a common structure, each will have its own options. You will need to read a particular API’s documentation to understand what is available and get results.


Step #3 - How to structure APIs projects?


When I started working on a flask application for the first time, I had a tough time figuring out the correct structure and packages to achieve the necessities of my flask app. Hence, I decided to make this very basic lightweight boilerplate available for the community while I figured out the structure working on multiple live projects in the past.

When we start a project from scratch, especially the backend services, there are some basic functionalities that it should possess. The following are some of the points in building up an efficient app explained concerning this flask boilerplate.

Folder Structure
Flask applications generally have many variations in their folder structure. The choice of keeping files in different folders is dependent on an individual/team/project type. On a high level, flask application can be created even with a single page or via a factory pattern method. Let's talk a bit more about the concepts here:

  • Factory Pattern: Well-structured web apps separate logic between files and modules, typically with separation of concerns in mind. This seems tricky with Flask at first glance because our app depends on an app object that we create via app = Flask(__name__). Separating logic between modules means we would be importing this app object all over the place, eventually leading to problems like circular imports. The Flask Application Factory refers to a common pattern for solving this dilemma. The reason why the Application Factory is so important has to do with something called Flask's Application Context. Hence, we are making use of factory pattern inside this boilerplate project to make our lives a bit easier.
  • Blueprints: In a normal project, the number of views, and templates, and models, and forms, etc. will grow. You can (hackily) separate them, but wouldn’t it be nice to have something that groups related views, templates, models, and forms? Blueprints! They are a great way to modularly compose Flask applications, which will help scale up projects.

Well, in this boilerplate, there's one blueprint core serving as a base and more can be added later as shown below:


# app/__init__.py
def create_app():
    ...
    app = Flask(APP_NAME)
    ...
    from .core.views import core as core_blueprint
    app.register_blueprint(
        core_blueprint,
        url_prefix='/api/v1/core'
    )

 
So, the folder structure looks as 👇

├── app
│   ├── core                # blueprint
│   │   ├── __init__.py
│   │   ├── constants.py
│   │   ├── enums.py
│   │   ├── tasks.py        # celery tasks specific to this blueprint
│   │   └── views.py
│   ├── config.py
│   └── __init__.py         # app initialization logic
├── .env
├── .gitignore
├── Pipfile
├── Pipfile.lock
├── README.md
├── authentication.py
├── celery_worker.py
└── run.py

 
Managing External Packages
One of the most important steps is to create a virtual environment for python projects to manage dependencies. There are various python packages for this purpose such as virtualenv, pyenv, etc. This boilerplate uses pipenv to address all the issues that I have faced using other packages for managing dependencies. Pipenv automatically creates and manages a virtual environment for your projects, as well as adds or removes packages from your Pipfile as you install/uninstall packages.

This boilerplate has those packages which are required to build up an app and hence you can always customize based on your need 😉

Workspace Setup
Though not directly related to an app, your workspace setup does affect your app creation process. Hence, setting up a workspace is a major step to speed up your working environment. I have written a post that specifically talks about setting up a python workspace in a visual studio code editor which you can refer and quickly get started

Step #4 - Using Flask Blueprints


Configuration
Configuring a Flask app (or maybe any app) generally requires a lot of effort (if we are not experienced). In general, there can be 2 aspects to it -

  • Storing secrets securely: Usually, a lot of secrets makes up a project such as DB credentials, API keys, etc. It is recommended to keep these secrets in a secured location not being tracked in git. Which is why we have a .env (git untracked, to be created from .env.example) file at the root location of this boilerplate which makes all the secrets present in it available in our project using python-dotenv package. Hence, it is super easy to get started with just creating .env from .env.example by setting correct values 😁
  • Creating multiple configurations (dev, staging, prod): Another important point is to separate global configurations based on the environment such as dev, staging, prod. That's the reason we have config.py, a file containing configuration classes based on the environments that get loaded in the Flask app based on the current working environment (which can be set in 

# configuration classes (app/config.py)
class BaseConfig(object):
    ''' Base config class. '''
...

class Development(BaseConfig):
    ''' Development config. '''
...

class Staging(BaseConfig):
    ''' Staging config. '''
...

class Production(BaseConfig):
    ''' Production config '''
...
config = {
    'development': Development,
    'staging': Staging,
    'production': Production,
}

 
# app/__init__.py
from config import config
...

def create_app():
    # loading env vars from .env file
    load_dotenv()
    APPLICATION_ENV = get_environment()
    app = Flask(APP_NAME)
    # loading config based on current environment
    app.config.from_object(config[APPLICATION_ENV])
    # APPLICATION_ENV is set in .env to set current environment

 
Logging
Logging is a means of tracking events that happen when some software runs. We add logging calls to our code to indicate that certain events have occurred.
As we already have environment based configurations in our app, all the other configurations can be changed according to the current app environment (dev, staging, prod). The loggers are configured in config.py and can be overridden in the respective app environments (dev, staging, prod) as shown below 👇

# config.py
class BaseConfig(object):
    ''' Base config class. '''
    ...
    LOG_INFO_FILE = path.join(basedir, 'log', 'info.log')
    LOG_CELERY_FILE = path.join(basedir, 'log', 'celery.log')
    LOGGING = {
        'version': 1,
    ...

 
Authentication
Authentication can be added authentication as per the need of the application. To start with, this boilerplate has API-Key based authentication whereas the API-Key is declared in .env file. In file, authentication.py, a decorator looks for x-api-key header in the request and returns 401 in case of mismatch. Once you have set up the app, you can test the test route (/core/restricted) for the same via curl with your API key set in .env.

$ curl --location --request GET 'http://localhost:5000/api/v1/core/restricted' --header 'x-api-key: <your-api-key>'

For more complex authentication logic, please write a custom middleware.

Task Queues (Celery) Setup
Task queues are used as a mechanism to distribute work across threads or machines. Commonly, even a small microservice would require to run tasks in the background (async). Hence, this boilerplate comes with a pre-loaded popular task queue Celery.

The configuration is quite simple as we are using Flask's factory pattern. We have also added a separate customized logger for celery logs which writes to a separate configurable file celery.log. Running celery is as easy as just running a command 

celery worker -A celery_worker.celery -l=info (from the root of our project).

Deployment
Flask Documentation provides a very good list of deployment options and the necessary steps as well. Also, you can refer to my earlier post on Deploying the Django application which is similar for flask application as well (with some minute changes). 😄

API Caching and Validation
We already covered a lot of points but two main aspects of having a robust microservice are to have an efficient validation and caching capability. This boilerplate comes with following two packages to achieve the same:

Redis | A Python Redis client for caching
Webargs | A Python library for parsing and validating HTTP request objects
 
All set! Flask boilerplate is up and running and waiting for you to write some cool stuff from it.


Daniel Morales Perez

Author


Daniel Morales Perez


Other posts