Lesson 8

A Social Blogging Application — Part 7: Application Programming Interfaces

Share Tweet

Contents

  1. Resources
  2. Part 7: Application Programming Interfaces
    1. REST Architecture
      1. REST resources
      2. Request methods
      3. Request and responses body
      4. Versioning
    2. RESTful Web Services
      1. API blueprint
      2. Error handling
      3. User authentication with Flask-HTTPAuth
      4. Token-based authentication
      5. Serialization
      6. Resource endpoints
      7. Pagination of large resource collections
      8. Testing APIs with Postman
    3. Conclusion
  3. Exercises
  4. Further Reading

Resources


Part 7: Application Programming Interfaces

Built on the familiar model of the World Wide Web, REST (Representational State Transfer) architecture has emerged as the favorite for web applications in recent years. In this lesson, we will upgrade the functionalities of the social blogging application to be available as RESTful web services. This lesson is organized as follows:

  • We first briefly introduce REST architecture;
  • And then, we present how to implement RESTful web services with Flask.


1/ REST Architecture

According to the Roy Fielding's PhD dissertation, there are six characteristics of the REST architecture:

  • Client-server architecture;
  • Stateless — Each request from the client to the server has to include all of the necessary information that the server is able to understand the request. And the server does not store any state about the client and of the request on the server. In other words, the server never relies on information (state) from previous requests of the client.
  • Cache — The cache can be used to store and retrieve data for optimization purposes.
  • Uniform interface — The interface terminology in this context include the protocol between server and client, resource identifiers, resource representations, self-descriptive messages between client and server, and hypermedia terminologies. And of course, they must be consistent, well-defined and standardized.
  • Layered system — Proxy servers, caches, gateways can be inserted between server and client to improve performance, reliability, and scalability.
  • Code-on-demand — Clients can execute the server's code in their context.
1.1/ REST resources

The key abstraction of information in REST is a resource. Any information that can be named can be a resource. For example, in the social blogging application, resources are users, blog posts, comments.

Each resource must have a unique identifier that represents it, and in the context of HTTP, identifiers for resources are URLs. For the social blogging application, a blog post could be represented as /api/posts/1 where 1 is the identifier for a blog post. A resource identifier can be a singleton or a collection in which a collection identifier would retrieve a collection of data such as /api/users/, and vice versa, an identifier such as /api/user/ would be considered as a singleton identifier. An identifier can be also defined as a logical subsets such as /api/posts/1/comments/ to retrieve all comments of the blog post 1. You should note that it is common to define URLs that represent collections of resources with a trailing slash.

1.2/ Request methods

HTTP defines a set of request methods to indicate the desired action to be performed for a given resource. For example, a GET request to the /api/posts/ would obtain a list of blog posts, and with that same URL but using POST method, it would mean that the client inserts a new blog post to the server's database. Fig. 1 shows some common HTTP request methods along with their purposes.

Fig. 1. HTTP request methods in RESTful APIs
Fig. 1. HTTP request methods in RESTful APIs

The GET, POST, PUT, DELETE are not the only ones. For a full list of HTTP request methods, you should refer to this document.

1.3/ Request and responses body

Do you remember the first lesson of this course? 🤔

Recap the first lesson when you used Postman software to test the programmed APIs on a local server. The resources are placed in the bodies of requests and responses, but REST architecture does not specify a specific body format for both request and response. Therefore, the Content-Type header is used to indicate the format that a resource is encoded in the body, and also for the decoding phase.

The two most common types of format used with RESTful are JSON (JavaScript Object Notation) and XML (Extensible Markup Language). Choosing the right format for your API services is very important, since you should not (and would not) change the format when releasing the APIs. Compared with XML, JSON is simpler than XML, but XML is more powerful. For common applications, JSON is a good choice since its terse semantics result in code. For applications with complex requirements relating to data exchange, the XML can significantly reduce software risk. For the social blogging application, the blog post resource could have the JSON representation as shown in Fig. 2.

Fig. 2. An example of blog post resource in JSON format
Fig. 2. An example of blog post resource in JSON format

The self_url, author_url, and comments_url are presented in fully qualified resource URLs in order to allow clients to discover new resources.

1.4/ Versioning

Take the current version of the social blogging application as an example, we can see that it is a server-centric application where all the source code and data are stored in the server-side. When the developer team makes any updates to the application, all of the clients will receive the up-to-date version of the application; in other words, the new version will be broadcasted to all of the clients.

Now, let's take a few seconds to think about the version management in API programming. And then, consider this situation: To obtain a list of blog posts, clients need to compose a request's body and send it to the /api/posts/. However, when the blog post data becomes enormous, the application would limit the number of blog posts that the server returns each time by introducing new elements in the request body such as "{"from": 1, "to": 100}". The developer team thus decides to immediately update the current API to apply this new request body; and you can see that all the current clients would receive error messages when trying to send the old version of request body to the server. The problem is that it is impossible to force all of the clients to update their client program to keep up with the new updates immediately. The question is how can we solve this situation?

Therefore, API programming needs to be more tolerant than the traditional server-centric applications and be able to work with old versions of their clients. The professional terminology for this case is backward-compatible. A common practice to address this problem is to include a versioning indicator in all URLs to acknowledge both the server and client the version they are working with. For example, the first API version of the social blogging application could expose all of its APIs via the /api/v1/…. By doing so, we do support both the old and new versions of the APIs. If we update the APIs to the new version, we just change the version indicator in the URL such as /api/v2/….

Maintaining multiple versions of the API can become complicated over time, but this is the most common (even only) approach to allow the application to grow without causing backward-incompatibility. Older versions of the application can be deprecated and removed from the server, once all clients migrate to the new version.


2/ RESTful Web Services

In this section, we will present the process to update the social blogging application to make it available as API web services. Flask makes it easy to implement RESTful web services. For instance, the route() decorator and its methods optional arguments can be reused to handle the resource URLs exposed by the service. Working with JSON in Flask is also easy, in particular, JSON data can be decoded into a dictionary format by calling request.get_json(), and a response contains JSON data can be encoded from Python dictionary to JSON format by using Flask's jsonify() function.

2.1/ API blueprint

The project structure now has a new /app/api/ directory which is a blueprint to keep all application's routes well organized. In Fig. 3, we show the general structure of the API blueprint within the application.

Fig. 3. The general structure of the API blueprint
Fig. 3. The general structure of the API blueprint

In Fig. 3, we can see that each resource is implemented in separated modules. The __init__.py is the blueprint constructor such as the ones defined in /app/auth/ and /app/main/ directories. In Fig. 4, we show the content of app/api/__init__.py. Then, as shown in Fig. 5, we need to register the API blueprint at the application level in app/__init__.py.

Fig. 4. app/api/__init__.py: API blueprint creation
Fig. 4. app/api/__init__.py: API blueprint creation
Fig. 5. app/__init__.py: API blueprint registration
Fig. 5. app/__init__.py: API blueprint registration

Recall the authentication blueprint that we presented in Lesson 4A, the API blueprint registration is just the same where we register with a URL prefix, so all of the API routes will begin with /api/v1/. By defining the prefix URL when registering a blueprint, we do not need to hardcode the version number to every route.

2.2/ Error handling

Since all requests and responses between server and clients are made based on HTTP, we can take advantage of the HTTP status code to inform the client the status of the request along with additional information attached in the response body. A full list of HTTP status codes is available here. In general, HTTP status codes can be categorized into five classes, as follows:

  • Informational responses (100 - 199),
  • Successful responses (200 - 299),
  • Redirection messages (300 - 399),
  • Client error responses (400 - 499),
  • Server error responses (500 - 599).

The 404 and 500 status codes are a little bit different and complicated than the other ones, since they are handled by Flask, and by default, Flask will return an HTML response instead of HTTP status codes. Therefore, in Fig. 6, we present the implementation of the 404 and 500 error handler. And then, in Fig. 7, we present an example of an error handler for the other HTTP status codes.

Fig. 6. app/errors.py: 404 and 500 error handlers now support both HTML and JSON
Fig. 6. app/errors.py: 404 and 500 error handlers now support both HTML and JSON
Fig. 7. app/api/errors.py: Error handler for 400, 401, and 403 status codes
Fig. 7. app/api/errors.py: Error handler for 400, 401, and 403 status codes
2.3/ User authentication with Flask-HTTPAuth

Protecting information and data from unauthorized users/clients is very important for web applications and web services. Therefore, we presented the User Authentication in the first part of implementing the social blogging application. In REST architecture, as mentioned above, one of its characteristics is stateless, which means that the server will not store any information about the client between requests. It thus requires the client to attach the user credentials together with the required information to fulfill an API request. It may raise a question, why don't we store user credentials into cookies as we have done in the User Authentication lesson. The main reason is that if we store user credentials into cookies, it will raise a problem for clients that are not web browsers.

HTTP authentication is the preferred method used to send credentials. The user credentials are included in the Authorization header with all requests. Instead of directly implementing the HTTP authentication in our application, we can take advantage of the Flask-HTTPAuth extension to wrap all HTTP authentication functions in decorators such as login_required of Flask-Login extension. Flask-HTTPAuth can be installed with pip, as follows:

$ pip install flask-httpauth

In Fig. 8, we initialize the HTTP Basic authentication based on the Flask-HTTPAuth extension in the verify_password() function. We do not need to initialize it in the application package like other extensions such as Flask-Bootstrap, Flask-Login, we just need to initialize it in the API blueprint.

Fig. 8. app/api/authentication.py: User authentication with Flask-HTTPAuth
Fig. 8. app/api/authentication.py: User authentication with Flask-HTTPAuth

The email and password are verified using the existing support in the User model. We first retrieve a User instance based on the input email; if there is a record in the database that matches the input email, we then call the verify_password() function in the User model via the retrieved User instance.

Besides the @auth.verify_password decorator, Flask-HTTPAuth extension also supports another function to return a 401 status code to the client when the authentication credentials are invalid. Therefore, we implement this decorator as shown in Fig. 8 to override the default behavior provided by Flask-HTTPAuth extension.

With the implementation of user authentication based on HTTP Basic authentication, we can now protect any routes that access sensitive data, as shown in Fig. 9.

Fig. 9. Using @auth.login_required decorator to protect route from unauthorized clients
Fig. 9. Using @auth.login_required decorator to protect route from unauthorized clients

If all the routes in the API blueprint need to be protected from unauthorized clients, we can implement the @login_required decorator in the before_request handler for the blueprint, as shown in Fig. 8. With the before_request() function has been implemented, the authentication checks will be done automatically for all the routes in the blueprint.

2.4/ Token-based authentication

Sending the credentials with sensitive information such as password for each request is not a good experience for clients, it is also a potential threat. The token-based authentication is a solution for this problem. Fig. 10 shows the implementations of the two helper functions to generate and verify authentication tokens.

Fig. 10. app/models.py: Helper functions to generate and verify authentication tokens
Fig. 10. app/models.py: Helper functions to generate and verify authentication tokens

The generate_auth_token() method returns a signed token that encodes the user's id field. The verify_auth_token() method takes a token and, if it is valid, returns the user stored in it. This is a static method, since the user will be known only after the token is decoded.

We need to update the verify_password() function to accept authentication token, along with the regular credentials, as shown in Fig. 11.

Fig. 11. app/api/authentication.py: Improve authentication verification with token support
Fig. 11. app/api/authentication.py: Improve authentication verification with token support

The improved version of the verify_password() function will now take either email address or authentication token as an argument, along with the password. If both the fields are non-empty, the function will be treated as a regular authentication. If the password is empty, then the email_or_token field is assumed to be a token and validated as such.

It is obvious that we also need to implement a function to return authentication tokens to the client, the implementation is as shown in Fig. 12.

Fig. 12. app/api/authentication.py: authentication token generation
Fig. 12. app/api/authentication.py: authentication token generation

If the value of g.token_used is True, the function get_token() will reject the request to prevent users from bypassing the token expiration by requesting a new token using the old token as authentication.

2.5/ Serialization

Serialization is a process to convert internal representations of resources from-and-to JSON format. In Fig. 13 and 14, we show how to implement methods to convert a post and a user to a JSON serializable dictionary, respectively.

Fig. 13. app/models.py: Convert a post to a JSON serializable dictionary
Fig. 13. app/models.py: Convert a post to a JSON serializable dictionary
Fig. 14. app/models.py: Convert a user to a JSON serializable dictionary
Fig. 14. app/models.py: Convert a user to a JSON serializable dictionary

The inverse of serialization is called deserialization. Deserializing a JSON back to a model such as Post has the challenge that some of the data coming from the client might be invalid, wrong, or unnecessary. Fig. 15 shows the implementation of the method to obtain a post in JSON format and create a Post.

Fig. 15. app/models.py: creating a blog post from JSON
Fig. 15. app/models.py: creating a blog post from JSON

Creating a Post instance just requires the body attribute, since the other attributes of Post model such as timestamp, author_id can be left as default values or obtained in a logical way. The error checking in this function comes from the ValidationError class in the app.exceptions package. In Fig. 16 and 17, we show the implementation of the ValidationError class and the validation_error() function, respectively. You should notice the directories where we place the files.

Fig. 16. app/exceptions.py: Define ValidationError class
Fig. 16. app/exceptions.py: Define ValidationError class
Fig. 17. app/api/errors.py: API error handler for ValidationError exceptions
Fig. 17. app/api/errors.py: API error handler for ValidationError exceptions

By separating the error handler from the logic code, the view functions can be written very cleanly and concisely. For example, in Fig. 18, we show the implementation of the new_post() function without including the error checking.

Fig. 18. app/api/posts.py: Create a new post
Fig. 18. app/api/posts.py: Create a new post
2.6/ Resource endpoints

Resource endpoint is a collection of routes that handle the different resources. The GET and POST are the two common HTTP request methods that we will employ to implement the RESTful web services for the social blogging application. Fig. 19 shows the two GET handlers for blog posts. Fig. 20 shows the POST handlers for inserting a new blog post to the database.

Fig. 19. app/api/posts.py: Functions employ GET method
Fig. 19. app/api/posts.py: Functions employ GET method
Fig. 20. app/api/posts.py: Function employ POST method
Fig. 20. app/api/posts.py: Function employ POST method

The view function shown in Fig. 20 is wrapped in a permission_required decorator to ensure that the authenticated user has the permission to create new blog posts. The implementation of this decorator is as shown in Fig. 21.

Fig. 21. app/api/decorators.py: Implement permission_required decorator
Fig. 21. app/api/decorators.py: Implement permission_required decorator

Besides the GET and POST methods, PUT method is also employed to update/edit existing resources. In Fig. 22, we show an example of a PUT method to edit a blog post.

Fig. 22. app/api/posts.py: Function employ PUT method
Fig. 22. app/api/posts.py: Function employ PUT method

In Fig. 23, we list a set of APIs which interact with resources such as post, comment, user, along with the corresponding HTTP request methods.

Fig. 23. Set of APIs
Fig. 23. Set of APIs
2.7/ Pagination of large resource collections

Handling a large collection of resources can be expensive and difficult to manage and transfer, we thus employ the pagination mechanism like what we have done in Lesson 5. Fig. 24 shows an implementation of pagination for the list of blog posts.

Fig. 24. app/api/posts.py: Post pagination
Fig. 24. app/api/posts.py: Post pagination
2.8/ Testing APIs with Postman

The content of this section is being composed.


5/ Conclusion

This lesson completes the series of lessons that demonstrate how to implement the social blogging application with Flask framework. In the next lesson, we will focus on how to deploy the application in a production environment.

🎉 🎉


Exercises

  1. Develop testcases or test scripts to evaluate the demo application.

Further Reading

  1. [Apple Podcasts] Why Digital Transformations Need APIs

Relaxing ☕