Strategy

Tips for Designing a RESTful API – Part 3

Even if you are an initiate to software development, chances are you’ve worked with an Application Programming Interface (API). For those outside of the tech world, many are using and/or benefitting from an API whether they realize it or not. Indeed, the interoperability of everything around us is achieved in large part due to an API someone made, and its ability to communicate with other systems. Just like anything in software development, creating an API for someone to consume can be challenging. You run many kinds of risks, such as creating something counterintuitive or difficult to use. In this three part blog post, I’d like to discuss some of the important points to take into consideration while designing a RESTful API, offer my advice, and discuss some mistakes I’ve made in the past while involved as an architect in API design.

Improving your API through ODATA

In my first post, when discussing querystrings, I mentioned how ODATA can helpful in returning more specific results. When you have to handle issues like pagination or filtering (let’s say you only want results 30-45 from the entire employee directory, or only employees named ‘Michael’) ODATA comes to the rescue. ODATA is an open protocol to allow the creation and consumption of queryable RESTful APIs in a simple and standard way. It helps you focus on your business logic while building RESTful APIs without having to worry about the approaches to define request and response headers, status codes, HTTP methods, URL conventions, media types, payload formats and query options, etc. OData also guides you about tracking changes, defining functions/actions for reusable procedures and sending asynchronous/batch requests, etc. Implementation details such as filtering, sorting, and querying are all handled for you so that you do not need to introduce custom logic to introduce this into your API. There are frameworks for ODATA in many popular programming languages.

ODATA Sorting & Pagination

$Top, $Skip, and $Orderby

When you get back a result set, ODATA allows the consumer of your API to append query string parameters to retrieve results by certain criteria. You can order your result set and take or skip certain items to achieve pagination. Below are some examples to show how the parameters need to be passed. The ODATA specifics are highlighted in red.

ODATA Pagination

$Filter

In addition to sorting and pagination, you can choose what you would like to get back from an API call with $filter (I only want back results that match something, like a price being < 100, for example).

Filtering in ODATA

As you can see, there are many complex things that can be performed including arithmetic and string operations. This of course can lead to very complex ODATA queries but the advantage here is that instead of having to write all of this logic yourself, you can import an ODATA library into your project, wire your controllers up to support ODATA requests, and then let it take care of everything for you. In most implementations (for example in .NET) there are many ways to customize what is supported – it doesn’t have to be all or nothing. This is important if, say, you do not want the employee list to support ODATA operations, or if you want to setup general rules such as filtering not being supported. There are many other types of ODATA queries that can be constructed, such as selecting specific columns back from a result set (if your employee entity has 20 properties being returned, you could just select 2 properties). Check out the ODATA Getting Started guide for more information.

HTTP Response Codes

HTTP Response codes allow the consumer of your API to understand what happened as a result of their request. I’d like to touch on the more important ones to consider when building an API.
As a general rule, 2xx (such as 200, or 201) is a good sign that something went right. 3xx means something was redirected (maybe the resource existed at this location, but no longer), but is also used for caching an entity tags (which I will aim to cover in another post). 4xx means the client did something wrong or didn’t pass the correct information to fulfill the request. 5xx means the server had an error or was unable to process the request (an issue on the API side, not the client side). If you’re building the API from scratch, it’s good to stay to with what’s popular since many consumers will have a certain level of expectation about your API.

200 OK, 201 Created

200 OK is the most common response code, and indicates the call was successful. It is used when you GET a resource and it comes back successfully. I recommend you use 200 for GET requests (which will probably be default in most technologies). For creating a resource, you should use a 201 response (Created) which indicates the resource was created successfully. For a PUT (to update an entity) I have seen 200 become the standard, even though other codes such as 204 (No Content) have been used.

3xx Codes

3xx codes mostly deal with a resource no longer existing at the asked-for location; if you want to simply tell the consumer something no longer exists or exists elsewhere. This sort of redirection is also seen for SEO-related purposes, for example if you want to redirect a crawler to the new home of the resource. These codes are also used for another interesting topic that I want to make a separate blog post out of called Entity Tags. The 304 – Resource Not Modified response informs the consumer that this resource has not changed since the last time you asked for it. This can be used if you specify a tag in the request. The server will then examine that entity based on that tag, and see if it has changed. The thought is if the tags don’t match (because the server updates the tag each time the entity is modified) then the server knows the requester had an older version.

401 Unauthorized

401 Unauthorized is used to tell the consumer that they did not provide credentials (or the credentials were incorrect) to access the endpoint requested. Let’s say, for example, I make a call to get the weather for a city. That is public and it would return a 200, presumably with my JSON data for the weather. If I tried to access a protected endpoint, however, like /api/users/{guid}/homework, I would get a 401 if the server requires credentials (which I hope it would)!. So let’s say I pass a JSON Web Token (JWT) that was issued to me. Assuming this is the correct resource that I have access to, I should then be able to access it successfully and get back a 200.

403 Forbidden

403 Forbidden Sometimes confused with 401, this response code indicates that I cannot access this resource due to who I am. The important difference here is that while I might be authorized to make a call to the server/endpoint, the particular resource I requested is not something I have access to. In the earlier example, if my ID is 2c38575a-b57f-4334-b60f-d9842aacd07f, then /api/users/2c38575a-b57f-4334-b60f-d9842aacd07f/homework is OK for me and I can retrieve it. But /api/users/991faa92-e46b-45e1-bc63-3308252a3fb9/homework should return a 403 because 991faa92-e46b-45e1-bc63-3308252a3fb9 is not my user id. This is obviously based on business logic, of course, since I might be an admin or supervisor who has access to certain kinds of users. So you could imagine 403 codes can be role-driven.

404 Not Found

404 Not Found is used to indicate that a resource requested was not found. While we are accustomed to seeing this on webpages that do not exist, from the REST perspective what this means is that the entity you requested does not exist. If someone queried your API for GET /users/1 they would get back the user with a 200 if it existed. If they then tried GET /users/9999 this would return a 404 if it did not exist. 404 is standard for many API frameworks and is something you come to expect when a resource doesn’t exist.

405 Method Not Allowed

405 Method Not Allowed is a good way to tell someone that while the action they are trying to take is valid, the method is not accepted for this action. In the above example, I retrieved a user with /users/1 and one could assume PUT would also be valid to update that same user. But would POST? Now obviously this depends on the API but more likely than not you wouldn’t create a user with /users/1. So in this case, a 405 would be returned to indicate to the consumer that POST is not permitted.

500 Internal Server Error

500 Internal Server Error is used to indicate an error occurred on the server. When code runs and something crashes, 500 is returned. It ends up being a catch-all for errors and is useful when combined with some sort of a message indicating what went wrong. There are many ways to hook into these errors, depending upon the programming language, and format them to be more specific to the problem. By default, many API frameworks tend to just respond with the exception which is a little too technical for the user. It can also pose a security risk when it exposes your code/line numbers of the error.

Popular REST Clients

Fiddler

Fiddler is a web debugging proxy and all-around awesome tool that I recommend you check out. I primarily use it to call my APIs but on occasion have been able to use the proxy end of it to “fake” responses back to my client when certain endpoints aren’t available yet. It has a great means of archiving requests/responses, and unlike other clients, you can examine the response over and over without having to reauthenticate or make a fresh call. I also appreciate how easy it is to share these calls with other team members and have them replay the same calls. For tech-savvy QA testers, this is amazing because they can run a bunch of Fiddler calls to see what passes/fails, and observe the response.

Postman

Postman is a more lightweight tool to call your APIs. While it doesn’t have the proxy features that Fiddler does (for example I can’t halt a web request, and tamper with the values before sending along), the reality is most of the time you aren’t going to need that much sophistication if you want to just test your API. It allows you to save your calls in the same way Fiddler does, and has a cloud-based approach to sharing calls. You can create groups and share with others in a collaborative way. The layout is a little more straight forward for making calls as well. The pro version offers data synchronization and more collaboration, but not much else by way of features.

API Documentation

While every good software developer should document their code, I’ve found it equally as important to have a good technical blueprint of the API itself. This documentation should include what endpoints are available, the types of verbs to use, and information on the request/responses. When creating an API of any size, having this documentation is critical.

I’ve worked on some large APIs that had 200+ routes, and it was really difficult for me to go into the code, find the right controllers, and then work through the whole pipeline to get to what I needed. Fortunately, we had great documentation that was far easier to navigate, and so that was where I went. When we onboarded new developers, the first thing I’d have them do is start becoming familiar with the API documentation. Having that leg up when they begin development is critical.

Swagger

Swagger boasts the world’s largest framework of API developer tools for the OpenAPI Specification(OAS), enabling development across the entire API lifecycle, from design and documentation, to test and deployment. There are many API documentation tools out there, and I can’t say I’ve worked with many, but Swagger is absolutely great. From the strategy side, it has visual tools to plan and design either new or existing APIs. You can then turn your designed OAS/Swagger definition into code, generating server stubs and client libraries in over 40 different languages. To document your API with Swagger, you use a plugin matching your programming language, found here to forward generate a specification by analyzing your API. This generated specification can then be utilized by Swagger UI to create an interactive, visual representation of your API. Check out their demo to see it in action. You can make calls in real time to your API, alter the requests, analyze the responses, and more. As you could imagine, having a reference like this is a great way to bring on a new developer to or help you track down the specifics of a route.

Final Thoughts

Designing a good API can be challenging. Your team will make many mistakes, and there will be regrets for some of the decisions that were taken in the architecture. Hindsight is always 20/20 so try not to let it bother you too much. Strike the right balance between future-proofing, and only developing for what is needed right now. Too much in either direction can be a bad thing. Some tips to help you achieve success:

  1. Consistency – Your API should follow a certain pattern and avoid deviating whenever possible. Your naming conventions, style of handling errors, interaction of controllers to service classes and entities, and the list goes on, should all be consistent. This consistency is good for the consumer of the API because they have a certain expectation about how it will work. The consistency is good for the developer because they understand how the codebase works, and have a pattern to follow when doing their own development. Working on teams can be both helpful and challenging for API design. Challenging because everyone may have an idea on how to do something. Furthermore, when development starts, they might start doing something in a way that’s different from the rest of the team. It’s helpful to have others to bounce ideas off of, but ultimately you need to agree on a consistent approach to the design. Involve the team for important decisions, and make sure changes are well-communicated.
  2. Auditing/Logging – When an API is deployed in a production environment, it’s much more challenging to find out what went wrong when an error occurs. Think about this before you go to production. When a random exception/500 occurs, where is it logged? Is it going to provide you enough actionable information to reach a resolution? Now think about multiple versions of your API. Without any logging it can be a real challenge to reproduce the error for a certain version.
  3. API Design Meetings – Have meetings with your team whenever important decisions arise. I’ve had meetings just to talk about naming conventions for controllers. Too many meetings can be aggravating, so make sure you hit the right level of group discussion and just plain leading by example.
  4. Code Review – This should be done as frequently as possible. Every check-in, at some point, should have some eyes on it. This is not only a good way to catch mistakes, but also to ensure your naming conventions are enforced, and the agreed-upon style is being adhered to. As an example, I was doing some review once and saw we had one controller with 20 actions in it. Everyone was so busy so it just seemed like a great place to put all the work. It was becoming far too bloated. I setup a meeting with our team, and we ended up moving out more than half of the methods into 2 additional controllers. Everything looked clean and made much more sense afterwards. A few sprints later, our work paid off even more because we had new homes for some upcoming code. No more god classes/controllers!
  5. Documentation – As I mentioned this is very key to staying organized. It’s useful for your development team and a great way to onboard new developers. If the API is public then it’s something you’re going to need anyway to describe how to interact with your system. The team should always document their code in general; there are ways to have a code analysis run on each build or commit so that you can enforce this. Finally, developer notes can be useful in providing some thoughts on why a particular decision was taken. Many project management software comes with the ability to add notes and link them to stories worked on. At some point in the timeline, you’ll end up going back to something to see how it works.
Add Comment...

Your email address will not be published.
Fields marked * are required

About us

We are experts in creating dynamic solutions in web and creative disciplines that enable our clients to realize broad success. Our desire for innovative and usable solutions drive a passion of continual improvement that leads to a constant stream of more intuitive and streamlined results.

Visit

Search

Social