How to Develop Resilient Software

Hi Folks,

In this article we’ll talk a bit about how to build resilient applications.

There are some times when the services your app depends on may not be 100% reliable. E.g. a database may fail to respond at one point, but when you try to access it again a few seconds later it starts responding. It could also be a web service that may not respond at one time but may start responding again after a few seconds.

Why do these happen? Sometimes it could be due to the database or service being updated. Sometimes it could be due to other issues like a timeout due to performance issues on the service. There could be other reasons.

Ideally we would like the services our apps depend on to work 24/7, but that’s not always possible and it’s often not in our power to make them work all the time. So what can we do?

Do we just give up and throw an error for the end user? No. We can do better.

A better approach is to build our own app in a way that accommodates the temporary outages in the services we depend on. One way to do this is by retrying operations. For example, let’s say we try to access a database. If the database fails to respond as expected, we try again. This makes our application more resilient. There are sophisticated techniques for implementing retry to avoid overwhelming the services we depend on. So, it’s best to use a library that is designed for resilience.

If your application is written in Python, you can use the retry library to implement retry strategies easily. This is an easy to use retry decorator that has no external dependencies.

If you’re developing a .NET application, an excellent resilience library is Polly. Polly can be used together with database libraries like Dapper ORM to implement database retry strategies.

So, in summary, when developing an application, if you realise the application depends on some services or database that sometimes fails to respond temporarily, retry the operation automatically using a good resilience library like retry, Polly, etc, depending on the programming language you’re working with. Till next time, happy software development.