So you think you can be resilient? Cool, me too ;) Anyways, as a backend developer, I am always looking for errors or problems that could cause a process to fail. Not only that, I even try to deal with the problem and make the software robust enough so it can handle these problems and automatically recover. This is called resiliency.

Recently I have been looking into the resiliency of one of my (hobby) Angular projects and found that the HTTP requests it sends to my backend easily fail for various reasons. For example, when the user works on a mobile device the connection may be dodgy. I am an Angular enthusiast, but to be honest, I am pretty confident this solution is not limited to Angular.

Retry & Circuit Breakers

I solved the problem using the retry pattern and this pattern is quite easy to understand. When an operation fails, just go and retry it. Although that sounds easy, it is not. Because you need do need to investigate why something went wrong. For example, when a server is exhausted and fails your request for that reason, you are making the problem worse when you keep on hammering that server with retries. For that reason, the retry pattern is often combined with the circuit breaker pattern that allows you to break the operation for a certain amount of time and stop your system from hammering that server. For this solution, I only implemented the retry pattern. But, I did look thoroughly into the responses and only retry when the server returns a retryable error.

For example, when the server returns a 401, 403, or 404, you are either not authorized, authenticated or the resource you are looking for is not found. Retrying the error doesn’t make sense because you will (pretty sure) get the same error. On the other hand, 408 (a timeout) or 500 (internal server error) are considered to be retriable errors and are therefore retried up to three times.

Take a peek at the code below:

return next.handle(copiedRequest).pipe(
  retry({
    count: 3,
    delay: (err, retryCount) => {
      switch (err.status) {
        case 408:
        case 500:
        case 502:
        case 503:
        case 504:
          return timer(1000 * (retryCount * retryCount));
        default:
          return throwError(() => err);
      }
    },
  }),
  catchError((error: HttpErrorResponse) => {
    if (error.status === 0) {
      // Server not responding, connection lost?
    }
    return throwError(() => error);
  })
);

This is where Angular handles the HTTP request and I sort of break in to the system and insert a retry that comes from rxjs. This retry expects a count (how many times should the request be retried before it fails), and a delay. For this delay, you can enter a number (in milliseconds). When you do, the retry will be delayed for that amount of milliseconds. The code above is a little bit more sophisticated. Here, I get the HTTP error and investigate the status code of the HTTP response. When the status code is a retriable error, I return a timer that exponentially increases. This is called an exponential backoff and is again, built in to prevent from over-asking the remote server.

Interceptors

In Angular, there is a concept called Interceptors. You can take advantage of these interceptors to break in to your HTTP request. This is really handy when you, for example, want to inject tokens into your request, or for example make sure your requests are resilient. My interceptor now looks like this:

@Injectable()
export class ResilientInterceptor implements HttpInterceptor {
  intercept(
    request: HttpRequest<any>,
    next: HttpHandler
  ): Observable<HttpEvent<any>> {
    return next.handle(request).pipe(
      retry({
        count: 3,
        delay: (err, retryCount) => {
          switch (err.status) {
            case 408:
            case 500:
            case 502:
            case 503:
            case 504:
              return timer(1000 * (retryCount * retryCount));
            default:
              return throwError(() => err);
          }
        },
      }),
      catchError((error: HttpErrorResponse) => {
        if (error.status === 0) {
          // Server not responding, connection lost?
        }
        return throwError(() => error);
      })
    );
  }
}

Last modified on 2024-02-18