Handling Service Timeouts using Istio


The Timeout Design Pattern states that you should not wait for a service response for an indefinite amount of time – you should rather throw an exception instead of waiting for too long. This will ensure that you are not stuck in the limbo state continuing to consume application resources.

In my earlier blog post, I discussed the fallacies of Distributed Computing and how service calls made over the network are not reliable and might fail —
Eight Fallacies of Distributed Computing

Service calls made over the network might fail. There can be congestion in network or power failure impacting your systems. The request might reach the destination service but it might fail to send the response back to the primary service. The data might get corrupted or lost during transmission over the wire. While architecting distributed cloud applications, you should assume that these type of network failures will happen and design your applications for resiliency. Implementing a timeout strategy ensures that your microservices are resilient and available.

Istio Timeout

When you encounter latency with your service communication and you are not sure what the root cause is, it is a preferred approach to just not wait for the response. Implementing a timeout strategy between your service to service communication over the network is critical. Istio makes it pretty simple to implement this functionality within your service mesh.

Setting Request Timeouts

In Istio you can set request timeout either by using the route rules explained below or on a per-request basis by adding a header entry in outbound requests. You can mention the timeout in milliseconds on the header.

By default, the timeout for HTTP requests is disabled, but it can be overridden by using a routing rule as below —

To test the timeout functionality you can inject a delay of say 10 sec between your services. You can verify that when you invoke the service endpoint there is a delay of 10 sec to get a response.

Next step is to configure the timeout rule to introduce a timeout period of say 3sec. Now you will see that the service call timeouts after 3sec and ddoes not wait for 10sec to get a response back.

If you want to quickly try out this capability without having to download the required tooling and installations, I would highly recommend you to try out the interactive platform right from your browser using Katacoda —
https://www.katacoda.com/courses/istio/increasing-reliability/handling-request-timeouts

Katacoda Training.png



Categories: Architecture, Istio, Microservices, Service Mesh

Tags: , , ,

2 replies

Trackbacks

  1. Chaos Testing your Microservices with Istio – dotnetvibes
  2. Speaking at O’Reilly Software Architecture Conference 2019 – dotnetvibes

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: