Asynchronous and unpredictable results all the time
Shall your IT system always produce the same result for the same request? For quite a lot of people that is kind of the one/zero way of thinking about IT systems. You will always get exactly the same answer to the same request, since it is a computer that answers.
But is that a good way of designing an IT system? Sometimes it might be, but in most situations you should really consider the options.
And those situations are occurring more and more frequently in today's world of integrated systems, cloud services, microservices and general complex IT solutions that combine functionality from multiple underlying systems.
If you, in this complex reality always rely on every system to be available and supply timely responses you will get a lot of problems including slow solutions and a low availability.
Instead we need to design for graceful degradation and robust solutions that always will produce a timely result - and yes, it might not always be the optimal result - but it will be a result and most of the times it will be good enough.
A system, for example a travel booking system, that process a user request to find available hotels from multiple suppliers and displaying them on a map. If that system sends queries to ten different hotel chains/systems you would probably consider doing that asynchronously and displaying them one by one when you receive the result. You might even go as far as consider displaying a list instead of a map if your map provider doesn't respond. This might of course mean that the most fitting rooms weren't displayed, but the customer hopefully went ahead and made a booking anyway since good options where available and you did keep your customer.
When doing a similar design in your web shop in order to display shipping methods it might not be as natural to exclude some options just because a service isn't responding in order to display the accurate shipping cost for both UPS and FedEx.
Doing orchestration in an enterprise system when processing those web shop orders it is very common to halt everything if one minor service for sending an SMS notification doesn't work, and resulting in delaying the shipping for a day or two. Just because the BPMN model is synchronous and nobody noticed for a couple of hours and the resulting back log led to very slow processing while overloading the backend database and might even affect the web shop since some resources were shared. Is that a good design choice?
The key word and recommendation here is embracing being asynchronous, and consider this for all situations. Even for those enterprise systems, and the reason is that the reality of services is everywhere, services with various SLA, services from various supplier, services running everywhere on various clouds really is here or will be there. Monolithic systems with predictable responses is a thing of the past. But, yes, it is usually so much easier to think about architecture in a synchronous was since algorithms, and computer program in general are synchronous. But please stop and don't.
Evaluating the quality of different responses, the lack of response and doing all calls in an asynchronous way and considering the importance of every service can be hard work. But it is hard work that needs to be done.
Embrace asynchronous service calls!
And design systems and processes that can work in an asynchronous way and will degrade gracefully.