A Foundations lesson, about 12 minutes. Keep the retries and stop the storm, with backoff, jitter and a budget.
Before the fix: p99 182ms, 0% of requests failing, 60/s answered. After it: p99 182ms, 0% of requests failing, 60/s answered.