The Catch Block #60 - Lessons from a Botched Deployment

The hell weeks continue. But at least we learned something. 2.5 things, in fact.

Also in this issue: Pwned Passwords, TypeScript 4.3, technical debt, ConnectFour, hot reload in .NET, and more.

2.5 Lessons from a Botched Deployment

I wrote in the last issue about our botched deployment that has been screwing up my last three weeks of work (my "hell weeks"). We're coming to the end of this madness, and in all the dust and all the bugs and all the insanity, we actually did learn a couple of things.

1. Don't be afraid to over-communicate

This is one of the things my team and I did well, if I do say so myself: we kept our customers in the loop on what we were doing. To be frank, we probably kept them in the loop more than was necessary (at one point, I was telling them individual steps I was taking to resolve their problems, on a minute-by-minute basis), but IMO it's better to over-communicate rather than under-communicate in these situations.

Botched deployments such as these shake the trust between customers and developers. Communication helps to minimize the damage, and then to repair the connection between programmer and user. By telling them that we were taking concrete, understandable steps toward fixing their problems, and then actually doing what we said we were going to do quickly and correctly, we showed them that:

• There were issues that needed to be resolved quickly.
• Those issues were our responsibility to fix (even the ones that weren't our direct fault).
• We took our responsibility to fix seriously.

Too often, the first response programmers have to situations like this is to dive into the code and fix the problem, then resurface only when they have found the solution. Don't get me wrong, you do need to actually fix the problem, but your first responsibility is to ensure that the users know what you're doing about the problem, or failing that, know when a fix will be available. Communicate first, and code second.