fear
Never Panic!
Whether it is an unexpected problem at an inconvenient time or an all-out crisis, panic is never a good way to solve problems, regardless of your position or role. It is especially bad when you are in charge of people or brought in for your expertise. Fear is a distraction, and panic leads to a myopic view of the problem. That hinders creativity and fosters dread, and those are not the ingredients for a solution.
The point in my career when this became readily apparent was when I was working for a small software company. We had a new product (Warehouse Management System) and were launching our third deployment. This one was more complicated than the rest because it was for a pharmaceutical company. In addition to requirements like refrigeration and lot control, there was a mix of FDA-controlled items requiring various forms of auditing and security, and storage areas significantly smaller than previous installations (think very small packages of medicine versus pallets of one product). It was a challenge, to be sure.
A critical component, “Location Search,” failed during this implementation. About 10-12 people were in the “war room” when my boss, the VP of Development, began to panic. He was extremely talented and usually did an excellent job, but his reaction negatively affected everyone else in the room. The mood quickly worsened.
I jumped in and took over because I didn’t want to be stuck there all weekend, and because I wanted this implementation to succeed. I asked my boss to go out and get a bunch of pizzas. Next, I organized a short meeting to review what we knew and what differed from our prior tests, and I asked for speculation about the root cause. The team came up with two potential causes and one potential workaround. We organized everyone into three teams and attacked each item independently and in parallel.
We identified the root cause, which led to an ideal fix a few days later, and a workaround that let us finish user acceptance testing and go live the following day. A mindset shift fostered the collaboration and problem-solving needed to move forward.
But this isn’t just limited to groups. I was a consultant at a large insurance company on a team redesigning their Risk Management system. We were using new software and hardware, and wanted to ensure the proper environment variables were set during the Unix login process for this new system. I volunteered to create an external function executed as part of the login process. To keep the code clean, I added an “exit” at the end of the function. It worked well during testing, but once it went into production, the function immediately logged people out as they tried to log in.
As you can imagine, I had a sinking feeling in my gut. How could I have missed this? This was a newer system deployed just for this risk management application, so no other privileged users were logged in at the time. Then, I remembered reading about a Unix “worm” that used FTP to infiltrate systems. The article stated that FTP bypassed the standard login process. This realization allowed me to FTP into the system and delete the offending function. In less than 5 minutes, everything was back to normal.
A related lesson learned was to make key people aware of what happened, noting that the problem was resolved and there was no lasting damage. Hiding mistakes kills careers. Then, we created a “Lessons Learned” log, with this as the first entry, to encourage people to share mistakes and lessons learned in order to avoid them in the future. Recognizing that mistakes can happen to anyone helps people plan better and keeps them from panicking when problems occur.
Staying calm and focused on resolving the problem is a much better approach than worrying about blame and the implications of those actions. And most people appreciate the honesty.
As the novelist James Lane Allen stated, “Adversity does not build character; it reveals it.” Maintaining your composure and sense of calm (“equanimity”) makes chaos more manageable and helps develop resilience within the team. That is key to thriving in any high-pressure environment.
