Latest Event Updates
Never Panic!
Panic is not a good problem-solving tool, regardless of your position or role. It is especially bad when you are in charge of people or brought in for your expertise. Panic leads to a myopic view of the problem, hindering creativity.
The point in my career when this became readily apparent was when I was working for a small software company. We had a new product (Warehouse Management System) and were launching our third deployment. This one was more complicated than the rest because it was for a pharmaceutical company. In addition to requirements like refrigeration and lot control, there was a mix of FDA-controlled items requiring various forms of auditing and security and storage areas significantly smaller than previous installations. It was a challenge, to be sure.
A critical component, “Location Search,” failed during this implementation. About 10-12 people were in the “war room” when my boss, the VP of Development, began to panic. He was extremely talented and normally did an excellent job, but his reaction negatively affected the others in the room. The mood quickly worsened.
I jumped in and took over because I did not want to be stuck there all weekend and mostly because I wanted this implementation to succeed. I asked my boss to go out and get a bunch of pizzas. Next, I organized a short meeting to review what we knew and what was different from our prior tests and asked for speculation about the root cause of this problem. The team came up with two potential causes and one potential workaround. Everyone was organized into three teams, and we began attacking each item independently and in parallel.
We identified the root cause, which led to an ideal fix a few days later and a workaround that allowed us to finish the user acceptance testing and go live the following day. A change in mindset fostered the collaboration and problem-solving needed to move forward.
But this isn’t just limited to groups. I was a consultant at a large insurance company on a team redesigning their Risk Management system. We were using new software and wanted to be sure that the proper environment variables were set during the Unix login process for this new system. I volunteered to create an external function executed as part of the login process. Trying to maintain clean code, I had an “exit” at the end of the function. It worked well during testing, but once it was placed into production, the function immediately logged people out as they attempted to log into the system.
As you can imagine, I had a sinking feeling in my gut. How could I have missed this? This was a newer system deployed just for this risk management application, so no other privileged users were logged in at the time. Then, I remembered reading about a Unix “worm” that used FTP to infiltrate systems. The article stated that FTP bypassed the standard login process. This allowed me to FTP into the system and then delete the offending function. In less than 5 minutes, everything was back to normal.
A related lesson learned was to make key people aware of what happened, noting that the problem had been resolved and that there was no lasting damage. Hiding mistakes kills careers. Then, we created a “Lessons Learned” log, with this as the first entry, to foster the idea of sharing mistakes to avoid them in the future. Understanding that mistakes can happen to anyone is a good way to get people to plan better and keep them from panicking when problems occur.
Staying calm and focused on resolving the problem is a much better approach than worrying about blame and the implications of those actions. And most people appreciate the honesty.
As the novelist James Lane Allen stated, “Adversity does not build character; it reveals it.”
Understanding the Real Issue using Root Cause Analysis
Too often people, including Consultants, spend time trying to solve the wrong problem due to having incomplete or incorrect information. Once, I was investigating a series of performance problems and unplanned outages that were assumed to be two separate problems. As I gathered information, several people provided anecdotal stories of anomalous behaviors in various systems, speculation about the “real problem,” and discussions about “chasing ghosts” during previous attempts to resolve the problem.
I remember stating that I was there to solve a real problem having a serious negative impact on production and that it was not my intent to chase ghosts or do anything else that would unnecessarily waste time. Next, I outlined the approach I would use to make a Root Cause determination and that we would reconvene to discuss the real problem and potential solutions. A few people scoffed and felt this was a waste of time and money.
The process followed was simple, structured, and logical. It took everything that was known to be true and mapped it out. I looked for patterns, commonalities, and intersections of systems and events. Within two days, my team and I had identified a complex root cause involving multiple components, which we demonstrated would reliably reproduce the symptoms that our client was experiencing. From there, we worked with their teams to make minor network changes, system configuration changes, and several small application changes.
By the end of the second week, they were no longer experiencing major slowdowns or unplanned outages. Each outage cost this company tens of thousands of dollars in lost sales due to the time-sensitive nature of their product. Within one week, they had recovered the cost of hiring me and my team. What stuck with us was how many really smart people “believed in ghosts” and failed to focus on the information they already had.
A few years later, we created a white paper to potentially help others needing a simple structured approach. Below is a link to that white paper written by one of the top people on my team. We received very positive feedback then, so it seemed that this could still be useful today. Please take a look and let me know what you think.
Innovations “Iron Triangle”
The concept of an Iron Triangle is that along each side of the triangle is one item constrained by the items from the other two sides. In Project Management, this is often referred to as a triple constraint. This identifies the fundamental relationships (such as Time, Cost, and Scope in Project Management) without addressing related aspects such as Risk and Quality. It provides a simple understanding of both requirements and tradeoffs.
Yesterday I spoke with Dave Mosby, an impressive person with an equally impressive background. He related Innovation to Fire, noting that in order to create fire, you need fuel, oxygen, and heat. He added that they must be in the right combination to achieve the desired flame. What a brilliant analogy.
Dave stated that for Corporate Innovation to succeed, you need the proper balance of Innovation, Capital, and Entrepreneurship. I found this enlightening because his description substituted “entrepreneurship” for “culture” in my mental model. While the difference is subtle, I found it to be important.
As noted above, simplified frameworks do not provide a complete understanding. But they help understand and plan around the foundational items required for success. Mapping this to past experiences, I gained a better understanding of things that did not move forward as desired and what I could have done differently to be more effective.
One idea was to create a fault-tolerant database using Red Hat’s JBoss middleware. We had a Services partner willing to create a working prototype, tune it for performance, document the system requirements and configuration, and package it for easy deployment. They wanted $10K to cover their costs.
I did not hold a budget at the time, so I created a purchase request supported by a logical justification. It modeled potential revenue increases for database subscriptions based on the need for a failover installation and growth from potential expanded use cases. This was a slam dunk!
In my mind, this was simple as it was “only $10K,” and I had funded many similar efforts when I had my own company. But that’s the rub. I viewed these efforts as investments in understanding, lessons learned, and growth. Not every investment had a direct payoff, but nearly each had an indirect payoff for my company. It was an entrepreneurial mindset that accepted risk as something required for rewards and success. I now see, many years later, how reframing my proposal as a way to foster innovation and entrepreneurship could have been far more effective.
It is never too late to gain new insights and lessons learned. A slightly different perspective on an important topic provided the understanding that should help position projects for future success. And this flowed from a discussion with an interesting person who has “been there, done that” many times.
Using Themes for Enhanced Problem Solving
Thematic Analysis is a powerful qualitative approach used by many consultants. It involves identifying patterns and themes to better understand how and why something happened, which provides the context for other quantitative analyses. It can also be utilized when developing strategies and tactics due to its “cause and effect” nature.
Typical analysis tends to be event-based. Something happened that was unexpected. Some type of triggering or compelling event is sought to either stop something from happening or to make something happen. With enough of the right data, you may be able to identify patterns, which can help predict what will happen next based on past events. This data-based understanding may be simplistic or incomplete, but often it is sufficient.
But people are creatures of habit. If you can identify and understand those habits and place them within the context of a specific environment that includes interactions with others, you may be able to identify patterns within the patterns. Those themes can be much better indicators of what may or may not happen than the data itself. They become better predictors of things to come and can help identify more effective strategies and tactics to achieve your goals.
This approach requires that a person view an event (desired or historical) from various perspectives to help understand:
- Things that are accidental but predictable because of human nature.
- Things that are predictable based on other events and interactions.
- Things that are the logical consequence of a series of events and outcomes.
Aside from the practical implications of this approach, I find it fascinating relative to AI and Predictive Analysis.
For example, you can monitor data and activities proactively by understanding the recurring themes and triggers. That is actionable intelligence that can be automated and incorporated into a larger system. Machine Learning and Deep Learning can analyze tremendous volumes of data from various sources in real-time.
Combine that with Semantic Analysis, which is challenging due to the complexity of taxonomies and ontologies. Now, that system more accurately understands what is happening to make accurate predictions. Add in spatial and temporal data such as IoT, metadata from photographs, etc., and you should be able to view something as though you were very high up – providing the ability to “see” what is on the path ahead. It is obviously not that simple, but it is exciting.
From a practical perspective, keeping these thoughts in mind will help you see details others have missed. That makes for better analysis, better strategies, and better execution.
Who wouldn’t want that?
Interesting Article about ADHD and Creativity
This “pocket” story is from Scientific American, originally published on March 5, 2019. The Creativity of ADHD.
Over the years, I have found that some of the most interesting, creative, and effective CEOs have ADHD-like tendencies. Strangely, they may not even be aware that they have it. Hyperfocus can be incredibly effective when attached to a driven person.
Below are links to a couple of posts that make these concepts and their benefits tangible:
These are examples of why it is best to look beyond labels, lay your preconceived notions aside, and explore each individual’s potential to contribute. The best managers and leaders tend to have this ability.
And if you want to take that a step further, let it guide you in finding the best approach to teaching, coaching, and motivating your team members. It may take a little extra effort, but the results are amazing.
Where Ideas, Experiences, and Lessons Learned Intersect 