Situation

Friday, December 22^nd, 2023 | 5:21pm

As our team is getting settled in for a great holiday weekend, our account manager receives a call. There’s been a flood and the computers aren’t working for one of our partners. No one with information is at the partner site, but the computers aren’t working and it seems to be associated with the flood. 

Solution

The Intuitive Technologies team realized that the IT systems are a critical component to success for this business during one of their busiest weeks. Coordinating efforts between our engineer and director of operations, Intuitive Technologies was able to dispatch and be onsite with our partner within 1 hour of receiving the initial call. 

Determining the extent of the damage, our team worked to identify the impacts to our partner’s operations, what options we had, and then plan the best and quickest course of action to alleviate the issue. 

Our key individuals and points of contact were out of town on holiday but were included in important updates and information along the way. These individuals knew the situation was a high-stakes situation, but creating an experience for them to relax and enjoy their time away was a priority of ours as well. This is done through strong, important and impactful communication practices as well as setting expectations. Our crisis management process went into play immediately.

1. Identify the issue causing the most pain
2. Determine how to alleviate that pain point as quickly as possible, even with a temporary solution.
3. Take action toward that goal and stabilize the environment
4. Create and develop a long term strategy and permanent fix to the problem
5. Implement the permanent fix

In the steps above, we determined the following:

1. The biggest issue: The point of sale system was down and not able to process orders which meant that the system wasn’t recording orders, nor was the kitchen staff receiving the orders of what to deliver to their customers.
2. We determined that we had options to pull a backup, attempt to repair/stabilize the existing system, investigate warranty options if they were available, or worst case scenario, rebuild an environment. We also noticed that we had limited resources available to us. The Point of Sale company support line was closed and not available after 5pm. During our time in this step, we identified that a critical component of the server that controlled the hard drives was damaged and in question. Warranty was out of the question as the system was no longer in warranty support and accidental/environmental incidents are not covered for the type of warranty on this server anyway.
3. The Intuitive Technologies engineer ran diagnostics on the system and was able to get a the server online. They then decided that the quickest option would be to export a copy of the server as is and move it to another available system in efforts to preserve real time data. The team moved quickly to begin the process of exporting the environment and then importing it to it’s new destination.
4. During this time, the long term strategy was developed and coincided with the conversion that was taking place to restore services in efforts to reduce any redundant labor. The triage migration had taken place and the point of sale was tested and operational.
5. Our permanent solution that was devised with our partner was put into place and cleanup tasks were performed in the following days after we had confirmed that everything was operational.

Outcome

Our team was able to deliver on an unexpected and unpredictable event for our partner in their time of need. Minimizing business impact and downtime, as well as eliminating as much cost as possible by curating the most efficient implementation strategy. We worked in conjunction with the partners employees for testing the environment and updated key individuals with important, impactful information in order to make sure everyone was on the same page and that those that were out of town had transparency into the situation.

Ultimately, our team was onsite with the client from 6:00pm to 4:30am and the system was operational for use the next day. Follow ups and communication continued throughout the weekend to make sure that the systems were functioning as expected and that there were no unexpected consequences or outcomes. 

Intuitive Technologies prides itself on it’s ability to respond and react quickly to IT problems that arise and escalate for all of our partners. Our priority and focus shift to the needs of our clients when the situation arises. We lead the way to get IT Systems back up and online, even while key players are away or unavailable. Our relationship with every one of our clients is based on trust. Their trust in Intuitive Technologies to do the right thing, make important decisions on our own, and do what it takes to get systems back up and online all while communicating throughout the process is what allows us to be great in the IT Services space.

‍