Sunday, January 24, 2021

Company Insists On Server Not Being Fixed, Massive Fallout Ensues


Some people and companies really are their own worst enemies. In this particular tale of tech support absurdity we get to sit back and witness the inevitable fallout of a company insisting that their faulty server shouldn't be fixed. This story also shows the importance of getting everything in writing. Those emails can end up being very helpful when it comes to keeping the incompetent folks accountable for their self created failures. 

Check out some more tech support glory with this man who insisted that he knew better than IT and then learned otherwise. 

1.

Blue - r/talesfromtechsupport + Join u/MorpheusJay • 1d 3 3 S 7 1 8 1 Don't want me to fix the servers? Fine. Long First time posting in this sub. Cross-posting because I was told you might enjoy this. Background - some time around 2000, I worked for a major finance/brokerage company in the IT department.

2.

Text - I worked the overnight shift alone and (among other things) my responsibilities included monitoring of the companies most important servers INCLUDING the trading servers as well as performing almost all repairs on these servers since my shift was the least impactful on business. These servers were how every trade from every broker worldwide was processed on behalf of clients. We had 8 servers all behind a load director. For those non-IT people, think traffic at an intersection with a cop

3.

Text - So at some point doing my job, I begin to notice issues with our trading servers. I determine the cause, come up with the plan to repair the failing parts. On the first night of the week, I will take down 2 servers, repair them, bring them back up, and put them back behind the load director. I will repeat this for the next 3 nights allowing all 8 servers to be repaired with minimal impact and have the last night of the week in case anything goes the way of the toilet. Understand that whil

4.

Text - As you can probably guess, I heard nothing back. 2 weeks later I follow up with another email reminding her of the issue and including all the documentation I had sent with the first one. Nothing. Another 2 weeks go by and I send a 2nd follow-up email noting that this isn't a question of IF these machines will fail but only a matter of WHEN. Crickets. Another 2 weeks go by. It is now about noon on Friday and I am home having just begun my weekend. I get a call that goes something like thi

5.

Text - Cathy: It's Cathy. Your boss. Me: OHH! Heya Cathy. What's... oh this cannot be good. (I am now realizing that my boss's boss is calling me at my house and that all the excrement must have followed an upward trajectory towards the device circulating air.) Cathy: All the trading servers have crashed. We need everyone on hand. Me: I'll be there in 20 minutes (It was usually a 35 minute drive) Basically, one server crashed and the load from that server was transferred to the remaining 7 which

6.

Text - I get to work Sunday night (my Monday) and the first thing I do is print out emails and those oh-so-precious read receipts. I place them in a nice folder on the corner of my desk. At 7AM Monday morning (end of my shift), Cathy walks into my office and asks me to join her in her office. I say sure and grab the folder and follow her. When we get to her office, present are me, Cathy, Dawn and a lady from HR. Cathy: So, MorpheusJay, I understand from Dawn that it is your job to monitor the tr

7.

Text - Me: Sure. (Opens folder) As you can see from this email dated xx/xx/xxxx, highlighted for your convenience, I notified Dawn of the problem and requested approval to go ahead with the fix. Here... (opens folder again) is the read receipt showing she read it the following morning at xx:xx AM, again, highlighted for your convenience. (Rinse and repeat for the other emails) Cathy: Ok. Thank you, MorpheusJay. Have a good night. We'll see you tomorrow morning. Fallout: The company lost a STUPID

Submitted by:

Posted by at
Labels:

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)