Update on network connectivity problems for servers 38,39,40,41,42,43

[regic]

Update on this morning's network connectivity problems that are affecting servers 38,39,40,41,42,43:  the outage is due to a problem in an underground transformer room owned by the utility company Centerpoint that supplies power to the data center building where servers 38,39,40,41,42,43 are housed.  It could be a few more hours before Centerpoint has restored power to the transfer switch (the servers are running but will be inaccessible until power is restored to the network transfer switch) .

The latest from the data center:

The transformer room for the building, where our transfer switch is located is flooded.
From what I've been able to gather, is that the building's fire suppression system has malfunctioned and is pumping water into the underground transformer room, and has caused our equipment, as well as our utility provider's equipment Centerpoint, to automatically shutdown to avoid damage.

The Houston Fire Department is at the scene now attempting to shut off water flow and pump the water out, so that Centerpoint can begin work on restoring power.

[cranky_hippie]

Thank you, Regi. I thought it was just me again. 

[xppman]

Mandy was kind enough to inform me at around 6.30 - 7.00 est about this.

This is weird because I was lying in bed this morning and thinking about
what if a disaster hits an area that has a high concentration of servers
(my thoughts were more of California and a major earth quake).

I thought WOW... that would shut down the Internet and bring e commerce to a halt.

Shrugged it off and went back to sleep only to wake up and find the servers down as described above. Must have been my woman's intuition kicking in again.  :blinkie:

[regic]

Issue resolved: Power is restored.



[quote author=cranky_hippie link=topic=5233.msg24368#msg24368 date=1159023466]
Thank you, Regi. I thought it was just me again.  [/quote]

Stop playing the violin and get to work on your site cranky.  

[cranky_hippie]

Must have been my woman's intuition kicking in again.

Change your socks and maybe it will go away.  

Stop playing the violin and get to work on your site cranky.

I was trying. Honestly I was. 

[xppman]

Yea CRANKY get to work.  >

[bargainbloodhound]

what if a disaster hits an area that has a high concentration of servers
(my thoughts were more of California and a major earth quake).

Impact of Hurricane Katrina on Internet Infrastructure (PDF)
http://www.renesys.com/tech/presentation...ep2005.pdf

If there is a major disaster you're basically screwed until the network can be rerouted.  Cell phone service and local TV broadcasts were disrupted in NY after 9/11 because the main transmitters were on top of the WTC.  It took a few days for alternate transmitters to be setup and service restored.

If there is a major earthquake that starts swallowing entire data centers and all of the servers in them into a bottomless hole in the ground then you're really screwed and better hope you have a backup of your site's files (and that you yourself haven't been swallowed into a bottomless pit)

Hope this helps.  :twistedevil:

[bargainbloodhound]

[quote author=xppman link=topic=5233.msg24372#msg24372 date=1159026940]
Yea CRANKY get to work.  >
[/quote]

Time to get a Gyro

[xppman]

I know it was a morbid thought. But it came up. In times of disasters or terrorists attacks
like 911 hurricanes and earthquakes... I can assure you my thoughts are not about if I
can get online or not or sell a widget. I'm thinking, bullets, water food...
survival stuff. 

We went through plenty of hell in the "H" season of 2004 and is why
we are now in the mountains. 
But in reality it was good for my son and myself
and we speak of it often.
We had 9 or 10 days of camping and doing with out all this electronic BS in out lives.
It was awesome to be able to survive in a devastated area and come out OK,
and we bonded even deeper as father and son. 

OK I have to be honest. I'd like to see at least one BIG building (OK just maybe a few of the cubicles in that building) get swallowed up in a little earthquake...
not to hurt the peeps who STILL
foolishly try to rely on them but in hopes those who have caused such grief for so many,
may be in the building (their cubicles) at the time of the swallowing.  :blinkie:

Anyway glad they got the servers back up an actually that was pretty
damn quick considering the situation.

Thanks supercenter and the Houston FD. You guys rock.

Something else noteworthy to say is:

Thanks TT team for being on top of it and informing your customers of this brief outage.
Now we know "others" who would have told us to just clear our cache and cookies.  :

[xppman]

[quote author=bargainbloodhound link=topic=5233.msg24378#msg24378 date=1159028687]
[quote author=xppman link=topic=5233.msg24372#msg24372 date=1159026940]
Yea CRANKY get to work.  >
[/quote]

Time to get a Gyro
[/quote]  > > > > > > > > > > > > > >