Websites Down!

[Alan]

A client called me this morning very worried that their business-critical website is down, and are unable to get important emails.

After opening a support ticket, all I get is apologies- which I pass on to my client with much embarrasment. Over two hours now and no resolution in sight.

This network malfunction also affected two other of my clients: No website, no email, and no control panel.

In this age and time, how can this type of disruption be at all possible??

My client called again very worried. I told him that even if there was a nuclear strike in the west coast, this sort of network problem could not possibly happen, especially with modern network continuity technologies, such as CDN. In other words- there are no excuses.

Is this happening because of the "free" reseller account as opposed to the cpanel?

[clivejo]

Hi Alan,

I must say I’m on the free reseller program and I've seen no downtime

Maybe there was a problem in the US data centre, but didn’t affect the UK one? Did you check on http://www.properstatus.com/# ?

[doneritehosting]

Hi Alan,

I must say I’m on the free reseller program and I've seen no downtime

Maybe there was a problem in the US data centre, but didn’t affect the UK one? Did you check on http://www.properstatus.com/# ?

Must have been just the US datacenter my site was down as well.

[vlasi47gr]

I haven't had any downtime either on my clients websites. Proper Status showed downtime on UK and US servers but didn't affect my accounts. Must have been trouble on a specific server and unfortunately. Alan, your sites must have been on that server.

Having to explain and apologize to clients is definitely no fun.

[Alan]

Hi clivejo,

Thank you for the link to the properstatus site. I wish I had known about it earlier. The issue is now resolved.

I must now 'move on' and not dwell on the issue. However, I feel that things ought to have been addressed a bit differently. For instance:

1. As the administrator of my client's accounts, I could have been notified by email that something had gone wrong and could expect an outage.

2. A report of the problem could have been made available upon discovery: "Here's what we found...", "This is what we are going to do...", "It will take aproximatelly x amount of time..." This would be very useful to ease some of my client's pain- and the wait was painful.

3. Upon resolution, a detailed report of the incident could be posted on line or emailed to affected admins. We need to relay more detailed info to our clients other than simple, general, and not-so-useful one-liners such as "... hardware broke down ..."

You have no idea how badly this hurts credibility with customers. The whole incident makes it seem like their critical websites are being hosted in someone's dingy cellar. If we make guarantees about the up time, we need to apply a certain level of professionalism when things do go horribly wrong.

By the way, with network redundancy and CDN systems, this sort of thing should never, ever, happen.

[clivejo]

I dont mean to be rude or offensive here, but if your customers website is as critical as you say, maybe he/she shouldn’t be on shared hosting.

Even at 99.9% uptime, that still leaves 8-9 hours of downtime a year. Being on a shared hosting plan means that you dont have control over when software upgrades and reboots take place. Not to mention the slowdowns that will happen as other account holders run into problems! RSP do try their best to limit our exposure to this downtime, but sometimes that just isn’t possible.

From the proper support log this incident seems to have lasted for about 5:30 hours, from my experience that is very rare. As for the details of what happened I do not know, maybe someone in tech support will be able to answer that for you.

[Alan]

I agree with you here, clivejo, perhaps shared hosting isn't a good idea for websites deemed critical.

Nevertheless, the up-time guarantee is promienently displayed as part of the package deal, shared or not.

The expectation that nothing will ever go wrong is not realistic. But I do expect a lot more transparency when this sort of thing does happen.

Again, network redundancy by today's standards should have prevented an outage this bad. If the 'switch' went down, there should have been 3 - 5 other switches taking over. Moreover, communicating very openly with resellers about the problem helps a lot.

[clivejo]

Alan did you see this post from Rsp?

http://forum.resellerspanel.com/showthread.php?t=7880

[Alan]

Yes! I even got an email with the same message, which is appreciated.

You know what they say, "An ounce of prevention..."

It is very hard as it is for small-time sellers such as myself to convince potential clients about the benefits of choosing "us" over the 1&1's, Netsol's, and G'daddy's (ugh!)

[clivejo]

I think to be honest you have been unlucky and just hit an unfortunate major incident, which in my 8 years of being with RSP have never seen before

I know that’s no consolation to you, as you have experienced a major outage and feel embarrassed in front of your customers/clients. Not a great start, but I’m pretty sure its a rare event, at least for me it is.

I’ve never heard that saying, "An ounce of prevention...", what’s the rest of it?

[Alan]

Well, the fact is that a great many people have had to suffer becuasue of the outage. I'm just hoping that resellerspanel is ensuring that this sort of thing never happens again. Never ever.

As I previously said, with today's network redundancy technoligies it should be virtually impossible to lose a whole website (and email services) for that many hours, regardless of the type of hardware malfunction.

"An ounce of prevention is worth a pound of cure"
-- Benjamin Franklin

[vlasi47gr]

Today, craigslist.org was down. No explanation, no nothing, just unavailable with the "Firefox can't find the server at www.craigslist.org." error message.

If a large and popular website such as Craigslist can be down with all their resources it can happen to anyone.

[doneritehosting]

Hello Alan,
Listen to clive and vlas They've been here a long time. I also have not experienced any serious downtime with rsp's services which is why I am still here, reliability is great support is superb.

My cousin has a reseller account with one of those big companies you mention and he does experience issues with them, I know this for a fact. RSP has been in business a long time and eventually given enough time something critical is gonna break, guaranteed. hopefully your clients have been understanding and all is well.

Merry Christmas all

[Guest]

Sorry to hear of the downtime. But I have to say, I agree that if you have clients that rely so heavily on their sites and email, then you should not be selling them shared accounts...this will eventually create a lot of stress for everyone and make you look bad, eventually. If clients who need dedicated hosting do not want to pay for it, then they are not clients worth servicing as they will become problem customers who create too much hassle for too little profit.