Syncpocalypse!

This past Tuesday we had what you might call a terrible, horrible, no good, very bad day. Wednesday too.

Early Tuesday morning our user database kicked the bucket. Technically, one of our underlying database files became corrupted, but the result was that no one could sync. And for a company in the sync business, that's no good.

Of course we expect these kinds of problems to crop up from time to time, and we back up our data accordingly. Unfortunately, it turned out to be a lot harder and more time consuming to restore from our backups than we had planned.

The good news is we were able to get back up and running without spilling a drop of data. The bad news is it took us almost a day and a half, and we were offline during that time. And frankly, that's embarrassing. We hold ourselves to a higher standard than that, and we failed to meet it.

So, what to do? Well, a couple of things.

First, we're revamping our internal backup and restore processes so we'll be able to recover from problems like this much more quickly in the future.

Second, we're putting in place redundant systems so problems like this don't even require a restore from backup; instead they simply trip a switch that causes the redundant system to take over with no interruption in service.

And finally, we're saying thank you to our customers and trial users for your patience. We were offline for a day and a half, so we've added three days of service to every customer's paid subscription and every trial user's free trial period. We haven't looked it up, but we believe that's a better service credit you'd get from any of the big guys who offer Service Level Agreements.

Regardless, if you're a paid customer and you'd like a refund, just let us know at support@spanningsync.com. We'll refund your purchase, no questions asked.

We really value what we do, and we believe our customers do too. We had a predictable problem and failed to solve it in a satisfactory amount of time. We'll do our best to make sure that doesn't happen again.

Thanks,
Charlie

Comments

[Rolfje.wordpress.com]

A classic. Many companies backup religiously and never test the restore process, only to discover that their backups are worthless when that time comes. I'm glad you were able to recover.

I'm als glad to see that you're now building redundant servers. With the uptime you had lately I thought you'd already done that, because it seems that it always worked.

Thanks for the extra days of syncing, that was a nice gesture.

Keep 'em running! Cheers,
Rolf

[Brad Heitmann]

all of my contact phone numbers just disappeared. is this related to the outage?

[Charlie Wood]

HI Brad-

No, not at all. Please create a problem report (Advanced > Create report) and send it to support@spanningsync.com so we can figure out what caused that to happen.

Thanks,
Charlie

[David Payne]

Yesterday, I chose to accept the automatic update option. While updating, it indicated the library did not load. Now I cannot launch the program and when I go to iCal, I get an annoying error message every time I enter or edit an item.....very annoying. Please help.