Sorry, folks, unplanned site outage

[mathieu Bauwens]

Thanks for the time working to fix it !

[Henry Ambrose]

Thank you all for keeping the place running.

[rdenney]

Thanks, Tom, for posting Brian's message. I didn't want to post his name without his specific authorization.

I also have experience with RAID systems, and it's important to remember that RAID was not originally devised as a reliability strategy, but rather as a performance strategy to get more out of many disks that what was affordable to get out of one. By pulling successive chunks of data from an array of disks, one could read a number of chunks at the same time and then assemble them. Mirroring really only works with data disks, not with boot disks--when a boot disk goes, it often takes with it whatever might be needed for the system to function on the other disk. This was the reason I chose UNRAID for my backup storage server--they realized that disks these days don't need a performance improvement--systems are network-constrained not disk-constrained--they needed reliability. But that system doesn't boot from its array, though it does boot from a thumb drive. I had an outage recently when my parity drive failed, and it also took me several extra days to get it back up and running. And the reason? Yup. The power-supply cable going to the parity drive was intermittent. When I fixed that, I was able to get a new parity disk built and I lost no data. That is the same philosophy that Brian and Tom have employed, and why we have never lost significant data when faced with an outage.

The use of online redundant servers is beyond the scope of this forum, as I'm sure everyone can appreciate. But that's the only thing that would have prevented this outage. Brian and Tom have a rhythm going that they have proven is sustainable with the current arrangement, and an outage of a day or four every several years seems a small enough price to pay for allowing them to stay in tune with their rhythm year in and year out.

Rick "no apologies necessary" Denney

[Jac@stafford.net]

I also have experience with RAID systems, and it's important to remember that RAID was not originally devised as a reliability strategy, but rather as a performance strategy to get more out of many disks that what was affordable to get out of one.

Dating myself - used DNS Load Balancing (Round-robin DNS) in the old days.

Thanks for your good work.

[Tin Can]

Right now this site is working better than Facebook, which is not taking posts.

[Sal Santamaura]

On the technical questions, we are happy with things as they are. More "features" would lead to expenses, which we'd prefer to avoid. The data is all backed up to another location daily (last 7 days are kept in both places) and so far, after three other outages like this in the past decade, we haven't lost a thing. Our disaster recovery measures seem appropriate to the criticality of the data we store--no one is going to die or go to prison or go bankrupt if we lost all of it.

Our generous server provider, Brian Reid, asked my to post his take on this:

" Originally Posted by Brian Reid
The Large Format Photography Forum is hosted on a server in a commercial data center. The server itself is maintained by a person (me) who has decades of experience at running data center operations and maintaining servers. This server is a freebie. I bought it and installed it. The data center does not charge for its presence there and its staff do not maintain the server. I do....My stubbornness and my unwillingness to give up and buy a new server caused you guys a long delay. I apologize.

We will occasionally have outages of more than a day, but so far the ones we've had haven't been of sufficient impact to warrant charging for membership here or adding advertisement revenue, which would be required if we needed to invest in a high availability infrastructure environment. The infrastructure we have, thanks to Brian's generosity, works fine 99+% of the time.

Brian, please do not apologize! We are forever in your debt for being the primary force (along with Tuan, Tom and the moderators) enabling our community's blissfully non-commercial existence.

It annoys me greatly when anyone complains about or criticizes this free resource. I felt compelled to clear things up when that happened during the outage:

http://www.apug.org/forums/viewpost.php?p=1556567

Thanks so very much for all you folks do.

[John Fink Jr.]

Great to have it all back and running. Thank you to all who worked to get it there!

[AuditorOne]

Great to see you back.

[Robert Hall]

Glad to have you all back.

[SergeiR]

Aye. Welcome back. We missed ya.