August 2007

Gallery and Wiki were down for ~ 10 hours. The usual drill: Comcast changed my IP address and everything hosted in the basement became inaccessible. I didn’t notice right away because I had forgotten to update the the health checks to specifically test the Gallery and Wiki now that they’re hosted separately from everything else. Then again, Esmae keeps us so busy I probably wouldn’t have noticed the alert emails pouring in…

On July 25th rhubarb, the firewall machine in the basement, had a spectacular hard disk crash. It sounded like a cat fight and startled us out of bed!

Rhubarb kept routing traffic, so I was lazy and didn’t get around to rebuilding on a new disk until today. Carrot, which hosts the Gallery and the Wiki, was inaccessible during rhubarb’s rebuild. Kiwi, which hosts everything else and lives in a data center in Texas, was unaffected.

The gap in the traffic graph below reflects the time during which rhubarb had no working hard disk and couldn’t record log files. If you had been attempting to hack in to my network, that would have been a good time to do so surreptitiously!