April 2006

http/https (web) services down from 12:24 to 20:53 PDT. Somehow Apache’s getting completely tied up waiting for Tomcat. Added timeouts and periodic connection recycling. This happened once earlier this year and I forgot to log it.

Preserved for posterity from the old pre-blog HTML page:

2005-Dec-10

All services down from 11:45 to 12:05 PST. Migration to replacement hardware complete. Replaced disks, motherboard, power supply.

2005-Dec-03

All services down from 21:00 to 00:00 PST. Failed to migrate to replacement hardware. Downtime in the the new few days to try again.

2005-Nov-18

All services down from approx 6:00 to 20:00 PST due to hardware failures. The motherboard and/or the power supply went flakey and one hard disk failed. All services are back up on the remaining disk and borrowed hardware. There will be downtime 12/03 to install replacement parts.

2005-Oct-06

Web server down from approx 1:30 to 8:45 PDT; unknown reason.

2005-Jun-22

Web server down from 9:15 to 17:15 PDT; unknown reason.

2005-Jun-21

ISP outage from 15:45 to 16:50 PDT.

2005-Jun-10

Webmail now available. Log in using the mail link on the left.

2005-May-07

Late night downtime due to dislodged network cable. Public IP changed from 67.168.38.251 to 67.168.38.164.

2005-May-07

Firewall upgraded to “new” hardware. We escaped before the old hard disk completely died!

2005-Apr-01

Scheduled all-day power outage beginning 9:00 a.m.

2005-Jan-14

Public IP changed from 67.183.16.116 to 67.168.38.251. Servers unreachable while DNS updates propagate.

2005-Jan-05

Firewall updated without downtime. Happy new year’s!

2004-Oct-05

Brief downtime to update firewall to IPCop 1.4.0 Final.

2004-Sep-24 – Oct-01

Occasional evening downtime during transition from calvin to carrot.

2004-Aug-19

Downtime during transition to new firewall. Public IP changed to 67.182.135.20.

2004-Jul-06

Oops, out of disk space! Temporarily using an old 1.6 gig drive to hold the overflow.

2004-Jun-16

Power out. But it’s sunny!

2004-Jun 08

Power out

2004-Mar-25

Tomcat, MySQL updates. Unplanned downtime after I failed to notice that the upgrade tightened security and denied webapps the ability to access the database. No improvements to Tomcat stability.

2004-Mar-16, 17

Kernel upgrade, new UPS

2004-Mar-04

Tomcat goes down periodically. Looks like it’s not quite stable yet.

2004-Feb-15

Network card upgrade

2004-Mar-08

BackupPC is now making daily backups of all machines. The old server (Damosie) is no longer publicly accessible.

2004-Feb-08

The transition to the “new” Debian server (Calvin) went smoothly. I expect to keep the old one (Damosie) running on port 81 until mid to late February before converting it too to Debian and setting it up as a failover machine. Tomcat is occasionally throwing exceptions when a page is viewed for the first time in a while but a browser page refresh makes them go away. I haven’t tracked this down yet – but it’s a step up from the sporadic freezes on the old server! If you notice any patterns let me know.