Knowledgebase

Strangest server problem ever

Posted by larry2148, 09-09-2011, 01:22 AM
Hey Everyone, So I have run into the most bizarre server problem, we have a server with Centos 5.6 that reboots pretty much every hour: reboot system boot 2.6.18-53.el5 Fri Sep 9 14:44 (00:17) reboot system boot 2.6.18-53.el5 Fri Sep 9 13:30 (01:30) reboot system boot 2.6.18-53.el5 Fri Sep 9 12:30 (02:31) reboot system boot 2.6.18-53.el5 Fri Sep 9 11:30 (03:31) reboot system boot 2.6.18-53.el5 Fri Sep 9 10:29 (04:31) reboot system boot 2.6.18-53.el5 Fri Sep 9 09:29 (05:32) reboot system boot 2.6.18-53.el5 Fri Sep 9 08:28 (06:32) reboot system boot 2.6.18-53.el5 Fri Sep 9 07:28 (07:33) reboot system boot 2.6.18-53.el5 Fri Sep 9 06:27 (08:33) reboot system boot 2.6.18-53.el5 Fri Sep 9 05:30 (09:31) reboot system boot 2.6.18-53.el5 Fri Sep 9 04:30 (10:31) reboot system boot 2.6.18-53.el5 Fri Sep 9 03:29 (11:31) reboot system boot 2.6.18-53.el5 Fri Sep 9 02:29 (12:32) Crontab has now been disabled, we've tried new RAM, new PSU, everything except a whole new chassis. Even so, hardware seems unlikely, this is happening like clockwork. Has anyone seen this before? I'm pulling my hair out! Thanks in advance, Larry

Posted by twikamltd, 09-09-2011, 02:52 AM
Anything in /var/log/messages? I'd disable everything on the server software-wise and see if it still happens, update all packages, check again. You have completely disabled crontab right? Then I'd want to run a memtest on the new ram, and monitor CPU temps. After that, then assume hardware.

Posted by lordspace, 09-09-2011, 03:18 AM
check your crontab. the server reboots every 30mins of every hour.

Posted by TDS-chriss, 09-09-2011, 03:33 AM
Have a look in /etc/cron.hourly/ as well.

Posted by ssfred, 09-09-2011, 04:08 AM
Hello Do you have any monitoring tools like SIM,PRM installed on the server? If so check and make sure they are not configured to restart machine when it fails to meet a configured value.

Posted by larry2148, 09-09-2011, 09:59 PM
Hi All, Just as a follow up - I had already disabled the cron completely, I was checking this server out via an IP KVM and suddenly it rebooted again. I went into the BIOS looking for WOL settings (chasing a lead from some mailing list) and disabled a bunch of unneeded features while I was in there, stuff like onboard audio and a couple other things. It's been stable since then, hopefully this helps someone in the future! Very weird! -Larry



Was this answer helpful?

Add to Favourites Add to Favourites

Print this Article Print this Article

Also Read


Language:

Client Login

Email

Password

Remember Me

Search