| Posted by husaren, 08-22-2007, 11:27 AM | | Hi all.
Im having some issues keeping the correct time on an IBM x342
The time is perfect most of the time, but once every few minutes the time goes an hour and 12 mins into the future..
I dont have NTP installed.
see below for details
Wed Aug 22 16:30:55 CEST 2007
Wed Aug 22 16:32:28 CEST 2007
Wed Aug 22 16:32:28 CEST 2007
Wed Aug 22 16:32:29 CEST 2007
Wed Aug 22 16:32:30 CEST 2007
Wed Aug 22 16:32:30 CEST 2007
Wed Aug 22 16:32:30 CEST 2007
Wed Aug 22 16:32:30 CEST 2007
Wed Aug 22 16:32:31 CEST 2007
Wed Aug 22 16:32:31 CEST 2007
Wed Aug 22 16:32:31 CEST 2007
Wed Aug 22 17:44:03 CEST 2007
Wed Aug 22 17:44:03 CEST 2007
Wed Aug 22 17:44:03 CEST 2007
Wed Aug 22 17:44:03 CEST 2007
Wed Aug 22 17:44:04 CEST 2007
Wed Aug 22 17:44:04 CEST 2007
Wed Aug 22 17:44:05 CEST 2007
Wed Aug 22 17:44:05 CEST 2007
Wed Aug 22 16:32:31 CEST 2007
Wed Aug 22 16:32:31 CEST 2007
Wed Aug 22 16:32:31 CEST 2007
Wed Aug 22 16:32:31 CEST 2007
Wed Aug 22 16:32:32 CEST 2007
Wed Aug 22 16:32:32 CEST 2007
Wed Aug 22 16:32:32 CEST 2007
Wed Aug 22 16:32:34 CEST 2007
Wed Aug 22 16:32:34 CEST 2007
Wed Aug 22 16:32:34 CEST 2007
Wed Aug 22 16:32:35 CEST 2007
Wed Aug 22 16:32:37 CEST 2007
Wed Aug 22 16:32:37 CEST 2007
Wed Aug 22 16:32:37 CEST 2007
Wed Aug 22 16:32:38 CEST 2007
Wed Aug 22 16:32:38 CEST 2007
Wed Aug 22 16:32:38 CEST 2007
Wed Aug 22 17:44:10 CEST 2007
Wed Aug 22 17:44:10 CEST 2007
Wed Aug 22 17:44:11 CEST 2007
Wed Aug 22 17:44:11 CEST 2007
Wed Aug 22 17:44:11 CEST 2007
Wed Aug 22 16:32:38 CEST 2007
Wed Aug 22 16:32:38 CEST 2007
Wed Aug 22 16:32:38 CEST 2007
Wed Aug 22 16:32:39 CEST 2007
Wed Aug 22 16:32:39 CEST 2007
Wed Aug 22 16:32:39 CEST 2007
Wed Aug 22 16:32:41 CEST 2007
Wed Aug 22 16:32:41 CEST 2007
Wed Aug 22 16:32:41 CEST 2007
Wed Aug 22 16:32:41 CEST 2007
Wed Aug 22 16:32:42 CEST 2007
Wed Aug 22 16:32:42 CEST 2007
Wed Aug 22 16:32:42 CEST 2007
Wed Aug 22 16:32:44 CEST 2007
Wed Aug 22 16:32:44 CEST 2007
Wed Aug 22 16:32:44 CEST 2007
Wed Aug 22 16:32:44 CEST 2007
Wed Aug 22 16:32:44 CEST 2007
Wed Aug 22 16:32:45 CEST 2007
Wed Aug 22 16:32:45 CEST 2007
Wed Aug 22 17:44:17 CEST 2007
Wed Aug 22 17:44:17 CEST 2007
Wed Aug 22 17:44:18 CEST 2007
Wed Aug 22 17:44:18 CEST 2007
Wed Aug 22 17:44:18 CEST 2007
Wed Aug 22 17:44:19 CEST 2007
Wed Aug 22 16:32:45 CEST 2007
Wed Aug 22 16:32:45 CEST 2007
Wed Aug 22 16:32:46 CEST 2007
Wed Aug 22 16:32:46 CEST 2007
Wed Aug 22 16:32:46 CEST 2007
Wed Aug 22 16:32:48 CEST 2007
Wed Aug 22 16:32:48 CEST 2007
Wed Aug 22 16:32:48 CEST 2007
Wed Aug 22 16:32:49 CEST 2007
Wed Aug 22 16:32:49 CEST 2007
Wed Aug 22 16:32:49 CEST 2007
Wed Aug 22 16:32:51 CEST 2007
Wed Aug 22 16:32:51 CEST 2007
Wed Aug 22 16:32:51 CEST 2007
Wed Aug 22 16:32:52 CEST 2007
Wed Aug 22 16:32:52 CEST 2007
Wed Aug 22 16:32:52 CEST 2007
Wed Aug 22 16:32:52 CEST 2007
Wed Aug 22 17:44:24 CEST 2007
Wed Aug 22 17:44:24 CEST 2007
Wed Aug 22 17:44:24 CEST 2007
Wed Aug 22 17:44:24 CEST 2007
Wed Aug 22 17:44:24 CEST 2007
Wed Aug 22 17:44:25 CEST 2007
Wed Aug 22 17:44:25 CEST 2007
Wed Aug 22 17:44:26 CEST 2007
Wed Aug 22 17:44:26 CEST 2007
Wed Aug 22 17:44:26 CEST 2007
Wed Aug 22 16:32:52 CEST 2007
Wed Aug 22 16:32:52 CEST 2007
Wed Aug 22 16:32:52 CEST 2007
Wed Aug 22 16:32:52 CEST 2007
Wed Aug 22 16:32:52 CEST 2007
Wed Aug 22 16:32:52 CEST 2007
Wed Aug 22 16:32:53 CEST 2007
Wed Aug 22 16:32:53 CEST 2007
Wed Aug 22 16:32:53 CEST 2007
Wed Aug 22 16:32:54 CEST 2007
Wed Aug 22 16:32:54 CEST 2007
Wed Aug 22 16:32:55 CEST 2007
Wed Aug 22 16:32:55 CEST 2007
Wed Aug 22 16:32:55 CEST 2007
Wed Aug 22 16:32:55 CEST 2007
Wed Aug 22 16:32:55 CEST 2007
Wed Aug 22 16:32:55 CEST 2007
Wed Aug 22 16:32:56 CEST 2007
Wed Aug 22 16:32:56 CEST 2007
Wed Aug 22 16:32:56 CEST 2007
Wed Aug 22 16:32:56 CEST 2007
Wed Aug 22 16:32:56 CEST 2007
Wed Aug 22 16:32:57 CEST 2007
Wed Aug 22 16:32:58 CEST 2007
Wed Aug 22 16:32:58 CEST 2007
Wed Aug 22 16:32:58 CEST 2007
Wed Aug 22 16:32:58 CEST 2007
Wed Aug 22 16:32:58 CEST 2007
Wed Aug 22 16:32:58 CEST 2007
Wed Aug 22 16:32:58 CEST 2007
Wed Aug 22 16:32:59 CEST 2007
Wed Aug 22 16:32:59 CEST 2007
Wed Aug 22 16:32:59 CEST 2007
Wed Aug 22 16:32:59 CEST 2007
Wed Aug 22 17:44:31 CEST 2007
Wed Aug 22 17:44:31 CEST 2007
Wed Aug 22 17:44:31 CEST 2007
Wed Aug 22 17:44:31 CEST 2007
using debain 2.6.8-3-686-smp #1 SMP Thu Feb 9 07:05:39 UTC 2006 i686 GNU/Linux and it has only been an issue for the last few weeks.
any advice is greatly appreciated.
kind regards
/husaren
|
| Posted by ResellerPlanet, 08-22-2007, 11:37 AM | | Running any control panel software?
|
| Posted by husaren, 08-22-2007, 11:39 AM | | No.
Its a MX-frontend. Running courier-pop/imap, apache-SSL and mySQL
|
| Posted by A-Wing, 08-22-2007, 04:59 PM | | How odd, is there anything in dmesg or /var/log/messages that would indicate anything?
|
| Posted by husaren, 08-22-2007, 05:05 PM | | nothing at all.. its really strange.. nothing to see in top og with ps aux.
both are the same when the time is correct and when its not..
|
| Posted by mrjit, 08-22-2007, 06:00 PM | | Interesting.
Honestly I'd probably just point it at an NTP if that happened.
Course, then a month later I'd find out it was come CPU cycling problem when the CPU craps out, with my luck.
|
| Posted by A-Wing, 08-22-2007, 06:04 PM | | I would have said NTP as well.
Although a hosting company I once worked for bought a load of cheap (crap) AMD64 boards and CPUs and they did some really strange things with the clock. Usually incrementing a minute every second (or visa-versa). I forget the fix off the top of my head but it was a boot time flag which is similar to noapic.
|
| Posted by husaren, 08-22-2007, 06:32 PM | | Well, actutally i´ve just installed ntpd and for some reason the server started to lose time. ~ 20 mins/hour.
# date
Thu Aug 23 01:22:05 CEST 2007 <- the wrong time in future
# date
Thu Aug 23 00:10:33 CEST 2007 <- also the wrong time, now in the past
# /etc/init.d/ntp-server stop
Stopping NTP server: ntpd.
# ntpdate gps.dix.dk
23 Aug 00:28:54 ntpdate[19408]: step time server 192.38.7.240 offset 1087.460141 sec
Last edited by husaren; 08-22-2007 at 06:39 PM.
|
| Posted by husaren, 08-22-2007, 06:34 PM | | Well, like i said earlier. This is a proper IBM server, not the newest of servers but this has never been a problem before.
I planning to do a full distro upgrade this weekend and if that doesn´t help ill ebay it..
|
| Posted by mrjit, 08-22-2007, 07:01 PM | | Point it at a separate NTP, not one you're running locally
|
| Posted by husaren, 08-23-2007, 04:07 AM | | It is not using local NTP.
i have pointed it to
# You do need to talk to an NTP server or two (or three).
#server ntp.your-provider.example
server ntp.inet.tele.dk
server gps.dix.dk
It seems the ntpd tries to compensate for the future time by setting the clock further back..
besides im not running ntp-server on any local machines, and they all have the correct time anyway.
|
| Posted by macker, 08-23-2007, 04:19 AM | | you show a number of timestamps, it jumps forward, then back... are you changing the system time, or does it "jump back" on its own?
the clock seems to increment several seconds in this "forward time", but resumes on the same second counter. this would seem to suggest it's processing several seconds of accelerated time, or that it's reverting further than it should.
ntp will try to track and correct clock drift. manually setting the clock, or trying to run ntp (as a daemon) on a server with a bad clock source, is likely to have negative results.
I would suspect a problem with the hardware clock source or some software trying to adjust the time. Try setting the correct time, then running 'hwclock --systohc' (as root). if you recently upgraded any packages, kernel, scripts, etc., the usual suspects
|
| Posted by husaren, 08-23-2007, 06:12 AM | | Yes, its jumps back to the correct time all on its own. on the above timestamps i am not forcing the clock at all.
It is always the same amount of time it jumps, appox. 71.5 mins into the future.
With the NTPdaemon installed, it would still jump the 71.5mins forward, but then not back to the correct current time. Resulting in the loss of time and effectively setting the clock in the past.
Yes, obviously. But still worth a try imho. And still the hwclock source is dead-on time most of the time, just not when it "time-warps"
Well, you could be right, propably one of the CPU´s is game over and is giving me a hard time for whipping it all these years..
Actually i tried to commit the system time to hwclock which caused the hwclock proces to hang catatonic with 100%CPU usage. had to reboot to kill it. I might give it another try just before throwing a new kernel at it.
|
| Posted by hosein, 08-23-2007, 06:38 AM | | Hi,
I changed server time in WHM but Its reset to Default Time .
How to change Server time in SSH ? (CentOS)
Thanks
|
| Posted by macker, 08-23-2007, 02:05 PM | | husaren,
I forgot to ask, how are you measuring these timestamps. E.g. is this from web server logs? The main thing that disturbs me is it's not only jumping forward by a precise amount of time (1:11:32), but when it jumps back, it lost the passage of time, i.e. the "real" clock is drifting each time this happens.
I've seen hardware clocks fail, particularly on older 8088 computers, and as a result the time would always be some fixed date that the O/S decided to substitute as a starting time on boot (e.g. 11:11:11), but this is different. I don't think this is two different clock sources being used indepedently, because of the drift; the advancement is always a precise amount over the "correct" clock time, but a few seconds get lost each time. If it was two clocks, they should be drifting apart.
Lots of reasons why clock drift happens, but this is just odd. May be something more readily explained by the IBM architecture in your system, but it feels most to me like a software clock and hardware clock sync issue, with failing accesses to one of the clocks, and reverting to last known good state. If these logs are with ntpd running, that may explain the artifact. That said, I can't think of anything else that reasonably explains this behavior. If you ever find out what's causing it, drop me a PM, it's a very puzzling question.
hosein, this is one of many basic topics you should look into. You might try the Linux SAG (System Adminsitrator Guide; search and download), a book like Linux for Dummies (which does a very good job explaining many topics for people new to Linux), or various documentation which may be localized to your language. Otherwise, 'man date'.
|
| Posted by husaren, 08-23-2007, 03:29 PM | | I just spammed a 'date >> /root/date.txt'
And yes, the clock would drift if it wasn´t for a cronjob each minute to sync with ntp.inet.tele.dk
These logs are without NTP-daemon running. I think i will try to update the kernel and see if it helps. if not, i dont think i´ll spend more time on something that seems to be a hardware issue.
Ill share my findings in this thread one way or the other.
|
| Posted by Host Ultra, 08-23-2007, 03:52 PM | | Your config shows you are using two different NTP servers.
Could it be that one of this servers has the wrong time or different time zone, and you randomly get a different server each time you run NTP.
|
| Posted by husaren, 08-23-2007, 04:05 PM | | Well, i have tried it with the the servers, and with only the ordinairy ntp-pool.
gps.dix.dk is a stratum 1 class server, ( http://support.ntp.org/bin/view/Servers/ServerStratum) and is likely to be pretty precise.
ntp.inet.tele.dk is the biggest danish ISP´s public NTP server and is also likely to be pretty precise.
Actually, thats the whole idea with NTP, so i doubt that is the problem.
As stated earlier, the time shifts without NTP installed also.
|
| Posted by husaren, 11-16-2007, 11:07 AM | | Hello again. I promised to return with info when i found the fix.
Im happy to report at the time has now been spot-on for the last 40 days or so.
Funny thing is that i havent touched anything.. the problem started by itself and resolved itself apparently.
My only (far-fetched) theory is that the machine was getting some skewed timestamps from ntpserver. although the problem persisted with ntp removed.
oh well.. i guess some things are just too strange to understand.
cheers
/husaren
|
|
 Add to Favourites
 Print this Article
|
