| Posted by MrSaints, 01-03-2010, 07:18 AM | | Hello WHT,
I seem to be having a small issue with my VPS and the CPU spiking. Its an ordinary CentOS AMD (4Cores) 1GB RAM (2GB Burstable) VPS. Its currently running on suPHP (thinking about LiteSpeed since FCGI didn't work out so well but according to their website, you require > 1GB RAM). I've optimized the MySQL Configuration using those tuner scripts as well as tweaked the prefork mpm configuration.
The problem is, it goes well for around 1~2 hours at a load of around 0.70 if i'm not wrong, but after that, it suddenly spikes to > ~50.00, when I typed top -c, the processes and the CPU usage doesn't add up to that much.
I've checked the connections to Apache via: netstat -nt | grep :80 | wc -l and it returned 284 (which doesn't seem much to me - EDIT: 323 now. ).
What should I do?
free -m Output:
Load average at the moment (after restarting Apache + MySQL):
Should I upgrade my RAM or is there something wrong? Like hidden processes. Oh, I've forgotten to mention the VPS is hosting a regular licensed vBulletin v3.8.3 script with around 60 concurrent users in average (sometimes 100) - and yes, its not using lots and lots of plugins and its a fairly optimized vBulletin.
Thanks!
Last edited by MrSaints; 01-03-2010 at 07:26 AM.
|
| Posted by madaboutlinux, 01-03-2010, 08:37 AM | | What is the output of the following commands when load spikes up greater than 50?
Also when the load average goes high, execute the following command and allow it to run for a minute and forward the output
If you see mysql process is using up high CPU resources, execute
to check the resource hog queries. This will give a clear idea as what is causing the issues.
|
| Posted by MrSaints, 01-03-2010, 09:58 AM | | For the first one, I typed top c and got:
http://pastebin.com/m5e925b9
For ps -auxf, it was one long list, but here's part of it:
http://pastebin.com/m305dec
Here's vmstat 2 results (around 1 minute):
http://pastebin.com/m99ee919
watch mysqladmin proc (looks pretty normal?):
http://pastebin.com/m46207ab2
Oh and thank you very much for assisting me madaboutlinux
Seems to have recovered after a simple service httpd restart and service mysql restart, but comes back again after a while...
Last edited by MrSaints; 01-03-2010 at 10:04 AM.
|
| Posted by madaboutlinux, 01-03-2010, 10:23 AM | | Looking at results, it looks like large volume of emails been sent out from your server either by user leafreco OR bankaist which is using up high resources. Check out the email logs at
It will help you to figure out who the culprit is.
|
| Posted by CoreNetworks, 01-03-2010, 10:26 AM | | 90.2%wa
Looks like its your system I/O that's causing the high load, longer delays on loading, so more connections stay open, using more RAM. Those times in your mysqladmin output are in seconds, and are *very* high. Sounds like you may have some database issues to address, or your VPS is running on a system that has other high I/O users on it. Anything database intensive with a lot of read/write would likely not be optimal running on a VPS, especially if others start high I/O from the same server which will most definitely impact your usability.
|
| Posted by MrSaints, 01-03-2010, 10:34 AM | | I'm not really familiar with I/O issues, but cPanel team once told me my VPS had some "I/O Bottleneck", my question now is, is it because of my VPS System (Since its VMWare and cPanel people claims VMWare have been having issues running cPanel)?
Upon restarting my MySQL and Apache service, I executed those command again after a few minute to see the result:
http://pastebin.com/m2852236f
http://pastebin.com/m78344d90
And what sort of database issues could this be? Cause those 2 users are running a vBulletin forum.
This time the I/O CPU usage seems to be less, but the idle time more.
Funny thing is, this problem comes after a while. So i'm not exactly sure whats causing it. Before that, everything seems quite stable including Exim.
Just got this a few moments ago:
|
| Posted by madaboutlinux, 01-03-2010, 10:39 AM | | The queries of database leafreco_vbulletin is taking a long time to load and can create high I/O which in turn increases the server load. Also in the previous 'ps -uaxf' output, it looks like bulk emails are sent out from your server. You can watch the mail logs by executing
|
| Posted by MrSaints, 01-03-2010, 10:50 AM | | Ohh I see, is it possible to just kill the query once it takes so long? I already lowered the timeout and set a slow query time and log.
Also back to my previous point, could the VPS Software (VMWare) (http://www.webhostingtalk.com/showpo...14&postcount=6) be the result of this as well? Since I ran the same test again this time with no results from mysqladmin proc.
And I got this from top c:
http://pastebin.com/m3dc87f25
This time its the CPU idle stat; even when queries are not being executed.
At the moment, there's not much in the exim_mainlog.
Last edited by MrSaints; 01-03-2010 at 10:56 AM.
|
| Posted by madaboutlinux, 01-03-2010, 10:54 AM | | Doesn't look like if vmware is causing the issues else the problem would have started since the VPS was setup. BTW, I/O is not at all high in http://pastebin.com/m3dc87f25 but something is using up high memory as far as I can see.
|
| Posted by MrSaints, 01-03-2010, 11:01 AM | | I'm quite clueless what to do at the moment, any possibility of just killing those extremely long CPU consuming queries? or simply lowering my %wa. There must be a problem somewhere, perhaps in my my.cnf, cause the problem occurs only after a while. Besides, those 2 sites combined would probably have less than 100 concurrent users on average, I don't think the VPS isn't capable of taking that much. Must be a hiccup.
Last edited by MrSaints; 01-03-2010 at 11:04 AM.
|
| Posted by HD Fanatic, 01-03-2010, 01:45 PM | | Can you post my.cnf config? Also, install mytop.
|
| Posted by MrSaints, 01-04-2010, 03:56 AM | | Yeap, I do have mytop installed. And it seems to have stopped spiking now after tweaking the prefork configuration and disabling KeepAlives (which is resulting in huge number of IP bans now).
I based my configuration upon the recommended settings from the tuner scripts, didn't followed it completely though (under RAM allowance).
http://pastebin.com/m4e3608bd
Also, I got a High CPU alert E-Mail:
http://pastebin.com/m59712ba8
I got a lot of MySQL fail e-mail.
Last edited by MrSaints; 01-04-2010 at 03:59 AM.
|
| Posted by yajur, 01-04-2010, 07:42 AM | | i think now you have 1GB ram why not add few more gig
also check has ram started to swap or not
|
| Posted by RHS-Chris, 01-04-2010, 09:25 AM | | Sounds like you are experiencing a cPanel bug, that many have seen issues with, including myself. It seems to have to do with the new way they are analyzing the log files. Take a look at these threads on the cPanel forums:
http://forums.cpanel.net/f5/11-25-cu...ur-134281.html
http://forums.cpanel.net/f5/httpd-defunct-142509.html
|
| Posted by MrSaints, 01-07-2010, 05:28 AM | | Thanks for the heads up, but upon disabling the log rotations, and what not. I still tend to face a small CPU spike, even after changing to LiteSpeed for 3 days now, its around an average of 1.0 now from 0.15 before (When LS was first installed).
|
| Posted by hD-Scott, 01-07-2010, 08:41 AM | | I'm experiencing the same issue and didn't have the access files rotating.
|
| Posted by RHS-Chris, 01-11-2010, 11:13 PM | | Another thing to try is to downgrade eaccelerator. I've done this on one of my larger servers, and the loads have come back down to within a tolerable limit.
|
| Posted by LeaTrueman, 01-12-2010, 01:45 AM | | Hello,
Please check WHM>> Server Status >> CPU/Memory Usage and paste the top 3 domain results.
|
|
 Add to Favourites
 Print this Article
|
