| Posted by hichamnaima, 03-30-2011, 07:21 AM | | Hello
Server load 240.15 (16 CPUs)
Memory Used 100%
I use these server specifications :
AMD Opteron(tm) Processor 6128 * 2 (16 core)
Mem: 16GB
/dev/md0 5.3T 2.1T 3.0T 41% /
/dev/sda1 198M 18M 170M 10% /boot
tmpfs 7.9G 0 7.9G 0% /dev/shm
/usr/tmpDSK 485M 21M 439M 5% /tmp
Settings lighttpd
cat php.conf
cat optimizations.conf
Other Information :
http://www.webhostingtalk.com/showth...23#post7342423
Problem because of thumbs of the videos
And stop the problem when I change the name of the folder thumbs
Thumbs folder contains a very large number of images that are not larger than 14 kb
And extension of these images is: jpg
Regards
|
| Posted by hichamnaima, 03-30-2011, 02:46 PM | | Can anyone help please !
|
| Posted by Rapid2214, 03-30-2011, 04:46 PM | | Can you use an alternative to fastcgi?
Also, please show the output of the following "free -m".
What OS are you using?
Marty
|
| Posted by hichamnaima, 03-30-2011, 04:51 PM | | Yes, I have tried to change the fastcgi but the same problem
free -m
And use the centos 5.5 64-bit
|
| Posted by Rapid2214, 03-30-2011, 04:58 PM | | So your using 11GB of ram which is alot, are you heavily caching those images? How frequently are they viewed?
|
| Posted by hichamnaima, 03-30-2011, 05:04 PM | | Currently disabled images in the site
If activated it will consume 16GB and cpu 100%
And I do not know what is the thing now, which consumes memory !
|
| Posted by Acozilla, 03-30-2011, 05:05 PM | | I can understand what you're going through. Recently one of my websites has grown to the point where I too receive too an overwhelming number of requests for small (thumbnail) images. About 10-20 on a page for a single user.
I also used a single server to serve all the content, but have since switched to several atom servers running nginx which act as proxies and cache most of the images in cache. Once the total ram was consumed, things started loading off the disks. I have 5 atom servers each equipped with 2x 2TB HDs. Through some software load balancing, I assure myself that each disk pushes an even amount. My servers have a 1000mbps connection, so I try to make sure each disk pushes about 50mb/s max. Keep in mind this only occurs once all my RAM is exhausted.
Also, I'm not sure if this applies anymore, but I know lighttpd had memory leak issues, which over time would build up to consume most of your memory.
Finally, switching to a multiple atom solution, my prices have gone down and I have more control over the fail-over, which a single server could not have provided me. Also, if you wish for some sort of mirroring to keep your content redundant, then I suggest modifying your script/code that processes the images (thumbs) and distribute them in a way across your disks so you can easily copy over the lost images in the event of a disk failure.
This has worked well for me, and if you need more technical information, feel free to PM me.
|
| Posted by Rapid2214, 03-30-2011, 05:06 PM | | Have you restarted/stopped lighthttp since stopping the images?
Run "vmstat"
|
| Posted by hichamnaima, 03-30-2011, 05:10 PM | | Yes, he even rebooted the server after you disable all images.
vmstat
|
| Posted by Rapid2214, 03-30-2011, 05:30 PM | | Your getting alot of wait on your CPU process which is worrying, what speed are your harddrives (7200k)?
EDIT: Sorry, I think it shiffed along, read it wrong, do the steps anyway
Run: (Gets read/write speeds)
hdparm -tT /dev/md0
hdparm -tT /dev/sda1
|
| Posted by hichamnaima, 03-30-2011, 05:34 PM | | hdparm -tT /dev/md0
hdparm -tT /dev/sda1
|
| Posted by Rapid2214, 03-30-2011, 05:38 PM | | Run
"ps aux | awk '{print $4"\t"$11}' | sort | uniq -c | awk '{print $2" "$1" "$3}' | sort -nr"
This will check the most memory intensive processes.
|
| Posted by hichamnaima, 03-30-2011, 05:42 PM | |
|
| Posted by Rapid2214, 03-31-2011, 03:41 PM | | If you are serving just images use nginx
|
|
 Add to Favourites
 Print this Article
|
