| Posted by MACscr, 04-28-2010, 03:01 PM | | I am wanting to setup a central server that will store practically real time copies of all the different types of logs on cpanel servers (apache error/access, CSF/LFD, messages, etc). I have tried to play around with rsyslog and syslog-ng, but had to stop with syslog-ng when I realized it didnt do any encryption and then rsyslog when i found no good way get it to work with apache. I know there must be other people out there that are doing this, so any ideas would be appreciated on how to accomplish it without breaking cpanel, etc. Im close to breaking down to just rsyncing the files, but thats not an ideal solution.
|
| Posted by LiquidWebBenny, 04-28-2010, 03:32 PM | | According to this article, you can use stunnel to encrypt traffic with syslog-ng. Have you tried that?
http://www.enterprisenetworkingplane...le.php/3598146
A quick google shows a few other people that have tried it, though I haven't personally.
|
| Posted by MACscr, 04-28-2010, 03:56 PM | | To me, stunnel is just a waste of resources and just adds another complexity layer to things. We also basically had the same problems with apache ad we did with syslog-ng as well. Other reasons for not using syslog-ng would be the fact that its kind of hard to consider it purely open source since its controlled by a commercial entity now and rsyslog is officially included with the CentOS distribution now (though we had to use a third party rpm to get the newest).
Benny, thanks for the quick response. You guys using any type of central logging at your location?
|
| Posted by LiquidWebBenny, 04-28-2010, 04:06 PM | | That makes sense, for sure, but since you'd said it didn't do any encryption I thought I'd offer some help.
Yup, we use both syslog, and syslog-ng to do our centralized logging, in different applications. What do you plan to do with the logs once you centralize them? If you're just looking for a good way to monitor them, this looks like something you might be able to install on each server to accomplish that.
http://www.ossec.net/
|
| Posted by MACscr, 04-28-2010, 05:07 PM | | My main goals are to have an offsite copy of every log that just in case a server gets compromised, we have a realtime cpoy of the logs (though i do understand that they can be "poisoned". I also am looking to possibly use splunk to analyze those logs.
Key requirements:
1) Compatible with cPanel (though many of the servers dont use cPanel), so regular logging needs to stay in tacked im assuming, so this would basically just be a way to send the log data to a secondary source (central logging server)
2) Logging to remote server needs to be encrypted. Would be foolish to pass that type of data over WAN connection unencrypted.
3) Data needs to be stored in files at the destination server (aka, no central mysql db, etc).
I really think rsyslog is the answer, but unfortunately I dont think there is a good sway to implement it on a controlled environment like a cpanel server where you cant modify things as much as you would like. If everything was going through the syslog, it would be much easier, but thats not how cpanel works.
Last edited by MACscr; 04-28-2010 at 05:17 PM.
|
| Posted by Hoopla-Brad, 04-28-2010, 09:30 PM | | Splunk should be able to do this. As well as monitoring log files it can also monitor ports, ie Syslog/514
Not sure what your budget is but free version allows "Data Forwarding" but has a limit of 500MB/Day of indexing volume.
We use it at work to monitor AAA logs, over 52 million entries and searching is still lightning fast. Even with autocomplete!
If you do go down this path please let me/us know what happens
http://www.splunk.com
http://www.splunk.com/web_assets/pdf...Tech_Brief.pdf
|
| Posted by MACscr, 04-29-2010, 01:20 AM | | Yeah, i know i can do it directly with splunk, but I want to be able to use the records for other purposes and if splunk doesnt work out or we go above our limit, we can easily switch to something else since we will already have the central logging in place.
|
| Posted by ZenMonk, 04-29-2010, 01:50 AM | | IMHO, I think scripts using rsync is the best bet that you have since its the only tool that give you the flexibility that you need.
|
| Posted by Steven, 04-29-2010, 09:38 PM | | Honestly I wouldn't run apache logs piped into syslog. There is a performance hit depending on how busy the server is. I'd rdiff-backup the logs.
|
|
 Add to Favourites
 Print this Article
|
