Rsyslog - Logging to a Remote Server

Ubuntu has 'rsyslog' installed by default which contains the facility to act as a remote logging server, as well as act as a client that can log to a remote server. 'rsyslog' uses port 514 (either UDP or TCP) to log messages.

Key things to remember:

  1. The client and server configuration files need to be updated -- '/etc/rsyslog.conf'.
  2. The server's security group (firewall rules) needs to allow connections to UDP or TCP port 514 from logging clients.
  3. Realistically, each Apache site configuration file needs to be updated in order to log to 'syslog'.

Configuring the remote server

The 'rsyslog' configuration of the remote server needs to be changed to allow remote connections, and configure where log messages will go.

Edit the '/etc/rsyslog.conf' file so that the modules section is as below: 

#################
#### MODULES ####
#################

module(load="imuxsock") # provides support for local system logging
#module(load="immark")  # provides --MARK-- message capability

# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")

# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="514")

This should only involve uncommenting some of the lines above.

Then underneath, add the following lines between the "GLOBAL DIRECTIVES" comment and the "Use traditional timestamp format.' comment.

Change YOUR_CLIENT_IP_ADDRESS_RANGE to an appropriate address range, e.g., '172.1.2.0/24'. 

###########################
#### GLOBAL DIRECTIVES ####
###########################

$AllowedSender UDP, TCP, 127.0.0.1, YOUR_CLIENT_IP_ADDRESS_RANGE

$template remote-incoming-logs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log"
*.* ?remote-incoming-logs
& ~

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#

That is all that needs to be configured on the server. Restart the daemon using: 

$ sudo service rsyslog restart

Optional: Logging to MySQL from Rsyslog

It is also possible and fairly simple to log to MySQL from 'rsyslog'. This can achieved in approximately four steps:

  1. Install MySQL/Mariadb.
  2. Login to MySQL/Mariadb and create an appropriate database.
  3. Create a MySQL user for 'rsyslog' and grant it appropriate permissoins.
  4. Install the 'rsyslog-mariadb' package.

Read on for more details.

First, install MySQL or Mariadb: 

$ sudo apt install mariadb-server

Then, login: 

$ sudo mysql
MariaDB>

Paste into the MySQL the contents of the file createDb.sql, which is provided under the 'plugins' directory of the 'rsyslog' source.

Then create an appropriate user account: 

MariaDB> CREATE USER 'rsyslog'@'localhost';
MariaDB> GRANT INSERT ON Syslog.* TO 'rsyslog'@'localhost';

Finally, on Unbutnu, you can simply install the 'rsyslog-mysql' package. 

$ sudo apt install rsyslog-mysql

This will add the required modules; and add a configuration file at: '/etc/rsyslog.d/mysql.conf'.

For generic instructions, See the reference below [3] for generic instructions that appy to a range of operating systems and databases.

Configuring the client server

On the client, edit the '/etc/rsyslog.conf' file.

Again, between the "GLOBAL DIRECTIVES" comment and the "Use traditional timestamp format.' comment, add the following lines, replacing YOUR_SERVER_IP_ADDRESS with your server's IP address. 

###########################
#### GLOBAL DIRECTIVES ####
###########################

*.* @YOUR_SERVER_IP_ADDRESS:514

$ActionQueueFileName       queue
$ActionQueueMaxDiskSpace   1g
$ActionQueueSaveOnShutdown on
$ActionQueueType           LinkedList
$ActionResumeRetryCount    -1

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#

Be sure to keep the '@'. Note, add an additional '@' symbol to log using TCP. For example. 

*.* @@YOUR_SERVER_IP_ADDRESS:514

Configuring Apache websites

Important: for many operating system default configurations, doing the following will start also outputing error_log messages to '/var/log/syslog' on the client machine. To prevent this, on the client machine, edit '/etc/rsyslog.d/50-default.conf', and change the following line, replacing 'local6' with whatever facility you specify in your Apache configuration below: 

*.*;auth,authpriv.none      -/var/log/syslog
to: 
*.*;auth,authpriv.none;local6.none      -/var/log/syslog

Note, the following seems to override any other ErrorLog or ErrorLogFormat directives. In each Apache site configuration file, add the following to an appropriate Directory element replacing 'YOUR_IDENTIFIER' with the name you want for the log file without '.log'. E.g. 'my_websiite'. 

<Directory /path/to/website/>
php_value       error_log       syslog
php_admin_value syslog.ident    YOUR_IDENTIFIER
php_admin_value syslog.facility local6
</Directory>

If you want to continue logging to the client machine, add an 'rsyslog' configuration file - say, '/etc/rsyslog.d/70-local6.conf' - similar to below: 

# Log local6 generated log messages to file
local6.*                          /var/log/local6.log

#OR for each website:

:syslogtag, isequal, "YOUR_IDENTIFIER:" /var/log/YOUR_IDENTIFIER.log

& stop

Configuring PHP command line programs

Important: for many operating system default configurations, doing the following will start also outputing error_log messages to '/var/log/syslog' on the client machine. To prevent this, on the client machine, edit '/etc/rsyslog.d/50-default.conf', and change the following line, replacing 'local6' with whatever facility you specify in the call to 'openlog' below: 

*.*;auth,authpriv.none      -/var/log/syslog
to: 
*.*;auth,authpriv.none;local6.none      -/var/log/syslog

Using 'syslog' in PHP command line programs is relatively straight forward. Within your code you first need to initialise logging by calling the 'openlog' function. Then, whenever you want to log something, you call the 'syslog' function. Finally, before the program finishes, call the 'closelog' function. 

openlog( "PROGRAM_NAME", LOG_PID | LOG_PERROR, LOG_LOCAL6 );
...
syslog( LOG_ERR, "Your log message" );
...
closelog();

Refer to the references below for details. In short:

  • LOG_PID - causes the process identifier to be logged with each message.
  • LOG_PERROR - causes the message to be also logged to the std out.
  • LOG_LOCAL6 - causes the message to be logged to the 'local6' facility.

By default, if no configuration has been changed, log lines will usually log to '/var/log/syslog'.

The same as the Apache instructions above, to log all 'local6' log lines to one file or log a program to its own separate log file, you could add an 'rsyslog' configuration file - say, '/etc/rsyslog.d/70-local6.conf' - similar to below: 

# Log local6 generated log messages to file
local6.*                          /var/log/local6.log

# Log program to its own file
:programname, isequal, "PROGRAM_NAME" /var/log/MY_PROGRAM_LOGFILE_NAME.log

& stop

If it is not clear, the programname you use in the PHP program, must match the value tested using 'isequal'; and the log facility you use in the PHP program, e.g., 'LOG_LOCAL6', must match the log facility used in the rsyslog configuration file, e.g., 'local6.*'.

References

The following websites were referred to while developing this content:

  2. How to Set Up Remote Logging on Linux Using rsyslog
    https://www.makeuseof.com/set-up-linux-remote-logging-using-rsyslog/
  3. Writing syslog messages to MySQL, PostgreSQL or any other supported Database
    https://www.rsyslog.com/doc/v8-stable/tutorials/database.html
  4. Log Files - Apache HTTP Server Version 2.4
    https://httpd.apache.org/docs/2.4/logs.html
  5. How to change configuration settings (PHP Manual)
    https://www.php.net/manual/en/configuration.changes.php
  6. List of php.ini directives (PHP Manaul)
    https://www.php.net/manual/en/ini.list.php