|
File write caching |
Top Previous Next |
|
File write caching considerably improves the performance of the "Log to file" action under heavy message load.
When enabled, the "Log to File" action will cache the output data for X seconds or X messages before writing to the log file. The data is cached in memory until the log file is updated in bulk. This is more efficient than writing a single message to a file as it arrives.
There is a separate memory cache for each output file. In most cases there is only a single output file, but if AutoSplit or filters are used to split the messages into separate files, there could be additional active output files.
When an output file cache is not being used X seconds, the cache is destroyed to save resources.
When the program shuts down, all the caches are written to the appropriate files so that no data is lost.
Enable File write caching
Section: HKEY_LOCAL_MACHINE\SOFTWARE\SolarWinds\Syslogd\Properties
Value (STRING): FileWriteCacheEnabled
Min value: 0 Max value: 1 Default value: 1 Type: Enabled = 1, Disabled = 0
When enabled, the "Log to File" action will cache the output data for X seconds or X messages before writing to the log file. The data is cached in memory and the log file is updated in bulk. This is more efficient than writing a single message to a file as it arrives.
Cache timeout
Section: HKEY_LOCAL_MACHINE\SOFTWARE\SolarWinds\Syslogd\Properties
Value (STRING): FileWriteCacheTimeout
Min value: 1 Max value: 120 Default value: 5 Type: Timeout in seconds
After the timeout period the contents of the cache are written to disk. The timer is started when the first message arrives in the cache. If the cache is not full and has not been flushed before the timeout period has expired, the cache will be flushed automatically. This value sets the maximum time that the cache will hold a message before writing it to disk. The less frequently the disk is written to, the more efficient the file logging process becomes.
Maximum number of Cache entries
Section: HKEY_LOCAL_MACHINE\SOFTWARE\SolarWinds\Syslogd\Properties
Value (STRING): FileWriteCacheEntries
Min value: 10 Max value: 100,000 Default value: 1000 Type: Maximum number of cache entries (messages)
Sets the maximum number of messages to be cached for each output file before being written to file.
Messages are added to the cache until the maximum is reached or the timeout period elapses. The less frequently the disk is written to, the more efficient the file logging process becomes. The messages are stored in memory in UNICODE which requires two bytes for each character in the message. For example, a 100 character message requires 200 bytes of memory for storage.
Maximum memory size per cache
Section: HKEY_LOCAL_MACHINE\SOFTWARE\SolarWinds\Syslogd\Properties
Value (STRING): FileWriteCacheMaxSizeKB
Min value: 1 Max value: 2,000 Default value: 50 Type: Maximum size in KBytes for each cache
Sets the maximum cache size in KBytes. When the cache exceeds this size, it is written to file.
Messages are added to the cache until the maximum memory size is reached or the timeout period elapses. The less frequently the disk is written to, the more efficient the file logging process becomes. The messages are stored in memory in UNICODE which requires two bytes for each character in the message. For example, a 100 character message requires 200 bytes of memory for storage. If you experience any "Out of Memory" errors, lower this value or disable the file write caching.
Cache cleanup time
Section: HKEY_LOCAL_MACHINE\SOFTWARE\SolarWinds\Syslogd\Properties
Value (STRING): FileWriteCacheCleanup
Min value: 10 Max value: 1440 Default value: 10 Type: Time (in minutes) that a cache can inactive before being destroyed
When a cache becomes inactive and is not receiving any further messages, the cleanup process will destroy the cache to free up resources. No data is lost because the cleanup process only destroys inactive caches that have already been written to file.
Log file locking
Section: HKEY_LOCAL_MACHINE\SOFTWARE\SolarWinds\Syslogd\Properties
Value (STRING): FileWriteCacheFileLock
Min value: 0 Max value: 1 Default value: 0 Type: Enabled = 1, Disabled = 0
For efficiency and security reasons, the log files can be held open in "append shared" mode. This improves efficiency by not having to open and close the file with each write. While the file is held open, not other application can modify or delete the contents. Only new entries can be added to the file. The files can be opened for viewing, but not for modification.
If you are receiving high syslog message traffic, enable this option to improve performance. The only drawback is that the file may not immediately show the new log entries. The OS will cache the data until the internal buffers are full then it will write the buffers to file. Under heavy load, this happens immediately, but when traffic is low, it can take a while for the buffers to fill and the data to be written. The log file is automatically updated and closed when the cache has been inactive for FileWriteCacheCleanup minutes.
Maximum number of open log files
Section: HKEY_LOCAL_MACHINE\SOFTWARE\SolarWinds\Syslogd\Properties
Value (STRING): FileWriteCacheOpenFiles Min value: 1 Max value: 250 Default value: 100 Type: Maximum number of open file handles
When FileWriteCacheFileLock is set to 1 (enabled), each log file is held open in "append shared" mode. The program can only open a maximum of 255 files at once. This value sets the maximum number of concurrently open files. Once this limit is reached, the FileWriteCacheFileLock value for the current cache is disabled. Log files will then be opened and closed with each cache write. If the Log to File action uses the AutoSplit syntax to create separate files for each logging host, it is possible that more than 255 files could be opened at once (assuming more than 255 actively sending hosts). A value of 100 files is recommended to keep system resource usage to a reasonable level.
|