What is Graylog? Why should I use it?

Logging is a topic that does not get enough attention in the PowerShell community. When I first started writing PowerShell, I did not use any logging. I did not have any record of what my scripts did or if there were any failures during execution. I changed things up for a while, writing adhoc logging functions for each script that I wrote. However, these did not always include all the needed information. The biggest problem I had was that I never would check the logs until I realized something wasn’t working. I had no way of knowing if anything failed.

I took notice that our development team had a standard logging assembly that was used across all of our custom .NET applications. That is what we needed on my infrastructure team as well. I created an internal module that imports the other dependent modules and has a few functions that standardize our logging. This module is used in all of our PowerShell jobs and automation.

We use Graylog as our log management software. Graylog is the closest Open Source product to Splunk. It allows us to store, analyze, and alert on log data. We mainly use Graylog to gather Windows Server event logs, F5 web request logs, and logs from our custom .NET applications. A few years ago I wrote a module to send data to Graylog using the GELF protocol. This originally was used in a script that would query log data from SQL and then send it to Graylog.

Using PSGELF to send log data to Graylog

Here are few examples using the basic PSGELF module to send logs.

Extremely Simple example

Import-Module PSGELF

Send-PSGelfTCP -GelfServer graylog -Port 12202 -ShortMessage "This is a short Message"

Querying logs from SQL to send to Graylog

$sqlConnection = New-SQLConnection -ServerInstance SQLServer -Database "ApplicationLogs"

$query = "SELECT * ,DATEDIFF(s, '19700101',DATEADD(hour, +6, DateEntered)) AS TimeStamp FROM AppsLog"

Invoke-Sqlcmd2 -Query $query -SQLConnection $sqlConnection -As PSobject | Send-PSGelfUDPFromObject -GelfServer graylog -Port 12202

Example Internal Module

The internal module, uses PSGELF and EsOsO’s Logging module as the base. EsOsO’s module starts a new thread specific for logging to avoid bottlenecking. He also has support for custom targets. I wrote a custom target that uses PSGELF. Most of my scripts use the Graylog target and the File target. I have uploaded an example of the internal module here. The Owner, Environment, and ScriptPath are used in Graylog to alert appropriately.

Example Script using the internal module

Import-Module CNPSLogger
Use-CNPoshLogging -Owner "Infrastructure" -Environment "Testing" -ScriptPath "C:\scripts\example.ps1"

Try{
    Get-ChildItem "C:\balsdfsdfsd" -ErrorAction Stop
}Catch{
    Write-Log -Message $(New-ErrorString $_) -Level ERROR

    #If you want to stop the script use this function.
    Exit-CNPoshLogging
}

#The logging module uses a seperate thread, this makes sure all logs are written before exiting.
Wait-Logging

Below is what the logs look like inside Graylog.

pi1