nxlog – Send Citrix UPM Logfile to graylog

One of our distant site is using 2003 mandatory profil, set with the terminalservicesprofilepath attribute,both say that we had a surprise when they logged on our 2008R2 farms… So, we had reactivated UPM logs, and we decided to send them to graylog.

UPS used to log in CSV format with these fields :

Date
Time
Severity
Domain
UserName
SessionId
ThreadId
Function : Message

Nxlog is able to read log file, using im_file module, and to parse CSV entry, using xm_csv module. To be complete the file is in UTF-16 LE, we had to convert entry in UTF-8 format, using xm_charconv module.

CSV Configuration

We configure fields of the CSV file and the delimiter:

<Extension csv>	
	Module	xm_csv
	Delimiter ';'
	EscapeChar '`'
	Fields	$date, $time, $Severity, $UserDomain, $UserName, $SessionId, $ThreadId, $Message
	FieldTypes string,string,string,string,string,integer,integer,string
</Extension>

xm_charconv

<Extension charconv>
    Module      xm_charconv
    AutodetectCharsets utf-8, utf-16, utf-32, iso8859-2 #Si besoin d'autodetect
</Extension>

Configuration of im_file

We configure the module to read file in line based mode, as we want to avoid having one file per server,we used an generic file path. And then we apply these actions :
convert line into utf8;
if not empty line, we parse csv, we set the hostname with fsqn(source in graylog) et we fill EventType field with Severity Field


<Input upm>
	Module im_file
	File 'C:\Windows\System32\LogFiles\UserProfileManager\\*_pm.log'
	SavePos TRUE
	ReadFromLast TRUE
	PollInterval 1
	InputType LineBased
	Exec	$fullMessage = convert($raw_event,"utf-16le","utf-8"); \
		if($fullMessage == ''){ drop(); } \
		else { \
			csv->parse_csv($fullMessage); \
			$EventTime = strftime(parsedate($date + " " + $time + "Z"),'%Y-%m-%d %H:%M:%S+00:00'); \
			$Hostname = hostname_fqdn(); \
			$fullMessage = $raw_event; \
			$EventType = $Severity; \
		}
	
</Input>

The whole configuration file

## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
#LogLevel DEBUG

<Extension gelf>
    Module      xm_gelf
</Extension>

<Extension syslog>
    Module	xm_syslog
</Extension>

<Extension csv>	
	Module	xm_csv
	Delimiter ';'
	EscapeChar '`'
	Fields	$date, $time, $Severity, $UserDomain, $UserName, $SessionId, $ThreadId, $Message
	FieldTypes string,string,string,string,string,integer,integer,string
</Extension>

<Extension charconv>
    Module      xm_charconv
    AutodetectCharsets ucs-2le, ascii, utf-7, utf-8, utf-16, utf-32, iso8859-2, windows-1252, ucs2
</Extension>

<Input in>
    Module      im_msvistalog
# For windows 2003 and earlier use the following:
#   Module      im_mseventlog
#Filter Events 
    Query	<QueryList>\
	<Query Id="0">\
		<Select Path="Security">*</Select>\
		<Select Path="System">*</Select>\
		<Select Path="Application">*</Select>\
		<Select Path="Microsoft-Windows-PrintService/Admin">*</Select>\
		<Select Path="Microsoft-Windows-PrintService/Operational">*</Select>\
		<Select Path="Microsoft-Windows-GroupPolicy/Operational">*</Select>\
		<Select Path="Norskale Agent Service">*</Select>\
    		<Select Path="Citrix-Multimedia-Flash/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Citrix-Multimedia-Rave/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Internet Explorer">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Key Management Service">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
	        <Select Path="Microsoft-Windows-Application Server-Applications/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
	        <Select Path="Microsoft-Windows-Dhcp-Client/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
                <Select Path="Microsoft-Windows-DhcpNap/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
                <Select Path="Microsoft-Windows-Dhcpv6-Client/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-Diagnosis-Scripted/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-EnrollmentPolicyWebService/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-EnrollmentWebService/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-IIS-Configuration/Administrative">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-Kernel-EventTracing/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-MUI/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-PowerShell/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-RemoteApp and Desktop Connections/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="microsoft-windows-RemoteDesktopServices-RemoteDesktopSessionManager/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-TerminalServices-ClientUSBDevices/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-TerminalServices-LocalSessionManager/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-TerminalServices-PnPDevices/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-TerminalServices-RemoteConnectionManager/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-TerminalServices-SessionBroker-Client/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-TerminalServices-TSAppSrv-TSMSI/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Microsoft-Windows-TerminalServices-TSAppSrv-TSVIP/Admin">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="OAlerts">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="Symantec Endpoint Protection Client">*</Select>\
		<Select Path="Windows PowerShell">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="XenApp Cloning Service">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
		<Select Path="HardwareEvents">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
	</Query>\
</QueryList>\

    Exec if $EventType == "AUDIT_SUCCESS" drop();
</Input>

<Input upm>
	Module im_file
	File 'C:\Windows\System32\LogFiles\UserProfileManager\\*_pm.log'
	SavePos TRUE
	ReadFromLast TRUE
	PollInterval 1
	InputType LineBased
	Exec	$fullMessage = convert($raw_event,"utf-16le","utf-8"); \
		if($fullMessage == ''){ drop(); } \
		else { \
			csv->parse_csv($fullMessage); \
			$EventTime = strftime(parsedate($date + " " + $time + "Z"),'%Y-%m-%d %H:%M:%S+00:00'); \
			$Hostname = hostname_fqdn(); \
			$fullMessage = $raw_event; \
			$EventType = $Severity; \
		}
	
</Input>

<Output out>
    Module      om_udp
    Host        mygraylog
    Port        12201
    OutputType	GELF
</Output>

<Route 1>
    Path        in,upm => out
</Route>

CSV Configuration

xm_charconv

Configuration of im_file

Leave a Reply Cancel reply