Logging more detailed data about websites blocked by NetScaler Web Application Firewall (WAF)

L

last update: April 16th 2018

I had been asked recently: Johannes, how can we log data about NetScaler Application Firewall policy hits in detail?


The standard NetScaler Web Application Firewall log-files

NetScaler’s Web Application Firewall logs to /var/log/ns.log. These logs are fine for trouble shooting. There is a good description about these logs here. This is a sample log, stolen from a Citrix blog about NetScaler Web Application Firewall (WAF) logging:

Jun 22 19:14:37  10.217.31.98 06/22/2015:19:14:37 GMT ns 0-PPE-1 :d
efault APPFW APPFW_XSS 60 0 :  10.217.253.62 616-P
PE1 y/3upt2K8ySWWId3Kavbxyni7Rw0000  pr_ffc https://aaron.stratum8.n
et/FFC/login.php?login_name=abc&passwd=12345&drinking_pref=
on&text_area=%3Cscript%3E%0D%0A&loginButton=ClickToLogin&am
p;as_sfid=AAAAAAWEXcNQLlSokNmqaYF6dvfqlChNzSMsdyO9JXOJomm2vBwAMOqZI
Chv21EcgBc3rexIUcfm0vckKlsgoOeC_BArx1Ic4NLxxkWMtrJe4H7SOfkiv9NL7AG4
juPIanTvVo%3D&as_fid=feeec8758b41740eedeeb6b35b85dfd3d5def30c C
ross-site script check failed for field text_area="Bad tag: script"
 <blocked>

This log is showing a blocked cross-site script (XSS) attack. It’s a reply to a form using http GET method.


A custom error page?

Yes, this is something we could definitely do: Create a custom error page, displaying details about the reason for the attack to be blocked. It’s obvious: We don’t want to display informations like this to an attacker, so we will never use a page like this in production. However it would be a nice-to-have in a test environment. So I created a file, following CTX140293. This file can be imported easily into a WAF-Profile and will display information about transaction ID, session ID, Violation category, the log entry displayed in the event log and the corresponding session cookie. Feel free to download this file following this link.

I could think of using a page displaying the transaction ID only. So a person blocked by mistake could specify this ID to the help-desk and it would be easy for the help-desk to find this log entry in your Syslog server.


Unfortunately all this is not detailed enough during test and pre-prod / implementation phase. There is so much data missing: Cookies beyond session cookies, http PUT or POST data, headers and many more. So we have to do network traces. But traces are time consuming to read. So what to do?

I have written an other blog about logging responder and rewrite policies. Similar to responder and rewriting policies we may log app-fw policy hits. So turn on “User Configurable Log Messages” in “Change Auditing Syslog Settings”


Useful logging policies for NetScaler Web Application Firewall:

This part is outdated, as Citrix ADC version 13 brings way better methods.

Well, don’t do this during the production phase, it might lead to duplicated log entries and might print sensitive data like usernames and passwords into your logs (which is not desired). I suggest using this policies in test and pre-prod/staging environments only!

I use warning as severity as NetScalers Web Application Firewall’s messages (different to other messages) usually use warning as a severity.

Logging HTTP headers

add audit messageaction log_URL WARNING "\"Client with IP \" + CLIENT.IP.SRC + \" has accessed \" + HTTP.REQ.URL.PATH_AND_QUERY.MARK_SAFE + \" Headers \" + HTTP.REQ.FULL_HEADER"

This will result in a log entry like this:

Apr  5 09:22:50  192.168.10.101 04/05/2018:09:22:50 GMT ns_vpx_01 0
-PPE-0 : default APPFW Message 1263 0 :  "Client with IP 192.168.10
.10 has accessed /images/FormField.png Headers. GET /images/FormFie
ld.png HTTP/1.1^M Host: afweb.training.lab^M User-Agent: Mozilla/5.
0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0^M A
ccept: */*^M Accept-Language: en-US,en;q=0.5^M Accept-Encoding: gzi
p, deflate^M Referer: https://afweb.training.lab/defaultstyles.css^M 
Cookie: ASPSESSIONIDQQDTACAQ=OKDFMLOCFLFKBKOINCMPPMPN; ASPSESSIONIDS
SATBCBQ=KPCFHFPCOOMBBJOINHDNNLHC^M Connection: keep-alive^M Cache-Co
ntrol: max-age=0^M ^M "

So we can see not just the usual data, but also all HTTP headers in the log.

Logging HTTP POST data

add audit messageaction log_URL WARNING "\"Client with IP \" + CLIENT.IP.SRC + \" has accessed \" + HTTP.REQ.URL.PATH_AND_QUERY.MARK_SAFE + \" HTTP-Body: \" + HTTP.REQ.BODY(HTTP.REQ.CONTENT_LENGTH)"

Apr  5 09:56:34  192.168.10.101 04/05/2018:09:56:34 GM
T ns_vpx_01 0-PPE-0 : default APPFW Message 1593 0 :  "Client with
 IP 192.168.10.10 has accessed /cookie.asp HTTP-Body: name=Johanne
s;password=You'll_never_guess"

So we can nor clearly see all HTTP post data.

Logging both, full HTML-Headers and Post/Put data

add audit messageaction log_URL WARNING "\"Client with IP \" + CLIENT.IP.SRC + \" has accessed \" + HTTP.REQ.URL.PATH_AND_QUERY.MARK_SAFE + \" || Headers \" + HTTP.REQ.FULL_HEADER + \" || HTTP-Body: \" + HTTP.REQ.BODY(HTTP.REQ.CONTENT_LENGTH)"


I’d be happy to see some notes, remarks, feel free to tell me if you like my blog article, or if you don’t agree. Comments are a thank you to the blogger! Feel free to share this blog!

About the author

Johannes Norz

Johannes Norz is a Citrix Certified Citrix Technology Advocate (CTA), Citrix Certified Instructor (CCI) and Citrix Certified Expert on Application Delivery and Security (CCE-AppDS).

He frequently works for Citrix international Consulting Services and several education centres all around the globe.

Johannes lives in Austria. He had been borne in Innsbruck, a small city (150.000 inhabitants) in the middle of the most beautiful Austrian mountains (https://www.youtube.com/watch?v=UvdF145Lf2I)

3 comments

  • Hi Johannes.

    Really thank you for your post about APPFW Logging.

    This is really helpful.

    By the way, I configured “logging HTTP header” on my test lab, result log is truncated when HTTP header is too long.

    This happen when typically cookie is long.

    As my test, max syslog length is 1099 byte on external syslog server.

    On internal syslog server(Netscaler it self) is little longer but also truncated.

    Could you handle this issue?

    I want to see HTTP full header and body if possible.

    Nstrace cannot show past packet nor always be turned on.

    Thank you.

    • I guess, the limit on your external syslog server is due to restrictions there. I would have to do a network trace to see. I didn’t know about messages getting truncated, but if they have I have no idea how to overcome this (and currently no time to find out). I’m sorry :'(

      • Hi Johannes.

        Thank you for your fast reply.

        I cannot find out solution by myself anymore. My knowledge is exhausted:)

        So I will post this to Citrix Discussion.

        Thanks anyway.

        Have a good day!!

Recent Posts

Recent Comments