Last update: December 22^nd 2021

Many of us, today, struggle with the Log4J security issue (CVE-2021-44228). It will take a long time to fix all apps, as the Apache log4J framework is built deep into several apps. For many of my customer’s apps, it’s still not clear, if whether they are affected, or not. At the same time, there are already exploits out there, allowing attackers to get shell access to affected machines. So what to do?

I found a pretty good analysis from Qualis.

We could use Citrix ADC’s (NetScaler’s) WAF feature to mitigate. Citrix recently released 4 signatures #999077-9990780 for their WAF. These signatures will mitigate the issue (Credits to Ronan O., he posted it on LinkedIn). Just add four simple rules, and your application is back to a safe state. I would really do this, no matter, if your applications are affected, or not (we still don’t know this for most of our applications).

In Parallel, Citrix released a paper about the security of their own products. The good news: Neither Citrix ADC, nor Citrix ADM, uses Log4J. Therefore, these products are not affected. At the very bottom, where you stopped reading long ago because you thought everything had already been said, they published a responder policy. It is there no more, and I don’t know why it’s gone, but it seems worthwhile applying. So we have 2 approaches: The WAF and the responder approach.

Warning: This might not be a 100 % secure solution. But it might be the best solution we currently have! Read the original by Ronan O., on LinkedIn, including comments, for concerns. This is mainly about the WAF-solution, but may also be true about the responder-solution. I also have added some guidelines about testing.

What to do? The responder-solution

The responder policy uses a pattern set. It contains the strings typical for this exploit. The policy expression looks very confusing (if it weren’t for the pattern-set, it would look even more confusing).

This is the pattern-set:

add policy patset patset_cve_2021_44228
bind policy patset patset_cve_2021_44228 ldap
bind policy patset patset_cve_2021_44228 http
bind policy patset patset_cve_2021_44228 https
bind policy patset patset_cve_2021_44228 ldaps
bind policy patset patset_cve_2021_44228 rmi
bind policy patset patset_cve_2021_44228 dns
So you see it’s a list containing the usual verbs.

add responder policy res_pol_mitigate_exploit_cve_2021_44228
"HTTP.REQ.FULL_HEADER.SET_TEXT_MODE(URLENCODED).DECODE_USING_TEXT_MODE.AFTER_STR(\"${\").BEFORE_STR(\"}\").CONTAINS(\"${\")
||
HTTP.REQ.FULL_HEADER.SET_TEXT_MODE(URLENCODED).DECODE_USING_TEXT_MODE.SET_TEXT_MODE(IGNORECASE).STRIP_CHARS(\"${:
}/+\").AFTER_STR(\"jndi\").CONTAINS_ANY(\"patset_cve_2021_44228\") ||
HTTP.REQ.BODY(8192).SET_TEXT_MODE(URLENCODED).DECODE_USING_TEXT_MODE.AFTER_STR(\"${\").BEFORE_STR(\"}\").CONTAINS(\"${\")
||
HTTP.REQ.BODY(8192).SET_TEXT_MODE(URLENCODED).DECODE_USING_TEXT_MODE.
SET_TEXT_MODE(IGNORECASE).STRIP_CHARS(\"${:
}/+\").AFTER_STR(\"jndi\").CONTAINS_ANY(\"patset_cve_2021_44228\")" DROP
-logAction log_CVE2021_44228
The policy does a simple drop. That’s absolutely fine to do, an attacker does not need much feedback. The -logaction parameter is mine. I always log security-related information, that’s why I’m using it. Of course, there has to be a logging policy as well, and it has to be created ahead of creating the policy:

add audit messageaction log_CVE2021_44228 WARNING
"\"CVE2021_44228 exploit from \" + CLIENT.IP.SRC + \" with https://\" +
HTTP.REQ.HOSTNAME + HTTP.REQ.URL + \" Body: \" + HTTP.REQ.BODY(4096) +
\" Headers: \" + HTTP.REQ.FULL_HEADER"
You see, I log the client IP, the HTTP method, the URL (including all query strings), a potential HTTP body, and all headers. Don’t forget to turn on user-configurable log messages!

This logs a message starting with CVE2021_44228, as it’s always important to have a string like that to grep for. This is the typical outcome of this log (In this case, the problem was in the headers, I blurred sensitive information, as the log came from a customer’s ADC):

I bound this responder policy globally, so it affects everything going into my webserver.
bind responder global res_pol_mitigate_exploit_cve_2021_44228 10000 END -type REQ_DEFAULT

What to do? The WAF-solution

If you still don’t have WAF enabled, you should enable the feature.

enable feature AppFW

Next, navigate to Security → Citrix WebApp Firewall → Signatures. Click update signatures. To create new signatures, select the default signatures, click add.

enable the four security checks 999077 to 999080. Save the signature file.

If you don’t already use WAF, navigate to Security → Citrix WebApp Firewall →Profiles. Create a new profile, type default, HTML. Open “Security Checks” on the far right side and clear all checkboxes (that’s important to not destroy your application!). Open “Profile Settings” on the far right side, and add these signatures to the profile (signatures are almost at the end of these settings). Click OK.

Last, navigate to Security → Citrix WebApp Firewall → Policies. Create a new policy, select true as action, and bind it to your lb vServers, or, in complex scenarios probably better, globally to your ADC. That’s it. Thank’s Citrix, you saved my weekend!

Last, navigate to Security → Citrix WebApp Firewall → Policies. Create a new policy, select true as action, and bind it to your lb vServers, or, in complex scenarios probably better, globally to your ADC. That’s it. Thank’s Citrix, you saved my weekend!

Testing

Never believe, something is secure, just because somebody says, it’s secure! So you have to test. There are already plenty of “test-strings”, but also exploits, out there. Encode these strings, using URL-Encoding. The string might be utterly everywhere: In the URL, in headers, in form fields. Use excessive percent-encoding throughout your testing: For example, you could replcae the $ sign with %24, but the % sign in %24could get replaced by %25. So a $ sign could look like %2524. You might – of course – also encode these numbers, so instead of %24 you could use %24%32%34. For example, you might use this table.