Securing Citrix Gateway using Citrix ADC Bot Management, Citrix Web Application Firewall and DOS-Protection

S

last update: November 18th 2021

Recently, I had been asked, how to protect a gateway from threads. It’s easy, I thought, Citrix ADC has everything needed in good quality: A Bot Management, Web Application Firewall (WAF), and AppQoE (Application quality of experience, a DOS protection feature). So nothing easier than that: Create the policies desired and bind them to the gateway.

Shortly after starting, I found out, it’s not possible to bing policies like that. They can only get bound to a load-balancing or content-switching vServer.


Binding the Policies

It takes a trick to bind these policies. A dirty trick, to be honest.

Have you ever tried to use the WAF wizard? No? That’s great. Never do so. You won’t get rid of the policy you created using the GUI. You won’t even find the bind point. It’s bound to a bind-point called None(?). GUI will not allow unbinding from there (13.0 built 82.42). Command-line helps to find, what’s been going on:

show ns runningConfig | grep appfw_pol_WAF-Wizzard
bind appfw global appfw_pol_WAF-Wizzard 62990 END -type REQ_DEFAULT

It’s bound globally, but the type is REQ_DEFAULT. So we can unbind it easily from the command-line:

unbind appfw global appfw_pol_WAF-Wizzard

We need to use this trick to proceed. Our policies need to be bound globally, with -type REQ_DEFAULT. This can only be done from the command line. And of course, the policy expression has to contain the hostname of the gateway (HTTP.REQ.HOSTNAME.EQ("gw.mycompany.com")), so these policies don’t interfere with other vServers hosted on this ADC.


Bot Management

Maybe I’ll later create a more detailed blog about Bot Management. Bot Management deals with crawlers like Google bot, Bing bot, and several more. It allows you to select which of the well-known bots are desired and which ones are not. Have you ever tried to search Google for the term “Citrix Gateway”? You’ll find billions of them. Some branded, some not, but in most cases, the URL allows to guess, to whom it belongs. Even more: in the case, example.lab got its gateway indexed by Google, the term "citrix gateway" site:example.lab will give you the URL of all of example.lab’s gateways. In most cases, this is not desired (to say the least). So it would be a good idea to drop all connections from each and every crawler.

Citrix Bot Management comes with a more or less complete list of well-known bots. There are thousands of bots. And if we need to hide a site, we don’t like any of them to go there. So our first task would be: Create a new bot definition, denying access to all of them. Go to Bot Management -> signatures, check the checkbox for *Default Bot Signatures and click clone. This will create a JSON file containing all well-known bots. Before actually creating it, it will show the JSON file to you. I found out: The fastest way to disable all well-known bots is to copy this file into notepad++ and proceed there.
...
{
"version": "2.1",
"name": "AddThis.com",
"drop": "FALSE",
"redirect": "FALSE",
"reset": "FALSE",
"id": "2",
"type": "Good Bot",
"category": "Crawler",
"log": "TRUE",
"enabled": "TRUE",
"developer": "Oracle"
},eloper": "Mirafox"
...

This is just a small snippet of this file. You see, the AddThis.com bot is a “good one” (at least, that’s what Citrix thought, I won’t express my thoughts about it as they might institute proceedings against me), so connections would not drop. To make them drop, I have to change the term "reset": "FALSE" into "reset": "TRUE". I will copy/paste the content into Notepad++. Search and replace will set all “reset” to true within seconds for me. I’ll just have to paste the content back in, create a profile and a policy, and assign the policy to your gateway: All bots get reset, so you’ll never see your gateway on Google or any other search engine.

Usually, we don’t have just a gateway on the Citrix ADC (NetScaler), but also web servers. And we want crawlers to crawl most of them. However, we can’t bind the bot policy to the gateway, so we need to bind it globally, however, filter the policy based on hostname. That’s an example to filter the policy for gateway.example.lab:

HTTP.REQ.HOSTNAME.EQ("gateway.example.lab)

So that’s what my Bot-Profile would look like:

Citrix ADC / NetScaler Bot Protection Profile

And that’s the policy

Citrix ADC / NetScaler Bot Protection Policy

I am using the User-Agent switcher plugin for FireFox. If I change the user-agent to “Google-Bot”, I see a message, telling me, a secured connection can’t get established.

Bot protection could even do more, but it’s sufficient to block just well-known search engines and crawlers

If you are very concerned about bots, you could also block two tools: curl and wget. Curl is a windows tool to download websites, wget is a Linux tool with a similar purpose. Many tools simply use wget to fetch data. The respective user-agent strings would be:

  • curl/7.54
  • Wget/1.19.4 (linux-gnu)

(version numbers may vary of course)

So you would add a responder policy, action drop, with expression HTTP.REQ.HEADER("User-Agent").SET_TEXT_MODE(IGNORECASE).CONTAINS("curl") || HTTP.REQ.HEADER("User-Agent").SET_TEXT_MODE(IGNORECASE).CONTAINS("wget")


Web Application Firewall

The first question has to be: Do we trust authenticated users?

While this is usually the case with small to medium-sized businesses, it might not be true for a huge enterprise with thousands of users connecting: employees, contractors, and probably even some probably completely untrusted ones. Logically, we would just protect the login process, if we may trust authenticated users, while we would also protect everything behind in case we can’t.


Protecting the authentication process

Protecting an authentication process using the WAF is surprisingly difficult. The reason is, the WAF is processed only after the logon had been successful. So binding a WAF policy the way we bound the bot-protection policy, will not work as expected.

So we need a different trick. And the trick is binding an HTTP callout into the authentication policy. We will forward the original HTTP request to a callout vServer and bind a WAF policy to this server. This will not protect the AAA vServer, but the logon process gets secured.

This method has a very unpleasant side effect: it requires an HTTP load balancing vServer. This server must be active all the time. The authentication fails if this server is down. All requests are sent to this vServer and forwarded to the backend system, including username and password. The server must therefore not be of type HTTP , but must use TLS. Using the so-called “Always-On” service does not help, as the Always-On service can not respond in any way. Because of this, every callout that does not trigger the WAF policy times out and the policy returns an undefined.

I follow a Citrix white-paper, however, I did some changes, i.e. with the callout.


I don’t go into creating this vServer. I simply called it lb_vs_colors. I’m pretty sure, you can do this yourself. So my first step is:

Creating the HTTP callout.

Citrix ADC/NetScaler: HTTP callout to protect an AAA vServer using Citrix WAF
add policy httpCallout callout_waf -vServer http_vs_callout -returnType BOOL -fullReqExpr "HTTP.REQ.FULL_HEADER + HTTP.REQ.BODY(2048)" -scheme https -resultExpr true

You see, we do an expression-based callout to this vServer. The expression we use for this callout is the full HTTP request (so all the HTTP headers plus the first 2k of the body of the HTTP request). Schema, I already mentioned it, has to be HTTPS for security reasons (don’t follow my screenshot, it’s highly insecure!). The return type will be a boolean, a true. So, it returns true, if the callout does not get blocked by the waf policy.

If the WAF is hit, the callout will not return a true, instead, a non-boolean return type. Because of this, the policy expression neither evaluates to true, nor to false but to undefined. If the policy expression evaluates to undefined, the policy won’t get invoked.

If we use this trick with an authentication policy, authentication is only possible, if the WAF policy does not block the request. And that’s what a user will see: “No active policy during authentication”. The user’s logon will fail.

Citrix NetScaler Gateway Error message, "No active policy during authentication"

The authentication policy

For my test environment, I just used a simple LDAP policy. Creating LDAP profiles is simple, so I’ll just show the policy expression:

An authentication policy using an HTTP callout fo a WAF
add authentication Policy auth_ldap -rule "SYS.HTTP_CALLOUT(callout_waf) " -action act_ldap

The original Citrix policy also evaluates the HTTP.REQ.URL, but it does not make sense to me, as the LDAP policy only gets invoked if someone posted to the login process.


If you can’t trust your users

It’s not enough to protect the login process if you are not allowed to trust authenticated users. You’ll have to protect the gateway from attacks occurring post login. You’ll have to follow my guidelines for bot detection, to bind a WAF policy to a gateway. You may evaluate the logon cookies to distinguish between authenticated and non-authenticated users. The policy expression to select authenticated users would look like that:

HTTP.REQ.HOSTNAME.EQ("gateway.example.lab") && HTTP.REQ.COOKIE.CONTAINS("NSC_AAAC").NOT


Which security checks would you use?

The profile has to be of type HTML and XML. HTML is used for browser-based login, XML for apps like Citrix Workspace App.

Protecting the logon point is not that hard. I would define all the input fields (Allow all characters for the password field, but demand at least 6 and a maximum of 128 .{6,128} and only 2 to 128 word characters for the username \w{2,128}). Turn on learning and off blocking for all security checks possible. Don’t forget to set Buffer Overflow! Buffer overflow might be an attack vector. Be careful about SQL injection and Cross Site Scripting checks, they may cause false positives with the password field.

If you are scared of your authenticated users, the profile has to get way more complex.

Security checks I’d suggest to use as a minimum:

  • StartURL
  • DenyURL
  • Cookie protection (to protect the session cookie)
  • Field Formats (to make sure, just allowed characters appear in username and password fields. Keep in mind: Your users may use foreign language keyboards, so passwords may contain characters you didn’t think of)
  • Buffer overflow
  • XML format (workspace app widely uses XML)
  • XML deny of service

Never use cookie theft, as this one does not work with all browsers and TLS 1.3 (see limitations here)


AppQoE

Probably, AppQoE is the most important one of all these features. Similar to the other ones, AppQoE policies can’t get bound to the gateway directly but needs to get bound globally. AppQoE is a DOS/DDOS protection feature. It helps to avoid overload caused by attackers.

QppQoE policies, similar to bot management and WAF, can’t get bound to the gateway directly. So we have to use the same dirty trick I’ve used before: They have to get bound globally and filtered, probably a bit different this time.

I am quite sure, the gateway itself can hardly be overloaded. The ADC is neither vulnerable to SYN flood attacks, nor to slow attacks. The logon process, or backend components used by the logon processĀ  and it’s components like SAML IDP, LDAP server, RADIUS server, however, may die due to too many requests. So we need to block malicious requests to the logon process.

Logon attempts get posted to /nf/auth/doAuthentication.do. So the policy expression will be HTTP.REQ.URL.EQ("/nf/auth/doAuthentication.do"). We don’t really need to filter on the domain. All aaa vServers use the same authentication URL, so all would be protected by the same policy, but it’s rather unlikely for any other website to use the URL by chance.

I have already written, how to set up AppQoE, so I skip it here. Just keep on reading here.


Like always, I’d be happy to read about your thoughts and concerns. Dust drop me a comment!

About the author

Johannes Norz

Johannes Norz is a Citrix Certified Citrix Technology Advocate (CTA), Citrix Certified Instructor (CCI) and Citrix Certified Expert on Application Delivery and Security.

He frequently works for Citrix international Consulting Services and several education centres all around the globe.

Johannes lives in Austria. He had been borne in Innsbruch (https://www.youtube.com/watch?v=UvdF145Lf2I)

Add comment

By Johannes Norz

Recent Posts

Recent Comments