NetScaler Gateway: Internal Server Error 43531

After upgrading a HA Pair of NetScalers from 10.1 to the latest 10.5 Build (10.5-52.11 to be exact) I stumbled upon a critical Error when trying to access the Receiver for Web Site.

After authenticating successfully I would be presented with an Http/1.1 Internal Server Error 43531

After some Troubleshooting I could narrow it down to a specific Setting in my Session Policies for the NetScaler Gateway. If you used my previous Blogpost on how to configure the NetScaler Gateway for Storefront Remote Access you might also run into the same Problem.

The following Setting was working with the NetScaler 10.1 Builds:

After upgrading to the 10.5 NetScaler Build you need to edit your "Receiver for Web" Session Policy and move the Storefront URL from the "Home Page" Field under Client Experience to the "Web Interface Address" Field under Published Applications. You also need to enable "ICA Proxy" and set it to ON. Below are two Screenshots with the corrected Session Policy Settings:

I'm not sure if this is intended or just a Bug in the Code. Feel free to let me know in the Comments if you encountered the same Problem when upgrading from 10.1 to 10.5.

Configure NetScaler Gateway for Receiver Storefront Access

I have seen quite a lot of Questions in the CitrixIRC Channel on how to correctly configure the NetScaler Access Gateway (now named NetScaler Gateway) to provide access to Storefront Services (also often called CloudGateway Express) for all the different Citrix Receivers.

The following Configuration is intended for a CloudGateway Express Implementation (Storefront without the AppController). The Load Balancing of the Storefront Servers itself should already be working (see my other Blogpost on how to setup Storefront Load Balancing).

I'll just dump a lot of Screenshots of my NetScaler Gateway Configuration without any further explanations in the hopes to help someone out there in configuring the NetScaler Gateway + Storefront. If you have questions, feel free to leave a comment below.

Screenshots after the break.

INFO: Since NetScaler 10.5 you need to set the Receiver for Web Site via the Web Interface Address Field under Published Applications (and no longer via the Home Page Field as seen in the Screenshot) in the NetScaler GUI. See this Blogpost regarding the Changes.

Citrix NetScaler Storefront Load Balancing

A quick Guide on how to setup your Storefront Servers for Load Balancing under NetScaler.

The first Step is to add your Storefront Servers as Servers in the NetScaler GUI under Load Balancing -> Servers. My Storefront Servers are my existing Citrix WebInterface Servers who are currently running WebInterface 5.4 and Storefront 2.0 side by side.

The next Step is to configure the needed Storefront Monitors. Since NetScaler Release 10.1 there is a new builtin Storefront Monitor. For every Storefront Server you are going to load balance you should create a separate Monitor (see my older Blogpost for more Information).

pdate: Since Netscaler Build 10.1-123.9 the Storefront Monitor Script has been updated by Citrix and no longer requires the Hostname. So you can now use a single Storefront Monitor for all your Storefront Servers.

If you are loadbalancing a HTTPS/SSL Storefront Deployment make sure to tick the "Secure" Box when configuring the NetScaler Monitor under Standard Parameters (not shown in the Screenshot because of a Bug in the older Netscaler Builds before 10.1-123.9)

Under the "Special Parameters" Tab fill in the Hostname of your Server hosting the Storefront Services and also fill in the Store Name you choose during the initial Storefront Installation and Configuration.

Step 3 is to create your Storefront Services under Load Balancing -> Services. Bind your newly created Storefront Monitors to your Services. If you have created separate Monitors for every Storefront Server, make sure to bind the correct Monitor to the corresponding Storefront Service or else your Monitor will mark your Service as DOWN.

Under the "Advanced" Tab you'll have to enable the "Client IP" checkbox and put X-Forwarded-For into the Textfield (like in the Screenshot).

The last step is to create the Virtual Server who will be load balancing your two (or more) Storefront Servers. Choose an IP and activate the previously created Storefront Services (svc_sf01 and svc_sf02 in my case). I would recommend creating a new DNS A Record pointing to your new Virtual IP (should be the same Alias you choose during the Storefront Configuration).

Under "Method and Persistence" choose Round Robin or Least Connection as LB Method. Under Persistence you should select SOURCEIP and set the Time-out to 20 Minutes (Default Timeout in Storefront).

Finally create a new SSL Cert pointing to the DNS Alias you created earlier and bind the SSL Cert to the Virtual Server. Done. 

Feel free to leave a comment if you have questions.

Configuring the new Storefront Monitor in NetScaler 10.1

With the new NetScaler 10.1 Release Citrix is shipping an "built-in" Storefront Monitor so you no longer have to use an https-evc Monitor (or something else) as Monitor like I described in my previous Blogpost.

UPDATE: The Problem I describe below seems to be fixed in the new NetScaler Release 10.1 Build 120.13 according to the Release Log.

Issue ID 0398327: Monitoring of StoreFront servers fails if they are part of a cluster and the StoreFront monitor is bound to the entire service group. The StoreFront monitor probe fails because individual members have different host names.

In this Example my Storefront Servers are named storefrontserver01 and storefrontserver02 and they are load balanced under the DNS Record lb-storefront.domain.local which is pointing to the Virtual IP Address (VIP) on my NetScaler responsible for load balancing the two Storefront Servers.

When configuring the new Storefront Monitor don't put the load balanced Storefront DNS Record in the Field "Host Name". Here is the Screenshot how NOT to do it:

You shouldn't use the load balanced DNS Record because as soon as your two (or more) Storefront Servers are down at the same time and therefor your Storefront Virtual Server is marked as Down too, the Storefront Monitor will never report the Storefront Services as up again even though the Storefront Servers might have recovered in the meantime. This is because the Storefront Monitor is checking the DNS Record pointing to the marked as "Down" Virtual Server load balancing your Storefront Servers which in turn is down because the Monitors are "Down" and the Monitors are "Down" because your Virtual Server is "Down"... the NetScaler is caught in a Loop.

To prevent this from happening you should instead create a separate Storefront Monitor for every Storefront Server you are going to be load balancing and put the FQDN of your Storefront Servers in the Field "Host Name". In my example this would be storefrontserver01.domain.local and in the second Monitor it would be storefrontserver02.domain.local.

You should then bind the new Monitors to their corresponding Service as shown in the Screenshot below:

If something is not understandable or my explanations are too weird feel free to let me know in the Comments :) 

NetScaler LoadBalancing Monitor for Storefront 1.2

I'm currently in the process of deploying a Storefront 1.2 Infrastructure alongside our already existing Webinterface 5.4 Infrastructure and needed an appropriate LoadBalancing Monitor on NetScaler for the Storefront Servers.

With the latest NetScaler 10.0.x Releases there is unfortunately no "Out of the Box" Storefront Monitor (like there is for the Webinterface), so I used the https-evc Monitor as a Base and the Default.htm Document located in the StoreWeb Folder to query for the String: "JavaScript is required".

If you have choosen another Name for your Store during the Storefront Setup you have to edit the Send String Path accordingly.

Feel free to comment and let me know if this was helpful or if there is an better method/way for monitoring the Storefront Servers.

Update: Citrix is now including a Storefront Monitor with the NetScaler 10.1 Release. I have a new Blogpost on how to correctly configure the new Storefront Monitor in the NetScaler 10.1 Release