Delayed application install prompts on iOS

Citrix XenMobile
1

We have come across a scenario with delayed application install prompts on iOS devices.

Environment
2 XenMobile nodes 10.4 (or above) in active/active configuration
Netscaler 11.1.51.21
Using SSL offload on the netscaler (terminating MDM ssl traffic) and re-encrypting the traffic to XenMobile on the backend.
Session Persistence is set to SSLSESSION for 443 & 8443 MDM loadbalancers

Scenario
The application installation prompts take a long time to appear after enrolling a device
When a user accesses store via SecureHub and installs an application the installation prompts takes a long time to appear (At times up to 30 minutes).

Cause
We’ve discovered that the issue was caused by the loadbalancer configuration on the netscaler

Scenario below:

  1. User accesses SecureHub Store and selects and application and hits install

  2. APNS message is created on node 1 and is submitted to apple APNS

    [UID=1,usr=user@domain,dev=111] | DEBUG | http-nio-10443-exec-10 | com.sparus.nps.apple.push.ApplePush | Sending command to target ApplePushTarget[os=iOS, device=111, user=user@domain, type=DEVICE] at xxx.xxx.xxx.xxx; type=DeviceInformation, identifier=com.zenprise.zdm.push.apple.DeviceNetworkInfos, description=Loads network information from the device. Assigned UUID: 5e1bce55-c3e1-4295-8bdb-f2d285c864e4

  3. APNS pings the device to check in to the MDM server

  4. Device connects to node 2

  5. Node 2 is not aware of the APNS message and is unable to service the device

    [UID=11,usr=user@domain,dev=111] | WARN | http-nio-10443-exec-7 | com.sparus.nps.apple.push.ApplePush | Received Acknowledgement for command 5e1bce55-c3e1-4295-8bdb-f2d285c864e4 from target ApplePushTarget[os=iOS, device=111, user=user@domain, type=DEVICE] at xxx.xxx.xxx.xxx, but don’t know about that command…

  6. This keeps on happening until device is load balanced to node 1, which is aware of the APNS request and is able to service the device

Resolution
Resolution was to change the persistence of both MDM load balancers (443 & 8443) to use SourceIP as the persistence method from SSLSESSION

2

A few things I’ve learned over my years of working with XenMobile:

  1. Citrix does not support SSL offload and re-encryption. They may say they do in their documentation, but ask anyone higher than their frontline support, and they’ll say that it does not work. I actually wasted about 3 months when implementing XM10 cause I was trying to use re-encryption. I finally gave up, put my XMS servers in the DMZ, and used SSL bridge.
  2. I’m pretty sure the NetScaler wizard sets up the MDM LB VIPs as SSL Session for persistence, but I’ve always been told to change it to SourceIP, as you’ve discovered.
3

We’ve implemented SSL offload with re-encryption for a few customers of sizeable deployments. I find that it does work but requires some tweaking to load balancing.
i.e. Changing the persistence to SourceIP and disabling SSL session re-use and tweaking the timeouts.

This is using XM 10.3 or above I’ve not tried with older versions.