Citrix Storefront – Performance Testing and Tuning – Part 3 – PerfMon Counters, browser logon

Citrix Storefront – Performance Testing and Tuning – Part 3 – PerfMon Counters, browser logon

2017-06-01
/ /
in Blog
/

In looking at the performance of the Citrix Storefront Server, one of the thing I want is to understand what Storefront is doing at each stage of a session life cycle.  There are two session-types that I’m curious about, a web browser based connection and a PNA connection.  I’m going to examine a web browser based connection first.

Using my powershell script to simulate a user connection I put ‘pauses’ between each stage.  I then setup Perfmon to capture counters from the following objects:

I’ve added 6 farms to the storefront server to examine the load “in a real world” environment as Storefront does do some magic with concurrent enumeration.  What I love that Citrix has done, is actually time the transactions AND gives you the ‘rate’ the transactions are occurring at.  This will make it much easier to baseline what your load is vs how I had to do it to measure the ‘rates’ for Web Interface.


$stage = “Initial Connection”
$store = “http://storefront.bottheory.local/Citrix/StoreWeb/”

So for a web browser based connection, I ‘connected’ to the site and saw nothing.  Storefront does very little work for the initial connection.


$stage = “Client Configuration”
$store + “Home/Configuration”

Result:

This request is to get the client configuration.  Essentially, it pulls down an xml file from the store containing important links to various locations in Storefront. Again, no perf counter seems to occur and it doesn’t seem to generate any load.


$stage = “Get Authentication Methods”
$store + “Resources/List”

 

So this command starts to get a little exciting.  It’s a bit of a misnomer to call it “Resources/List” as it doesn’t actually return resources, yet.  It returns the challenge to logon.  We get our first Peformance Counter hit.

The four counters that register are

“Citrix Receiver for Web\List resources Calls / second”
“Citrix Receiver for Web\List resources Average Time (microseconds)”
“Citrix Delivery Services Web Application(dazzleresources:list:get:_citrix_store)\Controller Action Calls / second”
“Citrix Delivery Services Web Application(dazzleresources:list:get:_citrix_store)\Controller Action Average Time (Microseconds)”
Results:
List resources Calls / second: 1
List resources Average Time (microseconds): 3,714
(dazzleresources:list:get:_citrix_store)\Controller Action Average Time (Microseconds): 607
(dazzleresources:list:get:_citrix_store)\Controller Action Calls / second: 1


$stage = “Get Auth Methods”
$store + “Authentication/GetAuthMethods”

Response:

Another exciting call.  This call hits these four performance counters:

“Citrix Delivery Services Web Application(authservice:tokenservices:post:_citrix_authentication)\Controller Action Calls / second”
“Citrix Delivery Services Web Application(authservice:tokenservices:post:_citrix_authentication)\Controller Action Average Time (Microseconds)”
“Citrix Delivery Services Web Application(protocols:choices:post:_citrix_authentication)\Controller Action Calls / second”
“Citrix Delivery Services Web Application(protocols:choices:post:_citrix_authentication)\Controller Action Average Time (Microseconds)”

Results:
(protocols:choices:post:_citrix_authentication)\Controller Action Calls / second: 1
(protocols:choices:post:_citrix_authentication)\Controller Action Average Time (Microseconds) : 488
(authservice:tokenservices:post:_citrix_authentication)\Controller Action Calls / second: 1
(authservice:tokenservices:post:_citrix_authentication)\Controller Action Average Time (Microseconds): 138

The Controller Action Calls /second was “1”, obviously I’m doing just a single test.  For the action average time I got 711 microseconds.  Extremely fast.


$stage = “Domain Pass-Through and Smart Card Authentication”
$store + “DomainPassthroughAuth/Login”

Response:

Two more perf counters hit (rate + time):

Citrix Delivery Services Web Application(citrixfederationauthentication:authenticate:post:_citrix_authentication)\Controller Action Calls / second
Citrix Delivery Services Web Application(citrixfederationauthentication:authenticate:post:_citrix_authentication)\Controller Action Average Time (Microseconds)

Results:
(citrixfederationauthentication:authenticate:post:_citrix_authentication)\Controller Action Calls / second: 1
(citrixfederationauthentication:authenticate:post:_citrix_authentication)\Controller Action Average Time (Microseconds): 4,850


$stage = “Resource Enumeration”
$store + “Resources/List”

Response:

This call returns a list of all the resources for which you have access.  The counters important to this call are:
“Citrix Receiver for Web\List resources Average Time (microseconds)”
“Citrix Dazzle Resources Controller\List Whole Body Calls / second”
“Citrix Dazzle Resources Controller\List Whole Body Average Time (Microseconds)”
“Citrix Delivery Services Web Application(authservice:tokenservices:post:_citrix_authentication)\Controller Action Calls / second”
“Citrix Delivery Services Web Application(dazzleresources:list:get:_citrix_store)\Controller Action Calls / second”
“Citrix Delivery Services Web Application(authservice:tokenservices:post:_citrix_authentication)\Controller Action Average Time (Microseconds)”
“Citrix Delivery Services Web Application(dazzleresources:list:get:_citrix_store)\Controller Action Average Time (Microseconds)”
“Citrix Receiver for Web\List resources Calls / second”
“Citrix Xml Service Communication(10.10.10.11)\Network Traffic Calls / second”
“Citrix Xml Service Communication(10.10.10.11)\Network Traffic Average Time (Microseconds)”
“Citrix Xml Service Communication(10.10.10.12)\Network Traffic Calls / second”
“Citrix Xml Service Communication(10.10.10.12)\Network Traffic Average Time (Microseconds)”
“Citrix Xml Service Communication(10.10.10.13)\Network Traffic Calls / second”
“Citrix Xml Service Communication(10.10.10.13)\Network Traffic Average Time (Microseconds)”
“Citrix Xml Service Communication(10.10.10.14)\Network Traffic Calls / second”
“Citrix Xml Service Communication(10.10.10.14)\Network Traffic Average Time (Microseconds)”
“Citrix Xml Service Communication(xenapp5.bottheory.local)\Network Traffic Calls / second”
“Citrix Xml Service Communication(xenapp5.bottheory.local)\Network Traffic Average Time (Microseconds)”
“Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Calls / second”
“Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Average Time (Microseconds)”

We have lots of exciting things happening here.  We’re reaching back and querying the XML brokers, some kind of authentication occurs, etc.

The results of these counters were:

Citrix Receiver for Web\List resources Average Time (microseconds) : 249,172
Citrix Dazzle Resources Controller\List Whole Body Calls / second : 1
Citrix Dazzle Resources Controller\List Whole Body Average Time (Microseconds) : 182,206
(authservice:tokenservices:post:_citrix_authentication)\Controller Action Calls / second : 1
(dazzleresources:list:get:_citrix_store)\Controller Action Calls / second : 2
(authservice:tokenservices:post:_citrix_authentication)\Controller Action Average Time (Microseconds) : 4,549
(dazzleresources:list:get:_citrix_store)\Controller Action Average Time (Microseconds) : 8,391
Citrix Receiver for Web\List resources Calls / second : 1
Citrix Xml Service Communication(10.10.10.11)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.11)\Network Traffic Average Time (Microseconds) : 0
Citrix Xml Service Communication(10.10.10.12)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.12)\Network Traffic Average Time (Microseconds) : 36,820
Citrix Xml Service Communication(10.10.10.13)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.13)\Network Traffic Average Time (Microseconds) : 37,622
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Average Time (Microseconds) : 43,121
Citrix Xml Service Communication(xenapp5.bottheory.local)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(xenapp5.bottheory.local)\Network Traffic Average Time (Microseconds) : 95,575
Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Average Time (Microseconds) : 149,888

With these counters we can see how long it took the brokers to respond.  My broker that has a few hundred applications took the longest, followed by a broker of an older farm with dozens of apps, then three where only a single app each is published from them.  I got a 0.0 for enumeration for one service.  I’m not sure what that’s about (maybe it exceeded enumeration timeout and is now excluded?).  Total time to list resources was 0.249 seconds.


$stage = “Get User Name”
$store + “Authentication/GetUserName”

Response:

This call simply returns your display name (or logon name if no display name is specified).

Two counters were hit:
“Citrix Delivery Services Web Application(authservice:tokenservices:post:_citrix_authentication)\Controller Action  Calls / second”
“Citrix Delivery Services Web Application(tokenvalidation:tokenvalidation:get:_citrix_authentication)\Controller Action Average Time (Microseconds)”
“Citrix Delivery Services Web Application(authservice:tokenservices:post:_citrix_authentication)\Controller Action Average Time (Microseconds)”

The results of these counters were:
(tokenvalidation:tokenvalidation:get:_citrix_authentication)\Controller Action  Calls / second : 2
(authservice:tokenservices:post:_citrix_authentication)\Controller Action  Calls / second : 1
(tokenvalidation:tokenvalidation:get:_citrix_authentication)\Controller Action Average Time (Microseconds) : 2,200
(authservice:tokenservices:post:_citrix_authentication)\Controller Action Average Time (Microseconds) : 4,551


$stage = “AllowSelfServiceAccountManagement”
$store + “ExplicitAuth/AllowSelfServiceAccountManagement”

Response:

This call appears to return True or False.

“Citrix Delivery Services Web Application(selfserviceaccountmanagement:allowselfserviceaccountmanagement:get:_citrix_authentication)\Controller Action  Calls / second”
“Citrix Delivery Services Web Application(selfserviceaccountmanagement:allowselfserviceaccountmanagement:get:_citrix_authentication)\Controller Action Average Time (Microseconds)”
“Citrix Delivery Services Web Application(authservice:tokenservices:post:_citrix_authentication)\Controller Action Average Time (Microseconds)”
“Citrix Delivery Services Web Application(authservice:tokenservices:post:_citrix_authentication)\Controller Action  Calls / second”

The results of these counters were:
(selfserviceaccountmanagement:allowselfserviceaccountmanagement:get:_citrix_authentication)\Controller Action  Calls / second : 2
(selfserviceaccountmanagement:allowselfserviceaccountmanagement:get:_citrix_authentication)\Controller Action Average Time (Microseconds) : 2,247
(authservice:tokenservices:post:_citrix_authentication)\Controller Action  Calls / second : 1
(authservice:tokenservices:post:_citrix_authentication)\Controller Action Average Time (Microseconds) : 6,014


 

$stage = “Download all the icons”
$store + $iconurl

Response:

<downloads all icon files>

This call hits the following counters:

“Citrix Receiver for Web\Get icon Average Time (Microseconds)”
“Citrix Receiver for Web\Get icon Calls / second”

The results of these counters were:

Citrix Receiver for Web\Get icon Average Time (Microseconds) : 285.351
Citrix Receiver for Web\Get icon Calls / second : 152.259


$stage = “ICA Launch.GetLaunchStatus”
$store + $appToLaunch.launchstatusurl

Response:

This call hits the following counters:

Citrix Receiver for Web\Get launch status Calls / second
Citrix Receiver for Web\Get launch status Average Time (Microseconds)
Citrix Delivery Services Web Application(dazzleresources:launchica:post:_citrix_store)\Controller Action Calls / second
Citrix Delivery Services Web Application(dazzleresources:launchica:post:_citrix_store)\Controller Action Average Time (Microseconds)
Citrix Xml Service Communication(10.10.10.11)\Network Traffic Calls / second
Citrix Xml Service Communication(10.10.10.11)\Network Traffic Average Time (Microseconds)

The results of these counters were:

Citrix Receiver for Web\Get launch status Calls / second : 1
Citrix Receiver for Web\Get launch status Average Time (Microseconds) : 152,721
Citrix Delivery Services Web Application(dazzleresources:launchica:post:_citrix_store)\Controller Action Calls / second : 1
Citrix Delivery Services Web Application(dazzleresources:launchica:post:_citrix_store)\Controller Action Average Time (Microseconds) : 5,091
Citrix Xml Service Communication(10.10.10.11)\Network Traffic Calls / second:  2
Citrix Xml Service Communication(10.10.10.11)\Network Traffic Average Time (Microseconds) : 65,709

Storefront will only fetch the status for the application from the broker that the application belongs.  If I run this query multiple times, hitting different brokers, the XML service communication populates more information.  The information here can be useful for you to find which broker is getting hit the most by looking at the “Network Traffic Calls / second” and you can find whether it’s overloaded by the “Network Traffic Average Time” being a number that grows as your load grows.  As with everything, it’s probably best to get some “base” numbers when load is low and then again when load is high to get your range.


$stage = “ICA Launch.LaunchIca”
$store + $appToLaunch.launchurl

Response:

We get an ICA file.  The performance counters that are important:

Citrix Delivery Services Web Application(dazzleresources:launchica:post:_citrix_store)\Controller Action Calls / second : 1
Citrix Delivery Services Web Application(dazzleresources:launchica:post:_citrix_store)\Controller Action Average Time (Microseconds) : 5,453
Citrix Receiver for Web\Get Ica file Calls / second : 1
Citrix Receiver for Web\Get Ica file Average Time (Microseconds) : 1,255,237
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Calls / second : 2
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Average Time (Microseconds) : 530,153

 


Final results:

Taking all the operations and the time it took, I sorted the list according to duration.

The following operations list the total duration of that specific task:

“Get Ica file”
“List resources”
“List Whole Body”
“Get launch status”
Excluding them shows us the sub-tasks that actually consumed the majority of the time spent:

The XML communications is what consumes the majority of the processing time.  The items starting in ‘brackets ()’ are the Citrix Storefront processing time that it spent.  Being in the single “thousands” of microseconds is super fast.  This essentially shows us that it takes next to no time at all for Storefront to execute the tasks that it needs.


Analysis:

Storefront has some great new performance counters that can help you determine your pain points.  Access to Citrix is usually talked about in terms of ‘rate’ so the per-second and average time counters are excellent in helping determine what is the component taking so long to do whatever task.  The breaking down of how long individual XML brokers take is a HUGE plus as you should be able to accurately gauge which Citrix FARM is now causing slowness in logon, enumeration or launching for your users.

Post a Comment

Your email address will not be published. Required fields are marked *

*