Citrix Storefront – Performance Testing and Tuning – Part 4 – PerfMon Counters, PNA Logon

I’ve added 6 farms to the storefront server to examine the load “in a real world” environment as Storefront does do some magic with concurrent enumeration.  What I love that Citrix has done, is actually time the transactions AND gives you the ‘rate’ the transactions are occurring at.  This will make it much easier to baseline what your load is vs how I had to do it to measure the ‘rates’ for Web Interface.

---

$stage = “Client Configuration”
$store + “config.xml”

So for a web browser based connection, I ‘connected’ to the site and saw nothing in the perf counters.  Storefront does very little work for the initial connection.

---

$store + “enum.aspx”

POST with:

Response:
I do  not have any PRELAUNCH applications.

However, we do get perfmon counter data with this request:

Citrix Resource Subscription\Dispose Calls / second : 1
Citrix Delegated Explicit Authentication\Authenticate Calls / second : 1
Citrix Delegated Explicit Authentication\Authenticate Average Time (Microseconds) : 95,392
Citrix Resources Common\Find all resources  Average Time (Microseconds)  : 57,277
Citrix Resources Common\Find all resources  Calls / second : 1
Citrix Xml Service Communication(10.10.10.13)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(xenapp5.bottheory.local)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.13)\Network Traffic Average Time (Microseconds) : 36,383
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Average Time (Microseconds) : 47,260
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Average Time (Microseconds) : 60,759
Citrix Xml Service Communication(xenapp5.bottheory.local)\Network Traffic Average Time (Microseconds) : 30,730
Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Average Time (Microseconds) : 44,155

---

$store + “enum.aspx”

POST with:

Response (truncated):
And the Perfmon counter data:

Citrix Resource Subscription\Dispose Average Time (Microseconds) : 1
Citrix Resource Subscription\Dispose Calls / second : 1
Citrix Delegated Explicit Authentication\Authenticate Calls / second : 1
Citrix Delegated Explicit Authentication\Authenticate Average Time (Microseconds) : 88,473
Citrix Resources Common\Find all resources  Average Time (Microseconds)  : 6,314,583
Citrix Resources Common\Find all resources  Calls / second : 1
Citrix Xml Service Communication(10.10.10.13)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(xenapp5.bottheory.local)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.13)\Network Traffic Average Time (Microseconds) : 42,697
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Average Time (Microseconds) : 37,653
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Average Time (Microseconds) : 64,622
Citrix Xml Service Communication(xenapp5.bottheory.local)\Network Traffic Average Time (Microseconds) : 125,629
Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Average Time (Microseconds) : 191,408

---

$store + “enum.aspx”

POST data:

Response:

And the Perfmon counter data:

Citrix Resource Subscription\Dispose Average Time (Microseconds) : 0.5
Citrix Resource Subscription\Dispose Calls / second : 2
Citrix Delegated Explicit Authentication\Authenticate Calls / second : 2
Citrix Delegated Explicit Authentication\Authenticate Average Time (Microseconds) : 140,062
Citrix Resources Common\Find all resources  Average Time (Microseconds)  : 219,619
Citrix Resources Common\Find all resources  Calls / second : 1
Citrix Xml Service Communication(10.10.10.13)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(xenapp5.bottheory.local)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Calls / second : 1
Citrix Xml Service Communication(10.10.10.13)\Network Traffic Average Time (Microseconds) : 44,896
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Average Time (Microseconds) : 40,508
Citrix Xml Service Communication(10.10.10.14)\Network Traffic Average Time (Microseconds) : 36,124
Citrix Xml Service Communication(xenapp5.bottheory.local)\Network Traffic Average Time (Microseconds) : 127,686
Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Average Time (Microseconds) : 165,869

I feel like I’m detecting a theme here…

---

$store + “launch.aspx”

This launch only hits the specific XML service from Storefront now:

Citrix Resource Subscription\Dispose Average Time (Microseconds) : 1
Citrix Resource Subscription\Dispose Calls / second : 1
Citrix Delegated Explicit Authentication\Authenticate Calls / second : 1
Citrix Delegated Explicit Authentication\Authenticate Average Time (Microseconds) : 88,206

Citrix Resources Common\ICA Launch Average Time (Microseconds) : 260,238

Citrix Resources Common\ICA Launch Calls / second  : 1
Citrix Resources Common\Find resource by id Average Time (Microseconds) : 217,447

Citrix Resources Common\Find resource by id Calls / second  : 1

Citrix Xml Service Communication(10.10.10.13)\Network Traffic Calls / second : 4
Citrix Xml Service Communication(xenapp65t.bottheory.local)\Network Traffic Average Time (Microseconds) : 142,010

---

Final results:

When PNA is used, it makes POST commands to the Storefront web services that then forwards those requests to the XML brokers, compiles the results and sends them back.  It’s pretty obvious that the performance of the brokers is imperative for PNA to operate efficiently.

I kind of wish Storefront would break down the XML perfmon requests into their sub categories.  By averaging out all requests to the XML brokers, it would make it difficult to pinpoint which sequence of events may be causing your pain point.  The values will allow to target a specific XML broker for further analysis, but some it could have been further helpful if you knew specifically that the “xenapp5.bottheory.local” broker was 10x longer to enumerate your full application icon list.  An average could ‘hide’ that bottleneck.  My account has over 400 published applications and gets 100 different individual icon requests after the ‘icon-all’ request.  If this counter truly measures the average it would obscure the high ‘icon-all’ time as 100 instances of would outweigh the single metric.

However, this is still fantastic information.  I have written a couple scripts for ‘querying’ each stage to measure the response time of Storefront/Web Interface and Citrix has effectively added the ability to just pull that information from real users using these counters!  It’s now easier and more important than ever to monitor your XML brokers from Storefront to ensure you are running at optimal efficiency.

To put these counters in context, the ‘Network Traffic Calls / second’ and ‘Network Traffic Calls / Total’ will tell you which of your XML brokers are getting hit the most.  The Total will give you that big picture, “this guy gets hit alot” and the per second counter will give you that real time data.

The “Network Average Time” will give you the data on how long or how hard that particular broker is working.  The higher the number the more work it’s doing or the more you should focus on optimizing it.  I don’t have a guide for what these numbers should be, as different users will have different expectations.  But I have shown what the values we get and I would argue we get great performance so I would try to keep your numbers below mine 🙂