AppV5 – Using Application Compatibility Toolkit to solve issues

/ / /


We have several applications that install folders in the root of the C: drive.  For most AppV5 implementations, this wouldn’t be an issue but we modify our PackageInstallationRoot folder so the token {AppVPackageDrive} turns into the drive letter specified in the PackageInstallationRoot registry key.  For us, that’s the D: drive.  This causes an issue because when you sequence an application to C: the application has an expectation for it to be there.  Ideally, AppV takes care of that by the use of the tokens, but this breaks down when applications are hardcoded.

So far, we’ve been able to work around these issues by using junctions or setting the PVAD to the folder as the PVAD acts literally on the value specified, essentially becoming a hard coded, custom, token.
But now we have an application with THREE folders that are installed on the root of the C: and the application is hard-coded to look for two of them.
Hello my nemesis’s
If we do nothing this is the error we get after attempting to login to the program:
Looking at the appvve we can see the D: is coming up with those folders.
So this isn’t going to work.  How can we make it work?
After publishing this application on the server, we install ACT on the Citrix server.
Launch the “Compatibility Administrator (32-bit)”
Right-click on “New Database” and select “Create New > Application Fix”
Enter the details and browse to the application and click “Next”
Click ‘Next’ on the Compatibility Mode screen
Select ‘CorrectFilePaths’ and then click ‘Parameters’
In command line, enter the path that should exist, then a semi-colon divider “;” and then the target path:
and click “OK”
Try a Test Run.
And the application launches without any error messages!
Click ‘Next’
Then Finish.
Click ‘Save’ then name your database and click “OK”:
Save your fix now:
At this stage you now need to put your SDB file somewhere accessible for when the package is published.  We put it on a fileshare.
Now, all we need to is install the fix.
Since we publish our application globally, I added the fix to the DeploymentConfig.xml:

And we are done! The application now works.

Read More

Citrix Receiver 4+ rants and “Your apps are not available at this time. Please try again in a few minutes or contact your help desk with this information”

/ / /

The environment I’m working in is a mix of XenApp 6.5, 5.0 and Presentation Server 4.5.  The 6.5 and 5.0 farm also have a nearly identical test farm.  We’ve been migrating the applications off the Presentation Server farms and are moving them to XenApp 6.5.  At this point in the migration we have around 5-10 applications left on 4.5 to move, with around 400-450 on the 6.5 farms and probably around 20-30 or so on the 5.0 farms.  These farms utilize a Citrix Webinterface 5.4.2 frontend for web interface and PNA.

We have standardized the environment on mostly Citrix Receiver 3.3 and some 3.4.  Time has marched and we’ve been tasked with getting Receiver 4+ working the Windows 7/8/10 rollout.  We were not able to do so with the earlier versions of Receiver 4 because things like sort icons into custom folders on the desktop and Start Menu.  This feature came in around 4.2.  In our environment memberships to applications are granted through group membership and Citrix PNA allowed the user to ‘roam’ from computer to computer only displaying the applications they have access to, as opposed to a bunch of applications they do not have access with the onus on them to pick and choose the correct applications.

So we started work on planning this migration and have started with the latest greatest (as of today) Citrix Receiver 4.3.  We have been able to come close to simulating all the features of the Enterprise editions of 3.3 and 3.4.  Namely:

Citrix Receiver automatically connects and populates a folder (MyApps) in the Start Menu and on the desktop.
No self-service.  Applications are automatically presented to you and defined by Group Membership.
Single sign-on.  Receiver will take your Windows logon credentials to use for authentication.

We do have some outstanding items.  We’ve set the client to have all applications as ‘MANDATORY’ which does populate the applications in the MyApps folders; but applications marked as ‘Create shortcut on the desktop’ in the applications properties in AppCenter are not created.

Anyways, onto the problem.  Now that we have Receiver 4.3 setup, SSON working, PNA working, we logged onto our system and watched the applications populate.


Really Slowly.

Eventually a dialog popped up.

Then another, and another, and another.

And these dialogs are completely custom!  They are NOT native Windows dialogs!

So if you have multiple ones of them, sometimes clicking the X (close button) or OK doesn’t work because the dialog appears ‘modal’ and you need to click the button on the ‘active’ window.  But you generally don’t know which one that is so you have to go through and select each dialog from the task bar and try clicking ok until you magically get lucky and select the one that has priority.  Then you do it all over again as the next primary window *may not be the one on top*.

1 minute 24 seconds to populate 470 applications

Just for giggles, how fast does Receiver 3.3 populate the same list?

8 seconds.  And all the icons show up.

Alright, so you’ve passed that point and are now looking at your applications.  But they are missing icons!

But not all applications are missing their icons…  Only some.

So let’s find out what’s consuming all this time and maybe, just maybe, we’ll solve our “Your apps are not available” error message.

First thing we need to do is enable Citrix Receiver Logging:

Next is to exit and restart receiver and logs will start to generate.  They are located here:

The most important log tends to be the ‘SelfService.txt’ log.  If you search that log for the “Your apps are not available” error message it pops up in locations like this:

So this dialog popped up for an application called ‘BMTServe’.  And what does BMTServe look like?

Generic icon!

But BMTServe was not the only application that encountered this dialog.  From my video it popped up numerous times.  Searching the SelfService.txt file for ‘Your apps’ and looking for the application it references points to an application with a blank icon 100% of the time.  Not every application that produces a blank icon causes this prompt, as we literally have ~250 applications with blank icons and the dialog pops up anywhere from 0 to 10 times.  Sometimes it pops up 2 times, sometimes none, sometimes 10 times.

So, why are these icons blank?

Citrix Receiver 4.3 seems to only prefer 32bit icons or icons of a particular size.  I haven’t confirmed what exactly yet, but I do know that 8bit 32×32 icons don’t seems to get ‘translated’.  The Citrix logs all but confirm this as well.

I confirmed with Citrix that icons are required to be 32bit and the order they are checked is 48×48, 32×32 then 16×16.

This is how Receiver processes icons that are formatted correctly:

The icons were processed instantly. 00:00:00. But if they are formatted in a way that Receiver decides it needs to ‘reformat’ them:

This call to get an icon took 11.6 seconds!!! If it doesn’t get the icon formatted in the way it wants, it appears the SelfService.exe setups a queue of icons that it needs and ‘re-requests’ them from the server. Could it be that Receiver is submitting too many queries? The error mentions to check the authmansvr.txt log file. This log file shows the following:

The error appears to start at “CWindowsReceiver::CallARGetConnectedVpnGateway” When this call is successful it returns:

So, I guess it’s possible that trying to re-pull the icon data is causing authmansvr.exe to crash…?  Another crazy thing is I was attempting to automate this process of terminating Receiver and relaunching it to see if I could get a gauge on the frequency of this occurrence and this is what I saw:

Ok, I thought, not so bad.  Just two messages the first couple launches?  It shouldn’t be too much of an issue…  But then I looked at my application folder:

Left is when I get all my apps (and usually the message box) the left is all those ‘successes’

It appears Receiver removed all applications producing that dialog box.  When I was terminating and relaunching receiver it was ONLY populating 195 applications as opposed to 493 it was supposed to. No wonder I wasn’t getting any messages!  On a hunch, I looked at each of the 195 applications it kept and they all had good icons.  I then took a random sampling of about 30 of the 300 or so applications that it did not keep and none of them had proper icons, all blank.  So another bullet towards icons causing my issue.

Read More

Force Internet Explorer 10 or 11 to always use 64bit version

/ / /

I was working on an issue where a user was always prompted to ‘Install’ the Citrix ICA client.  No matter how many times they downloaded and installed the client it continuously prompted them to install it again from the Web Interface:

I checked the add-on’s and saw the following:

No Citrix plugins in site.  I then checked Task Manager to confirm the IE type (32bit vs 64bit) and this is what I saw:

Without the *32, Internet Explorer is running in 64bit mode.  Currently, Citrix does not provide a 64bit plugin to IE so it won’t run and it won’t be detected.  I then exited IE and browsed to the Internet Explorer folder (C:program files (x86)Internet Exploreriexplore.exe) and attempted to launch iexplore.exe from there.  Still came up as 64bit.  So now this got interesting…  Microsoft does not allow or provide a way to force a 64bit default for IE on Windows 7.

So how is this happening?

It turns out there is a registry key you can set that will force IE to ALWAYS be 64bit:

HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\TabProcGrowth (or HKLM)

If the REG_DWORD is 0x0 it will always force IE to be 64bit.  Deleting or changing this value will default IE to 32bit.  So this key *could* be used to force IE to be 64bit.  There is a potential issue to be aware of, this will force IE to use the same process as the launcher for tabs, as opposed to spanning new processes.  Whether that increases/decreases stability would be something you’d have to test.

Read More

AppV5 – Integrating Certificates into your AppV package

/ / /

These are the steps I’ve found to sequence root certificates into your AppV5 application.

Where do you get certmgr.exe from?

The visual studio downloads apparently contains this tool.

Once you’ve started your sequencer and run the command it will add the certificate to these two places:

And that’s how you add certificates to a sequenced package.

Read More

AppV5 – Sequencing Oracle 11g R2

/ / /

Apparently this is a fun topic.  How do you sequence Oracle 11G R2 on AppV5?

I believe I have an answer.  In my attempts to sequence Oracle 11G on AppV5 I came across a few issues and have come up with solutions that work for various applications that rely on this tool.

The first issue:
Oracle 11G only allows paths without spaces and special characters.

On Windows systems, if the path to your Java installation includes a space character, you must provide the path in DOS 8.3 format, as shown in the previous example.

This *maybe* fixed now, but I experienced issues with trying to install Oracle 11g to the 8.3 folder structure to place it under “Program Files” or “Program Files (x86)”.  The sequenced application would be broken.  This was a known/reported issue with AppV 5SP2 HF4 that was marked as ‘fixed’ by Microsoft for SP3+.  I have not had the ability to confirm that and will continue this post with what I know works…  I also believe that when expanded out it uses the full path with spaces as opposed to the 8.3 path.

Second Issue:
Installing Oracle 11G to the default ‘recommended’ directory will fail if you move your PackageInstallationRoot to a different drive.

This is because the second folder (apps – in this example) is not tokenized.  Forcing AppV5 to utilize the token “appvPackgeDrive” which can expand out differently then you expect, breaking the application.

Third Issue:
Cannot install to PVAD.

The reason I chose to NOT utilize PVAD is if you do then Oracle cannot be used in connection groups.

So how do you resolve all these issues and sequence Oracle 11G R2?

The direction I went was to ensure the directory I sequenced the installer to was a tokenized directory.  It also needed to a directory that, when expanded in the virtualized environment, does not contain any spaces or special characters.

The list of directories AppV5 tokenize’s can be found in the AppV 5.0 Sequencing Guide.

I’ll list them here:

Known Folder Token
Known Folder Path
Administrative Tools
C:UsersAppDataRoamingMicrosoftWindowsStart MenuProgramsAdministrative Tools
Application Shortcuts
C:UsersAppDataLocalMicrosoftWindowsApplication Shortcuts
C:UsersAppDataLocalMicrosoftWindowsTemporary Internet Files
CD Burning
Common Administrative Tools
C:ProgramDataMicrosoftWindowsStart MenuProgramsAdministrative Tools
Common AppData
Common Desktop
Common Documents
Common Programs
C:ProgramDataMicrosoftWindowsStart MenuPrograms
Common Start Menu
C:ProgramDataMicrosoftWindowsStart Menu
Common Startup
C:ProgramDataMicrosoftWindowsStart MenuProgramsStartup
Common Templates
Device Metadata Store
C:UsersAppDataRoamingMicrosoftInternet ExplorerQuick LaunchUser PinnedImplicitAppShortcuts
Local AppData
My Music
My Pictures
My Video
C:UsersAppDataRoamingMicrosoftWindowsNetwork Shortcuts
Podcast Library
C:UsersAppDataRoamingMicrosoftWindowsPrinter Shortcuts
C:Program Files
C:Program FilesCommon Files
C:Program FilesCommon Files
C:Program Files (x86)Common Files
C:Program Files
C:Program Files (x86)
C:UsersAppDataRoamingMicrosoftWindowsStart MenuPrograms
Quick Launch
C:UsersAppDataRoamingMicrosoftInternet ExplorerQuick Launch
Roamed Tile Images
Roaming Tiles
C:UsersSaved Games
Start Menu
C:UsersAppDataRoamingMicrosoftWindowsStart Menu
C:UsersAppDataRoamingMicrosoftWindowsStart MenuProgramsStartup
User Pinned
C:UsersAppDataRoamingMicrosoftInternet ExplorerQuick LaunchUser Pinned
Custom Token
Custom Token Expansion
C:UsersAll Users


There are multiple directories we can choose.  I opted to use “Common AppData”.  That means I will install the Oracle client here: “C:ProgramData”.  It does not contain a space, is tokenized, and when expanded will remain on the C: drive.  I created a ‘response’ file for the Oracle install.
I called the script through this command:
And that’s it!
Read More

Symantec Endpoint Protection (SEP) 12 virus definition download and installation script

/ / /

A quick and dirty script to download and install the latest SEP12 virus definitions from Symantec’s FTP site.  We use this script to force the latest updates when we update our Citrix PVS vDisk.

This script output looks like so:

This script also requires the FTP for powershell module.  Download the entire script and dependency here.

Read More

Troubleshooting Citrix Desktop Service – “The Citrix Desktop Service is starting.” and then nothing.

/ /
in Blog

I’m troubleshooting an issue with the Citrix Desktop Service on my home lab.  I have a Citrix XenApp/XenDesktop 7.6 installation and I have setup a Server 2012 R2 box with the “/servervdi”.  Upon reboot, I see the Registration State as Unregistered.

Restarting the Citrix Desktop Service and checking the ‘Application’ Log for ‘Citrix Desktop Service’ yields only event ID 1028 – “The Citrix Desktop Service is starting.”

However, when I ‘Stop’ the Citrix Desktop Service the Application event log gives up a few more details:

Event ID 1003 – Citrix Desktop Service
The Citrix Desktop Service failed to initialize communication services required for interaction between this machine and delivery controllers. 

If the problem persists please perform a ‘repair’ install action or reinstall the Citrix Virtual Desktop Agent. Refer to Citrix Knowledge Base article CTX119736  for  further information. 

Error details: 
Failed to start WCF services. Exception ‘Object reference not set to an instance of an object.’ of type ‘System.NullReferenceException’

Unfortunately, this CTX article gives little to no details on event 1003 and is more of a shotgun attempt at solving issues as opposed to a nice, precise, surgical solution.

From the EventID 1003 we can see the Citrix Desktop Service is trying to reference WCF services and it has failed.

Citrix offers a tool called XDPing to try and diagnose issues, I ran it and had it return the following:

Googling the WCF errors with HTTP/1.1 and Error 503 results in lots of information on reconfiguring your IIS.  I’m not convinced this is the issue so I soldiered on…

When I procmon on ‘BrokerAgent.exe’ I see a few curious entires that maybe associated with ‘System.NullReferenceException‘ (aka, not found) and those are some permissions stating that access is denied to some registry keys and/or some ‘Name Not Found’ on some CLSID items.

During this capture I can see a ‘BrokerAgent.config’ file referenced.  (“C:\Program Files\Citrix\Virtual Desktop Agent\BrokerAgent.exe.config” ) Diving into it reveals some additional logging we can enable:

Verbose logging disabled by default

If we ‘enable’ the debug portions and create the C:cdsLogs folder we can get some more information on what is going wrong.

Logging Enabled

Stopping the ‘Citrix Desktop Service’, editing the .config file, creating the C:\cdslogs folder and starting the service yielded additional information.

We have a ‘COM exception’.  The nice thing about this log file is we can compare the time stamps to the procmon logs and determine what was happening when this failed.

It appears we are missing SCService64.exe from our Citrix installation.  What is ‘SCService64.exe’?  The registry tells me it’s the ‘IStackControl Type Library’.  Which matches up with the error ‘ConnectToStackControlCOMServer’.

So, it appears we need to install SCService64.exe.  I do not know why or how it went missing but I suspect we can copy it over or extract from the Citrix source files if needed.

To extract the SCService64.exe source file, we can create an administrative installation of the “TS” VDA:

msiexec /a “\x79-serversoftwareCitrixXenApp_and_XenDesktop7_6x64Virtual Desktop ComponentsTSIcaTS_x64.msi”
This makes a folder on the root of the C: called “Citrix” which installs all the files there:

I installed the Citrix VDA “WS” with /servervdi for some testing, but prior to that I had the regular “TS” VDA installed.  Perhaps some combination of my installation/uninstallation caused my issues.

Anyways, copying the SCService64.exe to the C:\Program Files (x86)\Citrix\System32 folder and restarting the ‘Citrix Desktop Service’ resulted in…

Registered, Hurray!

And what about our log file?

Previously it died around ‘Setting up ALL LaunchManager WCF service’; this time we see:

Hurray!  It continues and operates without issue.

Be sure to re-disabled the logging on the BrokerAgent.config file and restart the service because I’ve found this BA_1.log can get to become a huge file.

Running XDPing this time results in [OK] across where the errors once were.

Read More


/ / /

I rebooted my computer to this lovely Blue Screen Of Death (BSOD) message:

Attempting to reboot into Safe Mode also resulted in the same message.  I was able to boot into ‘Recovery Mode’ which is a ‘Windows PE’ mode that runs a stripped down version of Windows in RAM.  From here I enabled the network ‘Kernel Debugging’ by configuring some parameters in the BCD file.
The two parameters I set where:

bcdedit /store C:\boot\bcd /debug on
bcdedit /store C:\boot\bcd /dbgsettings net hostip: port:49152

I needed to set the “/store” parameter to ensure I was manipulating my non-booting BCD file, and not the BCD file that Windows Recovery boots from.  Write down the key or save it someplace, you’ll need it on the ‘host’ computer (see in the above screenshot).

Once here I downloaded and installed ‘WinDBG.exe‘.  Open windbg.exe and choose “File > Kernel Debug“.  On the ‘NET’ tab, enter your ‘Port’ number and ‘Key’ (everything to right of the equal sign) and click ‘OK’.
Even though I ‘enabled’ debug in my BCD file, I found I still needed to tap the ‘F8’ key while booting and select ‘Debugging Mode’.  Once selected, my windbg.exe on my host computer sprang to life!

It turns out you need to enable symbols or else you get an incomplete picture.  After enabling symbols and running !analyze -v I got the following:

ctxusbm.  This is a Citrix driver for their Receiver client that passes through USB to a Citrix session. I had updated Receiver to last month and I probably hadn’t rebooted my computer until Windows Update made me.  So that’s probably why I’m experiencing this issue now.  To fix this issue, I rebooted into the Windows Recovery mode and deleted all instances of ‘ctxusbm’ from the SYSTEM hive.  Specifically, I deleted these locations:

Upon the next reboot, my computer came back cleanly and operates without any issues.  I am going to keep this module removed until the next version of Receiver is released, hopefully, I won’t have any more issues.  Issues with ctxusbm seem relatively prevalent with Citrix.

Read More

Performance differences in Citrix HDX Thinwire Encoders

/ / /

Per my previous post, changing the Citrix HDX Thinwire Encoder on the fly, we can test the performance differences in the different encoder’s Citrix provides.  I have done so by running through a demo of the Uniengine Heaven benchmark.  The demo is exactly 4 minutes and 20 seconds long.  I did a perfmon trace of the CPU %, total bytes sent in MBits/sec and the Thinwire Output in MBit/sec.

Time for some results!

Compatibility Mode (Encoder 0x0)

DeepCompressionV2Encoder (Encoder 0x1)

DeepCompressionEncoder (Encoder 0x2)
(Rollover the mouse on the next images to compare graphs)

CompatibilityMode vs DeepCompressionV2Encoder

CompatibilityMode vs DeepCompressionEncoder

DeepCompressionV2Encoder vs DeepCompressionEncoder

The cumulative totals should help us get an understanding of the differences between the encoders:

   CPU Total ThinWire Total Network Total (Mbytes)
DeepCompressionEncoder 5531.00 3693.28 540.51
DeepCompressionV2Encoder 5621.67 3684.75 539.74
CompatibilityMode 4197.54 3690.58 553.21
   CPU Total ThinWire Total Network Total (Mbytes)
DeepCompressionEncoder 98.4% 100.0% 97.7%
DeepCompressionV2Encoder 100.0% 99.8% 97.6%
CompatibilityMode 74.7% 99.9% 100.0%
Interestingly, CompatibilityMode uses 25% less CPU then either DeepCompression Encoder.  From what I see though the frames per second appears less for CompatibilityMode then the other two.
Read More