Citrix Provisioning Services Target Device Upgrade Woes

Citrix Provisioning Services Target Device Upgrade Woes

2017-06-27
/ /
in Blog
/

We are running Citrix PVS 7.7 and we are attempting to upgrade to Citrix PVS 7.13 for the vDisks.  Starting with PVS 7.7 you are supposed to be able to do an in-place upgrade of the tools.  Our experience with this has been less than positive, with a success rate of ~50%.  This post is the issues we encountered and how we solved it for the other 50%.


Issue #1:

Error 2203. Database: C:\Windows\Installer\111cbf.ipi.  Cannot open database file.  System error -2147287037.

When I look for that ipi file I find it is not present.  Click ‘OK’ results in nothing happening.  Attempting to install PVS 7.13 as an in place upgrade results in Error 1923.Service Citrix PVS Device Service (BNDevice) could not be installed.  Verify that you have sufficient privileges to install system services.

In an attempt to move this along I stopped the BNDevice service and manually specified a deletion (“sc delete bndevice”).  This allowed the install to proceed without issue.


Issue #2:

Uninstalling PVS 7.13 Target Device Software fails with ‘Product key not found’.

Doing a Procmon trace I discovered that PVS 7.13 requires a registry key and value to uninstall cleanly:

For whatever reason, this key and value do not get created and do not exist on an upgrade or clean install but are required to uninstall.  Manually adding the key allows the PVS Target Device software to uninstall cleanly.


Issue #3:

When using the “Provisioning Services Imaging Wizard” an error occurs “The vDisk has no volumes to copy to.  Pick another vDisk.”

This error occurs because our uninstall did not remove the Citrix Storage controller driver, and when we did an install it added another Citrix storage controller.  This new controller adds an additional disk but since the disk is in use from the original connection it comes up as “offline”.

If you attempt to bring the Disk Online you get an “Incorrect function.” error:


Device Manager shows multiple devices:

There should only exist 1 of these Citrix devices.

To resolve this issue, simply select right-click one of the “Citrix Virual Hard Disk Adapter” and select “Uninstall”.

This may cause you to reboot.  On reboot you should only have a single device listed, and the offline disk should be gone from Disk Management.  At this point you can run the Imaging Wizard without issue.


Issue #4

“Citrix Provisioning Services Target Device x64” software is not displayed in ‘Programs and Features’, but it is obviously installed because the files exist, the registry keys exist, and the program is running.

It should be listed just under “Citrix Profile Management”

If you attempt to run the removal or modifyPath registry command

You get an error “Windows installer has stopped working”.  If you try and continue you MAY get stopped at “Error 1723. There is a problem with this Windows Installer package.  A DLL required for this install to complete could not be run.  Contact your support personnel or package vendor.  Action BNNS_Uninstall.FA231996_1469_4817_B7F3_61A648A18C07, entry: fnBNNS_Uninstall, library: C:\Windows\Installer\MSI6AFF.tmp”

As far as I can see, when in this state, ANY custom action by the MSI produces an error.  The Citrix Provisioning Services installer has numerous custom actions so this comes up a few times.  The first custom action that causes a crash checks to see if a reboot is pending and presents you a dialog if it detects a pending reboot.  I’ve not been able to find much about the second action…

The only way I’ve found to fix this issue is to ‘rip’ Citrix PVS Target Device software out of the system.  To ensure the bits and registry keys are all captured, I install the same version over top.  My hope is the ‘reinstall’ overtop will be a perfect overlay and any corruption or breaking will be corrected by the new install.  So far, this seems to be a successful strategy.

Here is how I ripped the software out.

  1. Reverse Image the vDisk
  2. Download MSIZap (from the Microsoft Windows SDK for Windows 7 and .Net Framework 3.5 SP1)
  3. Download SetACL from Helge Klein
  4. Extract both to the same folder and save this script to it (it targets PVS 7.7 specifically) and then run it:

     

So a little explanation on why the script is needed and what it’s doing.  The MSIZap removes the keys that ‘registers’ the product with the system.  This allows us to run the MSI as if it’s installing on a clean system.  This passes the first check of the MSI which would tell you to “Repair or Remove” instead.

However, after running MSIZap permissions are futzed pretty hard in the ‘Installer\Components” keys.  They just seem to be outright removed.  My understanding of the TWA! parameters are that permissions are supposed to be changed, but not removed.  However, specifying TWA! seems to just change permissions and not ‘delete’ anything.

So I specify just the “T” command with MSIZap afterwards and this will actually remove files and registry keys.

However, it seems to be semi-broken because the registry keys it touches are left broken without Ownership or Rights permissions applied.  Attempting to install when in this state generates messages like so:

“Error 1402. Could not open key: UNKNOWN\Components\84029B5E95851FA4EADA9BE7FB000B78\D4FA67639A2C23C4CA65B9CBB4AD5446. Verify that you have sufficient access to that key or contact your support personnel.”

If you navigate to that key you’ll find you get access is denied and exploring the permissions on the key shows other errors like no ownership set:

To correct these permissions being broken I run 3 SetACL commands.

The first command sets ownership on all keys to ‘EVERYONE’.  This corrects ownership on the bad keys so we don’t have error when setting permissions.

The second command resets inheritance so that permissions are now inherited from the root key.

The third command sets ownership back to SYSTEM from EVERYONE.

Lastly, it is necessary to remove the services as the PVS installer will check to see if they exist.  If they do they will stop the install.  We found we could not delete the BNDevice.exe because it was being held open by svchost.exe.  Rebooting fixed that and we could delete it and the “C:\Program Files\Citrix\Provisioning Services” folder.

And than, with all that, we can now install PVS 7.7 overtop of itself, and then removals and in-place upgrades work as expected.

2 Comments

  1. Webster 2017-06-27 10:09 am

    Nice work.

    Reply
    • trententtye 2017-06-27 7:37 pm

      Thanks!

      Reply

Post a Comment

Your email address will not be published. Required fields are marked *

*