The last little while we’ve had an issue with some applications seemingly being published to “C:\App-V Packages” instead of our defined path “D:\AppVData\PackageInstallationRoot”.
Video of the issue in action that I was able to setup a repro:
So what’s going on here that would cause this to happen?
I turned to the event viewer to find the time when this package “8FE4C99E…” was published and just looked to see if there was anything surrounding it.
This is the event showing the package being published to “C:\App-V Packages”. There is an odd thing different about this publish event compared to the rest (that you can see above). Between ‘Configure Package’ and when the ‘Streaming’ starts there is a ‘Publishing Refresh’ event.
Why is there a Publishing Refresh event? It turns out, it can be triggered by a group policy refresh.
Notice the time stamps?
So, is there something that occurs during Group Policy refresh that could cause this?
To test this I used procmon to monitor for PackageInstallationRoot registry key and ran a Group Policy Refresh:
The key is deleted then recreated and inbetween the AppV service is trying to read it.
So what is happening here?
At our organization we prefer to set Group Policies using Group Policies Preferences – Registry because of the power and flexibility of Item Level Targetting. We prefer to only apply registry keys/policies to specific systems in certain groups rather than having a complicated OU structure. Since I know we have a policy that configures our AppV5 values I went and looked into how this value was being configured:
We have this value set to ‘Replace‘. What does replace do?
|Replace||Delete and recreate a registry value or key for computers or users. If the target is a registry value, the net result of the Replace action is to overwrite all existing settings associated with the registry value. If the target is a registry key, the net result is to delete all values and subkeys in the key, leaving only a default value name with no data. If the registry value or key does not exist, then the Replace action creates a new registry value or key.|
Well, process monitor is showing EXACTLY that scenario. Replace is deleting the key and recreating it. In between that time ‘AppVClient.exe’ is trying to read that value. If PackageInstallationRoot doesn’t exist, then AppV defaults to ‘C:\App-V Packages” as the PackageInstallationRoot.
It appears we have a race condition between group policy being applied and when our AppVClient is refreshing. It is less than 600ms but that’s enough time for a package or two to start their refresh and get caught in ‘default folder’ purgatory.
How can we fix this?
The easiest solution and the one we are going to implement:
We’re going to change the action to “Update”. When I initially tried to change the action was greyed out. When “Remove this item when it is no longer applied” is checked, it forces the action to “Replace”.
I had to ‘uncheck’ that value
And then I could select ‘Update’.
After updating the GPO and refreshing policies on the affected system, I ran procmon to verify that it does not delete the value anymore:
It does not delete the value, it now ‘Sets’ the value to the same string we had before. I don’t know if the individual ‘Set’ action will cause an issue; I suspect not since the value will always exist and is technically the same value throughout the entire set of actions that this issue is now ‘resolved’ for us.