SCVMM: Install VM components Failed

/ / /

I’m attempting to deploy a virtual machine from a template but I get this error: Install VM components: Failed.



SO…  Long story short; if you are encountering this error, I would suggest booting your VHD file in a VM and re-sysprep /generalize it.  If you’ve maxed out on sysprep’s I have a post earlier in my blog on how to get around the 3-times limit and rerun sysprep.  Alternatively, you can try what I did, but I can’t gaurantee your success and replace the BCD file in your Library VHD with a BCD you *know* has been sysprepp’ed and try redeploying it.

I’ve ensured that HTTP and HTTPS is not blocked (firewall is disabled) and the agent on my SCVMM machine is installed and running.  So this error message is somewhat useless.

So I took my two boxes, 2012-SCVMM (the SCVMM server) and S5000VXN-SERVER (the Hyper-V host) and procmon’ed them while it was attempting to “Install VM Components” to try and understand what it’s trying to do.

After reinitating the task, we can see that the vmmAgent.exe on the Hyper-V host accesses the file about 2 seconds after I submit the command to retry the job.

A few seconds after this, it appears to “CreateFile” in the WindowsTemp directory; but this is not actually correct.

What it is really doing is *mounting* the VHD into a folder in the WindowsTemp directory.  You can actually watch this in action if you view Disk Management on the Hyper-V host while you execute the task.

So now that we know it’s mounting the file as a volume this helps us narrow down on what Hyper-V is attempting to do…  And I suspect what it is attempting to do is “offline servicing” of the attached vdisk/vhd.

After attaching the vdisk the next thing it does it query the BCD file on the system.  Maybe it needs to be in a certain mode to operate?  I’m not sure…

Continuing on we can see that events are written to the Microsoft-Windows-FileShareShadowCopyProvider Operational.evtx, System.evtx, and Microsoft-Windows-Bits-CompactServer Operational.evtx event logs.  Examining each log at the time stamp showed the FileShareShadowCopyProvider and System log were just noticed of volume shadow copy starting, but the BITS event log was interesting.



It showed that it was doing something with the BCD file.  The Hyper-V host was *serving* it out.  I suspect it was serving it to the SCVMM.
Looking back at the SCVMM server we see that was executing some WinRM commands.  Sadly, we do not know what commands it was trying to send.

If I mount the vhd file and check out the BCD file I can see that it appears to be corrupted in that it doesn’t know what the proper boot device should be.


It’s not actually corrupted though; the reason why the devices are unknown is because bcdedit isn’t finding the disk signature of the volume I mounted.  But it has the disk signature because I can boot with it without issue.

Continuing on…

In order to try and find out what commands WSMAN was sending I enabled debugview on the SCVMM server:

I then reproduced the error by rerunning the Create Virtual Machine job.

DebugView gave me more information to narrow down what was happening.  It appears that the process is failing with:

Doing some googling on this took me to a Korean Microsoft page where the following was stated:

Thinking about it, I do not think my VHD was SysPrep’ed.  I find it interesting that sysprep appears to do something to the BCD file.  To find out what sysprep does to the BCD file I booted up my 2012 VHD into Windows and ran sysprep, generalize and shutdown.

And……….?  Lets go to DebugView:


Voila!  It appears much better than before.  It found the drive correctly and checked the BCD file and found it is the “Generalize” state.  The *actual* image I originally made was NOT sysprep’ed and all I did was replace the BCD file, but it allowed it to continue beyond and the machine actually completed the imaging process properly.  It joined the domain and whatever else the answer file was I gave it in SCVMM.

It appears I was correct in my earlier assumption about what SCVMM is doing.  It goes and grabs the BCD file and transfers it from the target machine to itself, “fixes it up” (not sure what it’s doing at this stage precisely), then sends it back for injection.  Certainly a bit of a complicated process with a fair bit that can go wrong, but shame on Microsoft for having such a poor error message.  I suspect it wouldn’t have required much effort to push out a real message; something to the effect of, “The BCD of this vDisk does not appear to have been through the sysprep /generalize process.  Please rerun sysprep against the image and try again”.

SO…  Long story short; if you are encountering this error, I would suggest booting your VHD file in a VM and re-sysprep /generalize it.  If you’ve maxed out on sysprep’s I have a post earlier in my blog on how to get around the 3-times limit and rerun sysprep.  Alternatively, you can try what I did, but I can’t gaurantee your success and replace the BCD file in your Library VHD with a BCD you *know* has been sysprepp’ed and try redeploying it.

Read More

Manually add Windows startup scripts (or inject startup scripts into an offline image)

/ / /

Due to the CGP issue, our solution is to add a startup script to each vDisk.  Since I don’t want to make a version of each vDisk than attach it to a server than boot it up than gpedit.msc…  We have around 10 vDisks and that process would be annoying and take a while.  So I decided to investigate doing it offline as mounting a VHD using cvhdmount.exe and then injecting the startup script would be a lot easier.

To do that, one simply needs to browse to:
“C:\windows\system32\GroupPolicy\Machine\Scripts\Startup” (for machine startup script, aka, a script that starts when your computer starts up) or “C:\windows\system32\GroupPolicyUsers\Machine\Scripts\Startup” (for all users startup script [I’m assuming since I actually didn’t go through and test the user portion]) and copy your script file there.
Then, back out one level to C:\windows\system32\GroupPolicy\Machine\Scripts and edit Scripts.ini to include your new script file; incrementing the last line.
Read More

(OS 10061)No connection could be made because the target machine actively refused it. : Unable to connect to the CGP tunnel destination (

/ / /


This has been an ongoing problem for us (Unable to connect to the CGP tunnel destination (

I may have found out why it was happening in our environment.  We are using Provisioning Services and with it we are using two NIC’s, one for the Provisioning Services and one for Standard networking.

It appears the XTE service became configured to use the Provisioning Services NIC.  This was verified in the httpd.conf in the C:\Program Files (x86)\Citrix\XTE\conf folder.

Provisioning NIC and Production (network) NIC


httpd.conf as was when the system booted (and non-functional)

When I traced the XTE service using procmon.exe and wireshark with this non-functional conf this is what I saw when I attempted to launch the application:

You can see it attempt to connect to itself via 1494 but then nothing else happens
Wireshark shows virtually nothing on the network and nothing related to IMA
When I edited the file to have the Production NIC…

then restarted the XTE service and retraced via Procmon and Wireshark…
We now see tons of activity and the application now launches without issues.


We have now found why we are getting this error, and why we are getting it intermittently.  The issue is we are using PVS with multi-homed NIC’s.  One NIC (LanAdapter 1) is the “Provisioning” network, and the second NIC (LanAdapter 2) is the “Production” network.  The Provisioning network is on a completely seperate vLan and sees no traffic outside of it’s little network.  The ICA Listener was attaching itself to the Provisioning network instead of the production network, so when we tried to connect to the server it would fail with the CGP tunnel error because the outside network cannot talk to the Provisioning network.  To attempt to resolve this issue one of our techs (Saman) created a group policy preference registry key that set the following value (HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Terminal Server\WinStations\ICA-TCP – LanAdapter):

By setting it to “2” we could ensure the ICA listener is always listening on LanAdapter 2, our production network.  Unfortunately, a Windows Update appears to have caused either Group Policy Registry Preferences to execute (sometimes) *after* the IMAService service started, or allowed the IMAService service to start *before* Group Policy Registry Preferences.  IMAService will recreate that file every second restart.  To resolve this issue I created a startup script that executes after 65 seconds, deleting the httpd.conf file and restarting the appropriate services until the httpd.conf file is recreated.

In my testing it appears you need to restart the “IMAService” service twice to get it to recreate the httpd.conf file.  Because of this, I created the script to retry up to 3 times to try and regenerate the file.

Read More

Enable advanced logging on the XTE service of a XenApp server

/ / /

Back in February or March we did Windows updates on our PVS XenApp servers and then sometime after the servers do not allow anyone to login.

The issue can crop up as a CGP Tunnel error message, a protocol driver error message or something along those lines.  We do not know why it’s happening or why it only happens after we apply Windows updates from that time period.  The odd thing is it’s intermittent as well, we can launch 20 systems from 1 vDisk that has the updates applied and everything will be fine for 2 weeks then, suddenly, 5 of the systems won’t allow logins via ICA with the errors.  Rebooting the servers sometimes fixes it, sometimes not.  Very intermittent and very weird.  So I attempted to troubleshoot this issue again by adding:

to the httpd.conf in the XTE folder of a server that was exhibiting these issues.  The log levels are listed here:

After adding those lines and restarting the XTE service the issue resolved itself!  Frustrating to be sure, and I will look at adding this line to our vDisk image so that when it crops up we’ll have more diagnostic data to look at.

Read More

Citrix Provisioning Services (PVS) 6.1 – Automatic vDisk Update “Update device failed to shutdown within the timeout period.”

/ / /

I have setup our PVS environment to execute the vDisk Automatic Update feature utilizing a custom script (update.bat).  This script does a bunch of things, resync’s the time with NTP (to avoid daylight savings issues), refreshs GPO’s, executes Windows Update, cleans up temp files, runs the PVS optimizer, etc.

Unfortunately this can take longer than 30 minutes.  For some reason, when executing the ESD client as “None” (aka, so a script runs) the “Update timeout” doesn’t seem to take effect, instead the 30 minute shutdown timeout is on the clock.

ESD client is set to none
You cannot increase the shutdown timeout beyond 30 minutes
2013-05-22 10:10:59,690 [10] INFO  Mapi.MapiIPC - [xipProcessor] Starting an update on (System.Collections.Generic.Dictionary`2[System.String,System.Object]) 2013-05-22 11:02:07,763 [10] ERROR Mapi.MapiIPC - [xipProcessor] [WSCTXBLD303T] Update device failed to shutdown within the timeout period. 2013-05-22 11:02:07,763 [10] INFO  Mapi.MapiIPC - [xipProcessor] [WSCTXBLD303T] Update device failed to shutdown within the timeout period. 2013-05-22 11:02:56,186 [10] ERROR Mapi.MapiIPC - [xipProcessor] [WSCTXBLD303T] Submit image failed (VM: WSCTXBLD303T, Image: XenApp65Tn02, Update device failed to shutdown within the timeout period.) 2013-05-22 11:02:56,186 [10] INFO  Mapi.MapiIPC - [xipProcessor] [WSCTXBLD303T] Submit image failed (VM: WSCTXBLD303T, Image: XenApp65Tn02, Update device failed to shutdown within the timeout period.)
Total time for the updates was 49 minutes (10:11AM to 11:02AM)

The only solution I have been able to come up with so far is set to run the updates in less than 30 minutes.  I think I’ll attempt changing the “Update.bat” to “Preupdate.bat” and see if that avoids the “Shutdown timeout”.

Unfortunately, I do not know when the shutdown timeout period starts or why it starts.  I was hoping the “Update timeout” was started when running the “Update.bat” file.  This does not appear to be so, sadly.

Citrix documentation implies that you should work hard to keeping the timeout below 30 minutes.

“Citrix recommends to only apply updates that can be downloaded and installed in 30 minutes or less.”

Preupdate.bat does not appear to operate under the Update Timeout either.

======================EDIT 2=================
To increase the limit you need to create a registry key called “DiskProvider” and create a dword with the decimal value of the length of time you want the *total* time called “deviceShutdownTimeout”.

NOTE: This does NOT change the value in the GUI and will override the  value in the GUI, regardless of what it is set to.  You need to restart the SOAP service after making this change.  This registry key must exist on the PVS server that the site designates as the “vDisk Update Server”


Read More

ERROR 0x8024402c Windows Update

/ / /

Recently, I was applying Windows Update to a 2008 system and it failed on 4 updates after being successful for months.  I was unsure why, but the updates were Office updates.  I don’t think that the fact they were Office updates are important, but it’s something to mention anyways.

Symptoms of the issues I found and the resolution for this issue.

1) Getting “ERROR 8024402C” when running Windows Update.
2) Checking %WINDIR%\WindowsUpdate.log reveals lines like:

To determine the cause of the issue, I used the nicely revamped Event Viewer and looked at the BITS-Client logs.  Which was a waste, nothing showed up there.  I checked the WindowsUpdateClient log and nothing was there either.  I then learned BITS uses WinHTTP when I was googling for this issue and saw there was a WinHTTP log file.  (You may have to enable analytics and debug logs).  I enabled the Diagnostics Log.

When reattempting to execute Windows Update I went back into the log and scanned through it.  I found the following:

Windows update was going to the wrong server!  The event viewer said it was going to  This was our old server address and we since replaced it with going directly to the IP of that server via GPO.

Checking regedit for the WU preferences showed it was pointing to the correct server, but for some reason Windows Update wasn’t picking up the new server.  Rebooting the machine and refreshing the GPO did not resolve the issue.

This is the correct settings

Saman suggested some fixes:

net stop wuauserv
net stop BITS
net start wuauserv
net start BITS
wuauclt /resetauthorization /detectnow
wuauclt /reportnow

These did not appear to work however.  But, we did try the following:
esentutl /p %windir%securitydatabasesecedit.sdb /o
Gpupdate /force

And I believe this worked.  After running this command, WinHTTP reported that it was pulling the Windows Update from the Microsoft servers, not our WSUS server: is not our WSUS server

At this point I could have probably ran the net stop and net start commands and it may have worked, but I rebooted the server instead.

Upon the server coming back up I reran Windows Update and confirmed it was pulling from our WSUS server:


Windows Update then downloaded and installed the updates successfully!

Read More

Corrupted VHD files

/ / /

Corrupt VHD files…  We utilize PVS and when we were attempting to update the target device software utilizing Hyper-V we got several error messages saying the VHD is corrupted.  The actual root of the issue is that PVS makes 16MB block size VHD files (thanks SAMAN!) and Windows 2008 only reads VHD files that are 512KB or 2MB block sized files.

Attempting to mount the VHD file via disk management fails as well.  The only utility that would mount the VHD file that was generating the error messages at the end of this post is the Citrix CVHDMount utility available via PVS (C:\Program Files\Citrix\Provisioning Services\CVHDMount.exe)

Since our Hyper-V server was separate from the PVS server, we needed to install the drivers to allow CVHDMount.exe to work.  In order to do this you need to install the drivers in this folder:
C:\Program Files\Citrix\Provisioning Services\drivers

You can right-click “Install” on the cfsdep2.inf file.

For the other driver you need to open Device Manager, go through “Add new hardware…”, “Add legacy hardware” and then browse to the drivers folder and add the cvhdbusp6.inf.

Now we would mount the disk as a drive letter using this command line:

At this point you need to set the disk to “Offline” in Disk Management.

From here we can now add the disk as a physical disk to Hyper-V.

And we can now do the Target Device Update.  Error messages seen during this troubleshooting:


Read More

AppV Error 25109 – The installation program could not create the configuration data store. Please see the installation log file for more information.

/ / /

We recently encountered this issue with AppV 4.6 Management server when installing an additional AppV streaming server.

The issue appeared to us to be related to a permissions issue on the database.  For some awful reason, to resolve this issue we had to add the SYSADMIN right to the account on the SQL database.  Since we are running an enterprise solution here, adding the user to the Domain Admins as suggested all over the web would not work as we segregate those services.  It also makes sense that adding the account to the Domain Admins would work on the smaller environments because SQL Express uses AD permissions and would add the Domain Admins to the highest privileges to the local install of SQL Express.

So, for enterprise accounts out there, Microsoft has a horrible setup for adding servers to AppV as the account that does the install needs the highest rights on the DB.  I don’t know why but it really screams to me that this is poor design.

Read More
Everything you can imagine is real
Pablo Picasso

Execute Windows XP klist.exe purge without user intervention

/ / /

Using AutoIt:


Read More