Tag Archives: VMware

Unable to uninstall/upgrade VMware Horizon Client within the VMware App Volumes AppStack

22 Jul

We had a very long ongoing issue wherein we couldn’t uninstall or upgrade the VMware Horizon Client within the AppStack. We had successfully installed the Horizon Client within the AppStack. However, when it was time to perform an upgrade or uninstall to the latest version, it would fail during a reboot with the following error.

Unknown HardError

We initially saw the issue on App Volumes 2.14. While we were troubleshooting for an extended period, we upgrade to App Volumes 2.18.1, and both the versions exhibited the same failure during uninstall or upgrade.

Process to reproduce the error

  • Upgrade horizon client –> reboot –> hard error
  • Uninstall horizon client –> reboot –>hard error
  • Uninstall horizon client –> install horizon client –> reboot –> hard error
  • Upgrade horizon client –> complete provisioning without reboot –> completes successful –> during next update of AppStack it crashes with Hard error
  • Uninstall horizon client –> complete provisioning without reboot –> completes successful –> during next update of AppStack it crashes with Hard error

Environment Details

VMware Horizon 7.11
VMware App Volumes 2.18.1
VMware Dynamic Environment Manager 9.10
VMware Horizon Client 5.x

Process of elimination

  • Upgrade the Horizon Client to the various 5.x version to remove any version specific Client related issues
  • We didn’t have Antivirus running on the AppStack capturing template
  • We could build the AppStack from scratch with the newer version of Horizon Client but only upgrade/uninstall would fail
  • We were honestly running out of troubleshooting ideas

Resolution

After trying out all the usual steps and avoid re-creating AppStack every single time during life cycle management, we managed to open a VMware GSS case handled by Karan Ahuja(Very helpful support engineer), which ended been worked by the engineering team(Art Rothstein – Champ in AV Eng Team). Note quite alot of logs and Procmon were exchanged from the problematic application capturing VM template. Finally, the fix was determined as a AppStack snapvol.cfg exclusion. After putting this exclusion into the AppStack – App capturing VM during provisioning we could upgrade or uninstall Horizon Client.

exclude_registry=\REGISTRY\MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileService
Path exclusion in AppStack snapvol.cfg

Disclaimer – Due to the nature of the issue and time taken to resolve it we decided to move the Horizon Client from AppStack into the base image. However, the fix is validated, and 100% working post the exclusion.

I hope you will find this information useful if you encounter the issue. A big thanks to Manivannan Arul my teammate for his continuous effort while troubleshooting with GSS over a period of 4+ months.

Thanks,
Aresh Sarkari

Black screen when re-connect to VMware Horizon virtual desktop

27 May

We had an issue after we upgraded our EUC Stack, especially VMware App Volumes 2.14 to 2.18.1. Quite a few end-users started reporting black screen when they were trying to re-connect to their desktops post the original session launch. This would mean re-connect post breaks, endpoint screen locks, next working day re-connections, etc.

EUC Environment Details:
VMware Horizon 7.11
VMware App Volumes 2.18.1
VMware Dynamic Environment Manager 9.10
VMware Horizon Client 5.x
VMware Workspace One 3.3

Process of elimination

  • If we re-created the writable volumes of the problematic end-users the black screen issue would go away. This provided us with a clue that the problem lied with VMware App Volumes – Writable Volumes
  • No errors/failures observed within the VMware DEM/Horizon logs
  • Upgrade the Horizon Client to the latest 5.x version to remove any Client related issues
  • We already had the necessary anti-virus exclusion based on VMware Antivirus Considerations in a VMware Horizon 7 Environment

Resolution
After trying out all the usual steps and avoid re-creating writable volumes for problematic end-users, we managed to open a VMware GSS case handled by Karan Ahuja(Very helpful support engineer), which ended been worked by the engineering team(Art Rothstein – Champ in AV Eng Team). Note quite alot of logs, memory dumps, and Procmon were exchanged from the problematic VM using various remote gathering techniques. Finally, the fix was determined as a writable volume snapvol.cfg exclusion. (In our case, the problem is caused by smss.exe using a copy of winlogon.exe that is on the writable volume). After putting this exclusion into all problematic end-users, they stopped seeing Black screen issues upon re-connect.

exclude_path=%SystemRoot%\System32\winlogon.exe
Path exclusion in writable volumes snapvol.cfg

In this blog, I am not outlining the steps on how to add the snapvol.cfg exclusion as my ex-colleague Daniel Bakshi outlines on a VMware blog post on how to do it step by step. I hope you will find this information useful if you encounter intermittent black screen issues.

Thanks,
Aresh Sarkari

Swagger-UI and Postman Collection for VMware Unified Access Gateway

6 May

I aimed to perform a particular VMware Unified Access Gateway (UAG) tasks programatically. After some guidance from Mark Benson he introduced me to the Swagger-UI that is available within the product.

To access the Swagger-UI on UAG open the following URL within the browser and enter your username and password.

https://uagnameorip:9443/swagger-ui/index.html
Swagger-UI – UAG API Calls

One can do alot within the swagger-ui to make various GET, POST, PUT actions. However, my preferred tool is POSTMAN. I needed a way to figure out how to get all the swagger-ui converted to POSTMAN. Upon searching, I came across this method mentioned here.

To fetch all the swagger JSON output, go to this URL on the VMware UAG Appliance.

https://uagnameorip:9443/rest/swagger.json

We have two options here. #Option1 – copy all the data from the webpage and paste it under Postman – Import – Paste Raw Text. You will have all the VMware UAG Access Gateway Rest API listed. #Option2 – Paste the above URL into Postman – Import – Import from link (This didn’t work for me maybe authentication was required)

Postman – Import

Please find attached the POSTMAN export for the VMware Unified Access Gateway Appliance 3.9.1. (Note I believe swagger-ui was availble post UAG 3.7 onwards).

Postman – API Calls UAG

I hope you will find this post useful to start using the Swagger-UI and Postman collections to begin working with UAG appliance. My request if you further create interesting scripts or perform cool activities with UAG appliance, I hope you can share it back with me?

Thanks,
Aresh Sarkari

Intermittent Clipboard issues on VMware Horizon virtual desktop

18 Apr

Recently, we had an issue within our environment where-in end-users complained of intermittently one-way clipboard not working(Virtual Desktop to Endpoint will fail). The tricky part here was it would happen intermittently to anyone without any set pattern.

Environment Details:
VMware Horizon 7.11
VMware App Volumes 2.18.1
VMware Dynamic Environment Manager 9.10
VMware Horizon Client 5.x

Process of elimination

  • We were not using the Horizon Blast GPO for setting the clipboard.
  • The clipboard was setup using DEM Horizon Smart Policies – Enabled Both Directions
  • Upgrade the Horizon Client to the latest version to remove any Client related issues
  • We already had the anti-virus process exclusion of VMwareViewClipboard.exe
  • We disabled the Writable Volumes, and the clipboard issue will never occur.

Resolution

The above test made it evident that something within the Writable Volumes was causing the intermittent clipboard issue. The next thing that came to mind is adding path/process exclusion within the snapvol.cfg. One may ask how did you determine that path, but recently we have had many application issues that needed exclusion to make them work.

What I didn’t know was which path or process, until the task manager showed a clipboard process for Horizon called – VMwareViewClipboard.exe and its Path – C:\Program Files\Common Files\VMware\Remote Experience\x64. I read many communities post having mentioned this process. However, I wasn’t sure if adding the entire path exclusion made sense as I could see many Horizon process *.exe and wasn’t sure what additional repercussions it can have. I went ahead, adding the below process exclusion.

exclude_process_name=VMwareViewClipboard.exe
Process exclusion in writable volumes snapvol.cfg

Post adding the exclusion, all the end-users with intermittent clipboard issues were always able to do two side clipboard. In this blog, I am not outlining the steps on how to add the snapvol.cfg exclusion as my ex-colleague Daniel Bakshi outlines on a VMware blog post on how to do it step by step.

Update 2nd May 2020
We had a VMware GSS support case open on the same issue, and they came back with a suggestion to exclude this registry path instead of the process exclusions. Note we been told there is no impact with process or registry, but its a good practice to do registry/path exclusions instead of the process. This registry/subkeys are responsible for the Clipboard – DEM Horizon Smart Policies.

exclude_registry=\REGISTRY\MACHINE\SOFTWARE\VMware, Inc.\VMware UEM
Process exclusion in writable volumes snapvol.cfg

I hope you will find this information useful if you encounter intermittent clipboard issues.

Thanks,
Aresh Sarkari

Black Screen on mobile devices during logon – VMware Horizon and VMware Workspace One

17 Dec

We had a strange issue in which end-users reported a black screen when they clicked on their Desktop tile in Workspace One portal on their mobile devices on Android and iOS. The moment they clicked on the endpoint the black screen would go away and it would give the logon banner and normal Windows 10 logon.

Usual Suspects

Our investigation led to Windows Logon Banner applied via the group policy causing the black screen. We were soon able to rule out by disabling the logon banner and the black screen persisted.
The black screen only appear on mobile devices. The Desktop/Laptops you didnt observe the issue.

EUC Stack

VMware Horizon 7.6
VMware App Volumes 2.14.2
VMware Identity Manager 3.3
VMware User Environment Manager 9.4
Windows 10 1803

Resolution

We managed to open the VMware GSS case and a lot of troubleshooting was carried out from re-running the VMware OSOT tool and changing the Power Configuration policy.

The final configuration that resolved the black screen issue:

Open the master image and run PowerShell with administrative rights and execute the following commands:

powercfg -change -monitor-timeout-ac 0
powercfg -change -monitor-timeout-dc 0

(Note – Here 0 means Never)

ScreenSettings

Power and Screen Settings – Windows 10

Make sure you restart the master template post implementing the commands . Take a snapshot and perform “Push-Image” operation in Horizon Administror console.

I hope you will find this information useful if you encounter the Black Screen issue. A big thanks to Manivannan Arul my teammate for his continoursly effort while troubleshooting with GSS.

Thanks,
Aresh Sarkari

Continue reading

Horizon 7.2 – RDS Farm with View Composer fails on “Customizing”

21 Jul

While creating a RDSH Farm in Horizon 7.2 using View Composer – Linked Clones and Custom Specification Manager the creation would fail on “Customization” within the View Administrator console. Upon investigation within the vCenter the Windows Servers 2012 R2 RDS Session host VM’s where not getting a valid IP and receiving the169.x.x.x APIPA addresses.

After researching quite a bit the most common solution to the problem was:

  • Un-install and re-install vmwaretools
  • Un-install and re-install Horizon Agent 7.2 on RDS Master Image

After performing the above two steps the issue completely changed from getting 169.x.x.x APIPA address to a proper DHCP server routable address. However, we are getting a different error this time:

Windows could not finish configuring the system after a generalized sysprep”.

windows error-sysprep

Final Solution

Within the master image we were using the MacAfee VSE Agent Patch 7 as the antivirus protection. This particular version was causing the issue with the sysprep to fail during customization.

After following the below MacAfee KB and installing VSE Patch 9 the error was resolved and customizing of the RDS VM as per the Custom Specification Manager was successful.

Reference Link:
Windows could not finish configuring the system (Sysprep fails when VirusScan Enterprise Patch 7/8 is included in a Windows installation image)

I hope this solution will save time to get the Horizon 7.2 RDSH Farm created quickly.

Thanks,
Aresh

How to collect logs from Horizon View 6.x/7.x Instant Clones – Desktop VM’s

7 Feb

If you have desktops deployed via Horizon View 6.x/7.x Instant Clones technology it can get very difficult to collect the Horizon View Agent logs from the desktop VM for troubleshooting/analysis purposes. The moment the end-user logs-off from the desktop it gets into the Status = Disconnected –> Deleting.

vCenter Task for log-in and log-off of the desktopvCenter Task Log-in/Log-Off

vCenter Task for Deleting –> Customizing –> AvailablevCenter Task Delete - Customizing - Available

The above operations happen very quickly. Suppose in our scenario the desktop was failing on the Status=Customizing (View Administrator). The desktops status would change into the Error state and after couple of seconds get into delete will remain in a loop until the desktop becomes available. This is by design as the Instant Clone is trying to re-create the desktop There was no way to capture the logs for analysis or troubleshooting.

Resolution:Now you can disable the recovery of the Instant Clone desktop VM if they are in the Status=Error (Strictly for troubleshooting purposes). This setting can be enabled at Desktop Pool Level

Desktop Pool Setting (disable autorecovery):

  • Open the Horizon View ADAM – (DC=vdi,dc=vmware,dc=int)
  • Go to OU=Server Groups – on you right select OU=DesktopPoolName (this is the name of your desktop pool)
  • Search for pae-RecoveryDisabled and click Edit
  • Enter Value =1 and click Add – OK
  • ADAM

Now whenever your desktop within the Pool will be in Status=Error it will not delete the VM and keep it in the Error state for you to capture the logs and perform troubleshooting. Please revert the changes of this settings once you have finished analysis. I hope these steps would be helpful leave a comment down below

Additional KB:
Connecting to the View ADAM Database (2012377)

Thanks,
Aresh