Archive | Horizon RSS feed for this section

VMware App Volumes – Volumes were not mounted due to an issue with your Writable Volume

18 Mar

Random floating desktop pools within our environment would exhibit issues where in the end-user would login to their desktop and they will be presented with a black screen with the message – Volumes were not mounted due to an issue with your Writable Volume. Please try logging in again, or contact your administrator.

Error

When this issue would surface, neither the AppStacks nor Writable Volumes would mount to the end-user desktop and if the end-user clicked on OK the session would log-off.

Environment Details

VMware Horizon 7.11
VMware App Volumes 2.18.5
VMware Dynamic Environment Manager 9.10
Windows 10 1909 Enterprise

Process of elimination

  • The App Volumes (AV) agent is able to communicate to the AV Manager on port 443 without any issues.
  • There were no SSL errors or load balancing issues communicating with the Agent/Manager.
  • We thought a particular Writable Volumes (WV) would be causing the issue. Deleted and re-created the WV still the issue would persist.
  • The issue would happen randomly for few users again and again.

Resolution

My team managed to open a VMware GSS case handled by Sanjay SP (A very helpful support engineer), he mentioned there were quite a few cases opened on a similar pattern. Following were the assessments from our logs:

  • During the first startup of Instant Clones, App Volumes Agent queries below registry key to know the customization status and updates manager with the same
    • [HKEY_LOCAL_MACHINE\SOFTWARE\VMware, Inc.\ViewComposer\ga\AgentIntegration]
      • “CustomizationState”=dword:1
  • It has a timeout of 300 seconds, and if this task times out AppVolumes manager will fail to create a unique identity for the VM in its database
  • In the App Volumes Agent logs, we see the respective timeout
    • [2021-03-09 07:11:34.009 UTC] [svservice:P1564:T1976] HandleNGVC: Waiting for NGVC to complete (count 299)
    • [2021-03-09 07:11:34.009 UTC] [svservice:P1564:T1976] Timed out waiting on NGVC after 300 seconds, disabling
  • The customization itself is working fine and we do see the registry entries getting updated with appropriate values. However, its not completed within 300 seconds. 

Fix

  • The delay in cloneprep customization was not found with IPv6 disabled on the primary nic adapter. The recommendation was to disable IPv6 since we don’t use it within the NIC adapter properties.
Disable IPv6 in the network adapters

I hope you will find this information useful if you encounter the issue. If you manage to tweak or improvise further on this solution, please don’t forget to keep me posted.

Thanks,
Aresh Sarkari

Internet Explorer crashing on Windows Server 2016 – Remote Desktop Session Host

18 Feb

We encountered a strange issue on the Windows Server 2016 Remote Desktop Session Host (RDSH) used for VMware Horizon Application Publishing. The Internet Explorer would launch and get into “Not Responding” state, and eventually, the process would close out without any errors.

IE Opening and Crashing

Process of elimination

  • We thought either Windows cumulative updates introduced the issue as it was working fine earlier.
  • There were no errors in the Windows Event Viewer (Application, System or Internet Explorer)
  • We used the Deployment Image Servicing and Management (DISM) command line tool to disable/enabled Internet Explorer without any luck.
    • dism /online /Disable-Feature /FeatureName:Internet-Explorer-Optional-amd64
    • dism /online /Enable-Feature /FeatureName:Internet-Explorer-Optional-amd64
  • Procmon is showing IE tries to launch the process multiple times, but the sub-process keep failing, and IE finally gives up at the end
IE Process launching multiple times
  • We were running out of troubleshooting ideas

Resolution

My team ended up opening a Microsoft Support case, and they could see that “Name Not Found for the ieproxy.dll” which is due to ieproxy.dll registration issues. Support confirmed they had seen similar instances in the past.

Please open command prompt with Admin rights and re-register the dll from System32 and Syswow64 folders.

%SystemRoot%\System32\regsvr32 ieproxy.dll

%SystemRoot%\Syswow64\regsvr32 ieproxy.dll

 I hope you will find this information useful if you encounter the issue. If you manage to tweak or improvise further on this solution, please don’t forget to keep me posted.

Thanks,
Aresh Sarkari

Horizon VDI – Calculator – Photos – Edge Not launching for end-users – Windows 10

8 Feb

In Windows 10 1909 VMware OST optimized image the end-users report they cannot open the following three built-in UWP windows application.

  • Microsoft Calculator
  • Microsoft Photos
  • Microsoft Edge browser

When the end-users try to open any of the three applications, nothing would happen – No error messages or pop-ups. The application doesn’t launch.

Environment Details

VMware Horizon 7.11
VMware App Volumes 2.18.5
VMware Dynamic Environment Manager 9.10

Process of elimination

  • The AppX package for (Calc, Photos and Edge) did exist in the base operating system
  • We can launch all the three applications within the optimized golden image template.
  • We were running the VMWare OSOT tool with the default VMware Windows 10 template. No additional customization or options selected.
  • One thing was evident the base template was working fine. The suspicion was around AppStack – App Volumes (We disabled the AppStacks/Writable Delivery – Same issue observed) or Dynamic Environment Manager causing the application from launching
  • We were running out of troubleshooting ideas

Resolution

Upon searching, I came across this community page – https://communities.vmware.com/t5/Horizon-Desktops-and-Apps/Windows-10-UWP-Applications-and-Taskbar/m-p/523086 and it outlined a solution of re-registering the UWP AppX package for the built-in application. We tried the fix in the DEV environment and it worked. Further it was replicated to the production setup.

Step 1: A Powershell script to register the AppX packages

Get-AppxPackage -allusers *windowscalculator* | Foreach {Add-AppxPackage -DisableDevelopmentMode -Register “$($_.InstallLocation)\AppXManifest.xml”}
Get-AppxPackage -allusers *windows.photos* | Foreach {Add-AppxPackage -DisableDevelopmentMode -Register “$($_.InstallLocation)\AppXManifest.xml”}
Get-AppXPackage -AllUsers *edge* | Foreach {Add-AppxPackage -DisableDevelopmentMode -Register "$($_.InstallLocation)\AppXManifest.xml"}

Step 2 : Create a Dynamic Environment Manager – Logon Tasks

We selected to put the Powershell script within the UEM Share as the end-users have the read- access.

DEM - Logon Task
DEM-LogonTasks

 Quick Update based on 4th Aug 2021 (Thanks to Curtis for bring this up in the comments section)

The above DEM 9.10 logon task no longer works in situation where end-users dont have local administrative priviledges users not being able to run the script at logon.

In the latest version of Dynamic Enivornment Manager 20XX onwards, you can now hook logon tasks into Elevated Tasks by using Privilege Elevation rules.

In DEM:

1. User Environment > Privilege Elevation > Create new privilege elevation rule

2. In the “Type” drop down menu, select “Elevated Task”

3. Click “Add”

4. In the Executable field:
“C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe”

5. In the Arguments field type the path to your script logon script

6. In User Environment > Logon Tasks, select the logon task that runs and registers the UWP apps.

7. Check “Elevated Task” and in the drop down select the Elevated Task you just created in the list.

After this, the script should be able to run at logon regardless of whether or not the user has local administrator rights!

I hope you will find this information useful if you encounter the issue. If you manage to tweak or improvise further on this solution, please don’t forget to keep me posted.

Thanks,
Aresh Sarkari

Script create read-only account for monitoring VMware Unified Access Gateway

23 Sep

We have been using VMware Unified Access Gateway (UAG) for quite a few years. To monitor the appliance using vROPS or other monitoring tools or API calls scripts you need a read-only monitoring account created in the console under “Account Settings”.

Account Settings - UAG
Read-only account for monitoring

In our deployment we have 14 UAG appliances (Internal/External) – Yes we tunnel internal connections too. Post the upgrade we had to re-create the read-only account for the API call monitoring on all 14 appliances. The following script I wrote to create the read-only account per UAG server. Just change the IP and point to another UAG to create accounts.

####################################################################
# Create ready-only account in the VMware Unified Access Gateway Appliance
# for monitoring purposes using vROPS or API etc.
# Author - Aresh Sarkari (@askaresh)
# Version - V5.0
####################################################################


# Ignore UAG cert errors (self signed or 

add-type @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
        public bool CheckValidationResult(
            ServicePoint srvPoint, X509Certificate certificate,
            WebRequest request, int certificateProblem) {
            return true;
        }
    }
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]'Ssl3,Tls,Tls11,Tls12'


##API Call to make the intial connection to the UAG Appliance##

$Uri = "https://10.0.0.1:9443/rest/v1/config/adminusers/logAdminUserAction/LOGIN"
$Username = "admin"
$Password = "adminpassword"

$Headers = @{ Authorization = "Basic {0}" -f [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $Username,$Password))) }

Invoke-RestMethod -SessionVariable DaLogin -Uri $Uri -Headers $Headers


###API Call to create the user account with read-only access under VMware Unified Access Gateway##

$body = @{
  name = "UAG_vRops"
  password= "typeyourpassword"
  enabled=$true
  roles = @("ROLE_MONITORING")
  noOfDaysRemainingForPwdExpiry=0
} | ConvertTo-Json

$output = Invoke-RestMethod -WebSession $DaLogin -Method Put -Uri "https://10.0.0.1:9443/rest/v1/config/adminusers" -Body $body -ContentType "application/json"

Write-Output $output

GitHub https://github.com/askaresh/scripts/blob/master/uagreadonlyacct

I hope you will find this script useful to create the UAG read only accounts and would not have to create them manually on multiple appliances. My request if you further enhance the script or make it more creative, I hope you can share it back with me?

Thanks,
Aresh Sarkari

Unable to uninstall/upgrade VMware Horizon Client within the VMware App Volumes AppStack

22 Jul

We had a very long ongoing issue wherein we couldn’t uninstall or upgrade the VMware Horizon Client within the AppStack. We had successfully installed the Horizon Client within the AppStack. However, when it was time to perform an upgrade or uninstall to the latest version, it would fail during a reboot with the following error.

Unknown HardError

We initially saw the issue on App Volumes 2.14. While we were troubleshooting for an extended period, we upgrade to App Volumes 2.18.1, and both the versions exhibited the same failure during uninstall or upgrade.

Process to reproduce the error

  • Upgrade horizon client –> reboot –> hard error
  • Uninstall horizon client –> reboot –>hard error
  • Uninstall horizon client –> install horizon client –> reboot –> hard error
  • Upgrade horizon client –> complete provisioning without reboot –> completes successful –> during next update of AppStack it crashes with Hard error
  • Uninstall horizon client –> complete provisioning without reboot –> completes successful –> during next update of AppStack it crashes with Hard error

Environment Details

VMware Horizon 7.11
VMware App Volumes 2.18.1
VMware Dynamic Environment Manager 9.10
VMware Horizon Client 5.x

Process of elimination

  • Upgrade the Horizon Client to the various 5.x version to remove any version specific Client related issues
  • We didn’t have Antivirus running on the AppStack capturing template
  • We could build the AppStack from scratch with the newer version of Horizon Client but only upgrade/uninstall would fail
  • We were honestly running out of troubleshooting ideas

Resolution

After trying out all the usual steps and avoid re-creating AppStack every single time during life cycle management, we managed to open a VMware GSS case handled by Karan Ahuja(Very helpful support engineer), which ended been worked by the engineering team(Art Rothstein – Champ in AV Eng Team). Note quite alot of logs and Procmon were exchanged from the problematic application capturing VM template. Finally, the fix was determined as a AppStack snapvol.cfg exclusion. After putting this exclusion into the AppStack – App capturing VM during provisioning we could upgrade or uninstall Horizon Client.

exclude_registry=\REGISTRY\MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileService
Path exclusion in AppStack snapvol.cfg

Disclaimer – Due to the nature of the issue and time taken to resolve it we decided to move the Horizon Client from AppStack into the base image. However, the fix is validated, and 100% working post the exclusion.

I hope you will find this information useful if you encounter the issue. A big thanks to Manivannan Arul my teammate for his continuous effort while troubleshooting with GSS over a period of 4+ months.

Thanks,
Aresh Sarkari

Black screen when re-connect to VMware Horizon virtual desktop

27 May

We had an issue after we upgraded our EUC Stack, especially VMware App Volumes 2.14 to 2.18.1. Quite a few end-users started reporting black screen when they were trying to re-connect to their desktops post the original session launch. This would mean re-connect post breaks, endpoint screen locks, next working day re-connections, etc.

EUC Environment Details:
VMware Horizon 7.11
VMware App Volumes 2.18.1
VMware Dynamic Environment Manager 9.10
VMware Horizon Client 5.x
VMware Workspace One 3.3

Process of elimination

  • If we re-created the writable volumes of the problematic end-users the black screen issue would go away. This provided us with a clue that the problem lied with VMware App Volumes – Writable Volumes
  • No errors/failures observed within the VMware DEM/Horizon logs
  • Upgrade the Horizon Client to the latest 5.x version to remove any Client related issues
  • We already had the necessary anti-virus exclusion based on VMware Antivirus Considerations in a VMware Horizon 7 Environment

Resolution
After trying out all the usual steps and avoid re-creating writable volumes for problematic end-users, we managed to open a VMware GSS case handled by Karan Ahuja(Very helpful support engineer), which ended been worked by the engineering team(Art Rothstein – Champ in AV Eng Team). Note quite alot of logs, memory dumps, and Procmon were exchanged from the problematic VM using various remote gathering techniques. Finally, the fix was determined as a writable volume snapvol.cfg exclusion. (In our case, the problem is caused by smss.exe using a copy of winlogon.exe that is on the writable volume). After putting this exclusion into all problematic end-users, they stopped seeing Black screen issues upon re-connect.

exclude_path=%SystemRoot%\System32\winlogon.exe
Path exclusion in writable volumes snapvol.cfg

In this blog, I am not outlining the steps on how to add the snapvol.cfg exclusion as my ex-colleague Daniel Bakshi outlines on a VMware blog post on how to do it step by step. I hope you will find this information useful if you encounter intermittent black screen issues.

Thanks,
Aresh Sarkari

Intermittent Clipboard issues on VMware Horizon virtual desktop

18 Apr

Recently, we had an issue within our environment where-in end-users complained of intermittently one-way clipboard not working(Virtual Desktop to Endpoint will fail). The tricky part here was it would happen intermittently to anyone without any set pattern.

Environment Details:
VMware Horizon 7.11
VMware App Volumes 2.18.1
VMware Dynamic Environment Manager 9.10
VMware Horizon Client 5.x

Process of elimination

  • We were not using the Horizon Blast GPO for setting the clipboard.
  • The clipboard was setup using DEM Horizon Smart Policies – Enabled Both Directions
  • Upgrade the Horizon Client to the latest version to remove any Client related issues
  • We already had the anti-virus process exclusion of VMwareViewClipboard.exe
  • We disabled the Writable Volumes, and the clipboard issue will never occur.

Resolution

The above test made it evident that something within the Writable Volumes was causing the intermittent clipboard issue. The next thing that came to mind is adding path/process exclusion within the snapvol.cfg. One may ask how did you determine that path, but recently we have had many application issues that needed exclusion to make them work.

What I didn’t know was which path or process, until the task manager showed a clipboard process for Horizon called – VMwareViewClipboard.exe and its Path – C:\Program Files\Common Files\VMware\Remote Experience\x64. I read many communities post having mentioned this process. However, I wasn’t sure if adding the entire path exclusion made sense as I could see many Horizon process *.exe and wasn’t sure what additional repercussions it can have. I went ahead, adding the below process exclusion.

exclude_process_name=VMwareViewClipboard.exe
Process exclusion in writable volumes snapvol.cfg

Post adding the exclusion, all the end-users with intermittent clipboard issues were always able to do two side clipboard. In this blog, I am not outlining the steps on how to add the snapvol.cfg exclusion as my ex-colleague Daniel Bakshi outlines on a VMware blog post on how to do it step by step.

Update 2nd May 2020
We had a VMware GSS support case open on the same issue, and they came back with a suggestion to exclude this registry path instead of the process exclusions. Note we been told there is no impact with process or registry, but its a good practice to do registry/path exclusions instead of the process. This registry/subkeys are responsible for the Clipboard – DEM Horizon Smart Policies.

exclude_registry=\REGISTRY\MACHINE\SOFTWARE\VMware, Inc.\VMware UEM
Process exclusion in writable volumes snapvol.cfg

I hope you will find this information useful if you encounter intermittent clipboard issues.

Thanks,
Aresh Sarkari

Black Screen on mobile devices during logon – VMware Horizon and VMware Workspace One

17 Dec

We had a strange issue in which end-users reported a black screen when they clicked on their Desktop tile in Workspace One portal on their mobile devices on Android and iOS. The moment they clicked on the endpoint the black screen would go away and it would give the logon banner and normal Windows 10 logon.

Usual Suspects

Our investigation led to Windows Logon Banner applied via the group policy causing the black screen. We were soon able to rule out by disabling the logon banner and the black screen persisted.
The black screen only appear on mobile devices. The Desktop/Laptops you didnt observe the issue.

EUC Stack

VMware Horizon 7.6
VMware App Volumes 2.14.2
VMware Identity Manager 3.3
VMware User Environment Manager 9.4
Windows 10 1803

Resolution

We managed to open the VMware GSS case and a lot of troubleshooting was carried out from re-running the VMware OSOT tool and changing the Power Configuration policy.

The final configuration that resolved the black screen issue:

Open the master image and run PowerShell with administrative rights and execute the following commands:

powercfg -change -monitor-timeout-ac 0
powercfg -change -monitor-timeout-dc 0

(Note – Here 0 means Never)

ScreenSettings

Power and Screen Settings – Windows 10

Make sure you restart the master template post implementing the commands . Take a snapshot and perform “Push-Image” operation in Horizon Administror console.

I hope you will find this information useful if you encounter the Black Screen issue. A big thanks to Manivannan Arul my teammate for his continoursly effort while troubleshooting with GSS.

Thanks,
Aresh Sarkari

Continue reading

VMware EUC – Horizon, UAG, VIDM and AppVolumes – NSX Load Balancing – Health Check Monitors

2 Feb

There is no single place to find a consolidated list of Load balancer health check monitors (aka Service Monitors in NSX) for the VMware EUC products:

I have been using VMware NSX load balancer across the board. The below details will provide an overview of what to enter for the health monitors. Note – If you are using something more meaningful  for your environment leave feedback in the comments section. I will try to implement the same and update the blog later.

VMware Unified Access Gateway (UAG)

Create a new Service Monitor under NSX and call is UAG_https_monitor. Refer to the screenshot for more details.

UAG Service Monitor

Send String: GET /favicon.ico
Response code: 200s

VMware Identity Manager or Workspace ONE Access

Create a new Service Monitor under NSX and call is VIDM_https_monitor. Refer to the screenshot for more details.

VIDM Service Monitor
Send String: GET /SAAS/auth/login
Response code: 200s

VMware Horizon Connection Servers

Update 13th Sep 2021 – For all Horizon version 7.10 and above please start using the following service monitor within NSX.

Send String: GET /favicon.ico
Response code: 200s

You can use this string for versions 7.7 or upto 7.10. Create a new Service Monitor under NSX and call is Horizon_https_monitor. Refer to the screenshot for more details.

image
Send String: GET /broker/xml/
Receive string: /styles/clientlaunch-default

VMware App Volumes

Create a new Service Monitor under NSX and call is AV_https_monitor. Refer to the screenshot for more details.

AV Service Monitor

I hope you will find these monitors useful in monitoring the VMware EUC products.

Thanks,
Aresh Sarkari

Poor man’s Samsung DEX HUB and VMware Horizon Advantage

22 Jun

I had been intrigued by the Samsung DeX mode post its launch but didn’t have the courage to buy the 125$ (Rs. 10,000/- INR) Samsung DeX Station. I was on a look-out for an alternate Hub which could do the DeX mode on my Galaxy S8+ for a lot less. After searching @ AliExpress I finally managed to find a hub called EASYA Thunderbolt 3 USB C to HDMI Adapter DeX Mode for Samsung Galaxy S8/S9 which had some good positive reviews and for 33$

The moment of truth was to plug-in the Galaxy S8+ and try the DeX mode. Attempt-1: Managed to plug the phone to the hub and HDMI monitor as the output. Next thing I noticed was the screen mirror got enabled and DeX Mode pop-up wouldn’t come or get detected.

Attempt-2: Additional to the above I plugged in the Power in the USB-C 3.1 PD Port and magically the pop-up appeared on the phone “Start DeX Mode

If you don’t have the wireless mouse plugged in the entire Galaxy S8+ screen acts like a mouse trackpad which can come-in handy.

EASYA Thunderbolt 3 USB Type-C Hub To HDMI Adapter Dex Mode

Productivity with VMware Horizon:

The Horizon Client available on the Android Store has integration with DeX mode that enables you to use the Virtual Desktop in Full screen mode. I launch my Windows 10 Desktop and use it for an entire day. I was easily able to work on the following applications without any issues

  • Microsoft Outlook Client
  • Chrome and Firefox browser
  • Skype for Business (Audio/Chat Only) – Video was having issues
  • VMware Performance Tracker was showing the CPU and Network Bandwidth Usage graphs in real-time
  • There was no lag or any sign of slowness in any form
  • CPU Usage on the phone at an average of 4-6%

Known Observations:

  • The phone didn’t heat all day during its usage
  • The HUB was reasonably warm during the entire day usage
  • The limited DeX compatible Application works good in full-screen

More Picture on the Usage

DeX Mode and Horizon Client Launch
Horizon Client

DeX Mode and Full Screen – Windows 10 + Dell 24 inch Monitor
Full Screen - Windows 10 VDI

Hardware Setup – Logitech M140 Bluetooth Keyboard + Mouse
Hardware-Setup

More Documentation on Samsung DeX + VMware Horizon

Using Horizon Client with Samsung DeX
Enable the DeX Mode Auto Launch Feature

I hope you find this HUB review and DeX mode usage with Horizon useful and will be able to use it as a daily driver. Let me know if you would like to know more in the comments section

Thanks,
Aresh