Tag Archives: VMware EUC

Upgrade from VMware Horizon 7.13.1 to 8.6 (2206) fails with VMware Horizon View Blast Secure Gateway (VMBlastSG) could not be installed

22 Dec

Many customers are already in the process of upgrading from VMware Horizon 7.x to 8.x or will soon upgrade as the End Of Life dates are upcoming in April 2023. I want to share a rare experience wherein the Horizon upgrade from 7.13.1 to 8.6 version failed. In the rare occasion where in upgrade fails in the below mentioned manner, the workaround steps will come in handy.

We have only received the workaround from VMware support, and I intend to update the post once I get a complete RCA. At least the workaround can help someone not have to revert the entire environment instead, follow the workaround and avoid a lot of rework.

Issue

During the upgrade of the first connection server in the POD1, we encountered the following error five mins into the upgrade. Note before starting the upgrade, the entire health dashboard for the POD was green and included backup and snapshots.

Environment Overview

Let take a look at the environment details to provide an high-level overview:
Active Site (POD1)

  • 5 VMware Horizon Connection Servers 7.13.1
  • SQL Database on Microsoft SQL 2016 Always-on – EventsDB
  • The 5 Brokers are behind an NSX Load balancer

Active Site (POD2)

  • 5 VMware Horizon Connection Servers 7.13.1
  • SQL Database on Microsoft SQL 2016 Always-on – EventsDB
  • The 5 Brokers are behind an NSX Load balancer

Observations

The logs from the installer had the following message it’s complaining about insufficient privileges, which the installer is run with admin privileges already.

MSI (s) (A0:E0) [06:12:07:977]: Executing op: ActionStart(Name=InstallServices,Description=Installing new services,Template=Service: [2])
Action 6:12:07: InstallServices. Installing new services
MSI (s) (A0:E0) [06:12:07:977]: Executing op: ProgressTotal(Total=9,Type=1,ByteEquivalent=1300000)
MSI (s) (A0:E0) [06:12:07:977]: Executing op: ServiceInstall(Name=PCOIPSG,DisplayName=VMware Horizon View PCoIP Secure Gateway,ImagePath="C:\Program Files\VMware\VMware View\Server\bin\SecurityGateway.exe",ServiceType=16,StartType=3,ErrorControl=1,,Dependencies=[~],,,Password=**********,Description=Provides VMware Horizon View PCoIP gateway services.,,)
InstallServices: Service:
MSI (s) (A0:E0) [06:12:08:228]: Executing op: ServiceInstall(Name=VMBlastSG,DisplayName=VMware Horizon View Blast Secure Gateway,ImagePath="C:\Program Files\VMware\VMware View\Server\appblastgateway\nssm.exe",ServiceType=16,StartType=3,ErrorControl=0,,Dependencies=[~],,,Password=**********,Description=Provides VMware Horizon View Blast gateway services.,,)
InstallServices: Service:
Info 1923.Service VMware Horizon View Blast Secure Gateway (VMBlastSG) could not be installed. Verify that you have sufficient privileges to install system services.
Action ended 6:12:08: InstallFinalize. Return value 3.

Also noticed during the upgrade, it should uninstall the services from version 7.13.1, and new services would be created for version 8.6. In this case, they were listed as disabled.

Workaround

Later during the ongoing RCA investigation, a workaround provided was working well. The reason for socializing the workaround is that we spent a tremendous amount of time and effort in a revert operation, which was very time consuming and cumbersome. Only if we had known about this workaround during the failed upgrade would we have saved tremendous effort.

#ProTip – If you have two POD, make sure before you start the upgrade. Take a snapshot of all the PODS together at the same time. This could help you in scenarios where the POD1 upgrade fails, and you can revert the entire environment (POD1 & POD2) from snapshots.

Prerequisite / Rollback Plan 

  1. Power off all the Connection Servers part of Cloud POD Federation 
  2. Take a powered-off snapshot
  3. In case of an incident during the change activity which requires recovering Horizon Environment, Snapshots will be used as a fallback plan as the last option if break/fix, troubleshooting steps performed is not resolving the issue. 
  4. In troubleshooting scenarios of a failed upgrade 

Workaround Steps 

  1. The following steps need to be performed on each Connection Server at a time 
  2. Uninstall HTML and Horizon Connection Server 7.13.1 component keeping ADLDS and ADLDSG Instances intact. Check services.msc, and Horizon Services will appear in the Disabled state 
  3. Perform a reboot of the Connection Server, and Horizon Services will be cleared from services.msc
  4. Install Horizon 8.6 as Standard, which will pick the residing ADLDS and ADLDSG instance
  5. After successful install, it can be verified using ldp utility as instructed in KB https://kb.vmware.com/s/article/2064157 and look for the fields whenChanged , and whenCreated . This step can be performed prior upgrade for comparing the state of ADLDS

The VMware GSS case handled by Jezill Asharaf (A very helpful support engineer) and a few of the backend engineering team has been instrumental. I hope you will find this information useful if you encounter the issue and it should help you save time. If you manage to tweak or improvise further on this solution, please don’t forget to keep me posted.

Thanks,
Aresh Sarkari

VMware EUC stack upgrade – Legacy? or Modernizing? or Middleground?

14 Sep

It was that time of the year to perform a VMware End-User Computing (EUC) stack upgrade on the environment, and I thought of sharing the overall thought process and decisions made along the way. It will be interesting to share with others who might be in a similar situation or process of developing their upgrade/migration strategies. In this post, we shall take a look into these topics:

  • Current versions of the VMware EUC Stack
  • What version numbers did we upgrade/migrated to?
  • Why did we migrate to these versions?
  • Wishlist (Someone Listening?)
  • Valuable links to reference during upgrade/migration

Current versions of the VMware EUC Stack

  • VMware Horizon 7.11 (Connection Server/Agents)
  • VMware Horizon Client 5.5
  • VMware App Volumes 2.18.1.x Manager/VMware App Volumes 2.18.5 Agent version
  • VMware Workspace ONE Access 20.01/Connector 1903 (Not in scope for the upgrade)
  • VMware Dynamic Environment Manager 9.10
  • VMware Unified Access Gateway 3.10

What versions numbers did we upgrade/migrated to?

  • VMware Horizon 7.13.1 (Connection Server/Agents)
  • VMware Horizon Client 5.5.2
  • VMware App Volumes 2.18.10.10 (Manager/Agents)
  • VMware Dynamic Environment Manager 2103
  • VMware Unified Access Gateway 2103.1

Why did we migrate to these versions?

The obvious question everyone might ask is the latest versions are Horizon 8.x and App Volumes 4.x why are you picking older versions for the upgrade? The short answer is the limitations and trade-off, and the following matrix tries to uncover in more detail.

Note – Not all customers might fall under the limitation category, or the limiting feature/functionality could be different in your case. By no means this should be your defacto reasons. Make sure to evaluate your situation and create a matrix to make a data-driven decision. If the project is greenfield/no-limitations applied, it’s a no-brainer to opt for the latest product releases.

ProductUpgrade DecisionVerison of Choice
VMware Horizon++ We had all the boxes ticked from a feature/functionality standpoint to be able to upgrade/migrate to Horizon 8.x version. (Instant Clones, Printing, UAG etc.). Infact everything worked well in the development environment
++ The latest vROPS Horizon Adapter 1.2/Horizon 8.x version doesn’t include the built-in Horizon reports. We use the reporting feature for all sorts of custom reporting on Horizon PODs. The older version of vROPS Horizon Adapter 6.7.1/Horizon 7.x has all the existing metrics and reporting available but doesn’t include support for Horizon 8.x on the support matrix
++ The no reporting on the Horizon Adapter 1.2 + limited metrics on RDSH limited our ability to move to the latest version of Horizon 8.x. Once the built-in reports\metrics and guidance is made available, we shall jump onto the latest version (n-1)
Horizon 7.13.1
VMware App Volumes++ Lack of Writable Volumes (UIA+Profile and UIA) migrations from 2.18.x to 4.x. Need official guidance or tool/script/guidance to upgrade all the wrtiable of the 2.18.x environment to 4.x. I am sure alot of enterprise customers will have plenty of Writable Volumes to migrate and don’t have the flexibility to start from scratch on a new version
++ VMware AppStack Migration fling is the perfect migration utility to migrate AppStacks 2.18.x to 4.x need something similar for Writable Volumes
App Volumes 2.18.10.10
VMware Dynamic Environment Manager++ This was the only piece of software that didn’t have interoperability or upgrade complexity. The obvious choice was to upgrade to the latest (n-1)DEM 2103
VMware Unified Access Gateway++ The appliance has no interoperability issues with Horizon 7.13.1 or upgrade complexity. The obvious choice was to upgrade to the latest (n-1)UAG 2103.1
Upgrade Decision Matrix

The above stack provides us with the required General Availability support until Q2 FY2022 and beyond.

Wishlist

I am looking forward to vROPS Horizon Adapter XX to include the built-in Horizon Reports/Additional Metrics for RDSH in the new version or provide detailed guidance on creating meaningful reports in future releases. Additionally, the App Volumes team releases tools and advice on migrating 4000’s+ Writable Volumes from 2.18.x to 4.x. Once the above is released, I plan to upgrade to the branch of Horizon 8.x and App Volumes 4.x releases version numbers.

Valuable links to reference during upgrades

Here is the cheat sheet for all the useful links to review and formulate an upgrade plan:

DescriptionLinks
VMware Product Interoperability MatrixProduct Interoperability Matrix (vmware.com)
Product DocumentationVMware Horizon Documentation
VMware App Volumes Documentation
VMware Dynamic Environment Manager (Formerly Known as VMware User Environment Manager) Documentation
Techzone Migrating Legacy Horizon Components to Modern Alternatives

View Composer –> Instant Clones
Security Server –> UAG
Persona –> DEM
Persistent Disk – FSLogix
Modernizing VDI for a New Horizon | VMware
App Volumes Upgrade considerationsVMware App Volumes 4 Installation and Upgrade Considerations | VMware
Fling Migrate App Volumes AppStack from 2.18.x to 4.xApp Volumes Migration Utility | VMware Flings
Supported Windows 10 versions based on Horizon AgentSupported versions of Windows 10 on Horizon Agent Including All VDI Clones (Full Clones, Instant Clones, and Linked Clones on Horizon 7) (2149393) (vmware.com)
VMware EUC Stack Agent OrderAgent installation order for Horizon View, Dynamic Environment Manager, and App Volumes (2118048) (vmware.com)
Supported Windows 10 versions based on App Volumes AgentVMware App Volumes and Microsoft Windows 10 Support
VMware Product Lifecycle – End of LifeProduct Lifecycle Matrix (vmware.com)
Reference Material

I hope you will find the above information useful in your enterprise upgrade/migrate strategy for VMware EUC Stack. I would love to hear your strategy and similar situations limiting your ability to migrate to the latest and greatest versions.

Thanks,
Aresh Sarkari