Tag Archives: VMware

Upgrade VMware Horizon – An alternate method

3 Mar

In my previous blog post Upgrade from VMware Horizon 7.13.1 to 8.6 (2206) fails with VMware Horizon View Blast Secure Gateway (VMBlastSG) could not be installed. In this post, I will go into the details on the manual uninstall process and installation of the latest version of the VMware Horizon.

Why inplace upgrade fails?

The core reason for the in-place upgrade failing was that the VMware Service around Horizon was not getting deleted during the uninstall performed by the setup, which was rolling back the change.

What is the solution?

The quick solution is to uninstall the VMware Horizon 7 HTML Access, followed by VMware Horizon 7 Connection Server. Perform a reboot on the broker virtual machine, and this step will delete the services from about that were not getting deleted automatically. Install the VMware Horizon 8.x setup, and the installation will go smoothly.

What is the alternate method detailed steps?

The in-place upgrade method described here – Upgrade Connection Servers in a Replicated Group works 99.99% of the time in corner cases like mine. You will have to perform this method. Ensure you follow the basics of a full backup of the brokers, ADAM database, SQL database, backup of the locked.properties file and Disable vCenter provisioning.

Don’t perform this method without seeking proper VMware Support guidance. If you run into issues, you will be in unsupported territory and might ask yourself how you ended up here.

Uninstall existing VMware Horizon

  • Login to the broker you are going to perform the upgrade and open the Programs and Features
  • First, uninstall the VMware Horizon 7 HTML Access
  • Second, uninstall the VMware Horizon 7 Connection Server
  • You will be left with the AD LDS Instances (Local and CloudPod partitions). Make sure you leave them as-is.

Reboot

To get rid of the ghosted services perform an reboot of the broker VM.

Install the latest release 8.x

  • Login to the broker you are going to perform the upgrade
  • Validate whether the above services are deleted
  • Right click and run as administrators on the Connection server.exe of the 8.x setup
  • Make sure you select – Standard Server and click on HTML access and IPv4
  • It will detect the instance of the Horizon and you need to click on OK
  • Select Configure the Firewall within Windows
  • The Horizon 8.x will be installed successfully on the virtual machine
  • Repeat this steps on the other brokers within your POD

Validations

You must wait approximately 5-7 mins for the Horizon Administrator console to come online. Validate the Health dashboard for any errors and check the desired 8.x version is present.

I hope you will find this information useful if you encounter the issue and it should help you save time. If you manage to tweak or improvise further on this solution, please don’t forget to keep me posted.

Thanks,
Aresh Sarkari

Tags: , , , , , ,

Upgrade from VMware Horizon 7.13.1 to 8.6 (2206) fails with VMware Horizon View Blast Secure Gateway (VMBlastSG) could not be installed

22 Dec

Many customers are already in the process of upgrading from VMware Horizon 7.x to 8.x or will soon upgrade as the End Of Life dates are upcoming in April 2023. I want to share a rare experience wherein the Horizon upgrade from 7.13.1 to 8.6 version failed. In the rare occasion where in upgrade fails in the below mentioned manner, the workaround steps will come in handy.

We have only received the workaround from VMware support, and I intend to update the post once I get a complete RCA. At least the workaround can help someone not have to revert the entire environment instead, follow the workaround and avoid a lot of rework.

Issue

During the upgrade of the first connection server in the POD1, we encountered the following error five mins into the upgrade. Note before starting the upgrade, the entire health dashboard for the POD was green and included backup and snapshots.

Environment Overview

Let take a look at the environment details to provide an high-level overview:
Active Site (POD1)

  • 5 VMware Horizon Connection Servers 7.13.1
  • SQL Database on Microsoft SQL 2016 Always-on – EventsDB
  • The 5 Brokers are behind an NSX Load balancer

Active Site (POD2)

  • 5 VMware Horizon Connection Servers 7.13.1
  • SQL Database on Microsoft SQL 2016 Always-on – EventsDB
  • The 5 Brokers are behind an NSX Load balancer

Observations

The logs from the installer had the following message it’s complaining about insufficient privileges, which the installer is run with admin privileges already.

MSI (s) (A0:E0) [06:12:07:977]: Executing op: ActionStart(Name=InstallServices,Description=Installing new services,Template=Service: [2])
Action 6:12:07: InstallServices. Installing new services
MSI (s) (A0:E0) [06:12:07:977]: Executing op: ProgressTotal(Total=9,Type=1,ByteEquivalent=1300000)
MSI (s) (A0:E0) [06:12:07:977]: Executing op: ServiceInstall(Name=PCOIPSG,DisplayName=VMware Horizon View PCoIP Secure Gateway,ImagePath="C:\Program Files\VMware\VMware View\Server\bin\SecurityGateway.exe",ServiceType=16,StartType=3,ErrorControl=1,,Dependencies=[~],,,Password=**********,Description=Provides VMware Horizon View PCoIP gateway services.,,)
InstallServices: Service:
MSI (s) (A0:E0) [06:12:08:228]: Executing op: ServiceInstall(Name=VMBlastSG,DisplayName=VMware Horizon View Blast Secure Gateway,ImagePath="C:\Program Files\VMware\VMware View\Server\appblastgateway\nssm.exe",ServiceType=16,StartType=3,ErrorControl=0,,Dependencies=[~],,,Password=**********,Description=Provides VMware Horizon View Blast gateway services.,,)
InstallServices: Service:
Info 1923.Service VMware Horizon View Blast Secure Gateway (VMBlastSG) could not be installed. Verify that you have sufficient privileges to install system services.
Action ended 6:12:08: InstallFinalize. Return value 3.

Also noticed during the upgrade, it should uninstall the services from version 7.13.1, and new services would be created for version 8.6. In this case, they were listed as disabled.

Workaround

Later during the ongoing RCA investigation, a workaround provided was working well. The reason for socializing the workaround is that we spent a tremendous amount of time and effort in a revert operation, which was very time consuming and cumbersome. Only if we had known about this workaround during the failed upgrade would we have saved tremendous effort.

#ProTip – If you have two POD, make sure before you start the upgrade. Take a snapshot of all the PODS together at the same time. This could help you in scenarios where the POD1 upgrade fails, and you can revert the entire environment (POD1 & POD2) from snapshots.

Prerequisite / Rollback Plan 

  1. Power off all the Connection Servers part of Cloud POD Federation 
  2. Take a powered-off snapshot
  3. In case of an incident during the change activity which requires recovering Horizon Environment, Snapshots will be used as a fallback plan as the last option if break/fix, troubleshooting steps performed is not resolving the issue. 
  4. In troubleshooting scenarios of a failed upgrade 

Workaround Steps 

  1. The following steps need to be performed on each Connection Server at a time 
  2. Uninstall HTML and Horizon Connection Server 7.13.1 component keeping ADLDS and ADLDSG Instances intact. Check services.msc, and Horizon Services will appear in the Disabled state 
  3. Perform a reboot of the Connection Server, and Horizon Services will be cleared from services.msc
  4. Install Horizon 8.6 as Standard, which will pick the residing ADLDS and ADLDSG instance
  5. After successful install, it can be verified using ldp utility as instructed in KB https://kb.vmware.com/s/article/2064157 and look for the fields whenChanged , and whenCreated . This step can be performed prior upgrade for comparing the state of ADLDS

The VMware GSS case handled by Jezill Asharaf (A very helpful support engineer) and a few of the backend engineering team has been instrumental. I hope you will find this information useful if you encounter the issue and it should help you save time. If you manage to tweak or improvise further on this solution, please don’t forget to keep me posted.

Thanks,
Aresh Sarkari

Tags: , , , , , ,

Install VMware Horizon Client using Winget

14 Dec

The enterprise has been rolling out the application packages using various available methods (GPOs, SCCM, WS1 UEM etc.) in the industry. Today we are going to take a step further and see how to deploy the VMware Horizon Client using the new Micrososft Windows Package Manager (Winget)

Available Commands for various verison of Horizon Client

Following are the commands however, it’s recommended only to install the latest or the matching version based on your VMware Horizon environment.

VMware Horizon 8.xVMware Horizon Client 8.x
VMware Horizon 7.xVMware Horizon Client 5.x
# Latest GA version VMware Horizon Client version 8.7.0.31805
winget install -e --id VMware.HorizonClient
# VMware Horizon Client version 8.6.0.29364
winget install -e --id VMware.HorizonClient -v 8.6.0.29364
# VMware Horizon Client version 8.5.0.26981
winget install -e --id VMware.HorizonClient -v 8.5.0.26981
# VMware Horizon Client version 8.4.1.26410
winget install -e --id VMware.HorizonClient -v 8.4.1.26410
# VMware Horizon Client version 8.3.0.21227
winget install -e --id VMware.HorizonClient -v 8.3.0.21227
# VMware Horizon Client version 8.2.0.18176
winget install -e --id VMware.HorizonClient -v 8.2.0.18176
# VMware Horizon Client version 8.1.0.15949
winget install -e --id VMware.HorizonClient -v 8.1.0.15949
# VMware Horizon Client version 8.0.0.13243
winget install -e --id VMware.HorizonClient -v 8.0.0.13243
# VMware Horizon Client version 5.5.4.26353
winget install -e --id VMware.HorizonClient -v 5.5.4.26353
# VMware Horizon Client version 5.5.3.24986
winget install -e --id VMware.HorizonClient -v 5.5.3.24986
# VMware Horizon Client version 5.5.2.19788
winget install -e --id VMware.HorizonClient -v 5.5.2.19788
# VMware Horizon Client version 5.5.1.17068
winget install -e --id VMware.HorizonClient -v 5.5.1.17068
# VMware Horizon Client version 5.5.0.14558
winget install -e --id VMware.HorizonClient -v 5.5.0.14558

Installing, Listing, Upgrading and Un-installing the latest version HZ Client

Open the PowerShell with administrative privileges

Installing

winget install -e --id VMware.HorizonClient

Listing the installed package

List the package and its details of the previous installation step

winget list --name 'VMware Horizon Client'

Upgrading from version 5.5.4 to 8.7.0

winget upgrade --id VMware.HorizonClient

Un-installing

Following is the command to uninstall the Horizon Client. 

winget uninstall --id VMware.HorizonClient

Note after running the above command, the Windows endpoint rebooted immediately. I am not whether the product team has included the /norestart switches to the packages. If you come across the same leave a comment down below.

I hope you will find this helpful post about the winget and VMware Horizon Client details. Give it a spin in your lab and production environment, if you find anything interesting. I hope you can share it back with me?

Thanks,
Aresh Sarkari

Tags: , , , , , , ,

Making Microsoft Quick Assist work with VMware App Volumes – Writable Volumes

6 Sep

Microsoft Quick Assist is a tool widely used by service desk teams to take remote control of a computer and help end-users with a screen share. We noticed with the recent changes Microsoft made to Quick Assist, it stopped working when VMware App Volumes – Writable Volumes were present.

Error launching Quick Assist

We tried many steps of installing the Quick Assist offline using various Powershell commands, and nothing worked out. We also got to a point Quick Assist will launch without the Writable Volumes present or local admins with Writable Volumes. In the end, VMware Support GSS, with help from backend engineering, provided a working solution.

Solution (Workaround)

Step 1 – Download the offline version of the Quick Assist App from the Microsoft store and place all the files into C:\Temp\QuickAssist. Following are the steps to download the offline version of MS store apps.

Offline Files MS Quick Assist

Step 2 – Delete the old version of the Quick Assist App (Not part of the MS Store). Start Menu > Settings > Apps > Optional features > Microsoft Quick Assist > Uninstall. 

Uninstall Optional Features – Microsoft Quick Assist

Step 3 – Install the Offline version of the Microsoft Quick Assist package using PowerShell

PowerShell Add-AppxProvisionedPackage -PackagePath C:\Install\MicrosoftCorporationII.QuickAssist_2022.614.2314.0_neutral___8wekyb3d8bbwe.AppxBundle -online -SkipLicense
PowerShell Install Appx Package

Step 4 – Install WebView2 component which is a pre-requisite for Quick Assist. Note this step is quite important and not something that has been emphazied alot.

  reg add HKLM\SOFTWARE\Policies\Microsoft\EdgeUpdate /v InstallDefault /t REG_DWORD /d 1 /f
  C:\Install\MicrosoftEdgeWebView2RuntimeInstallerX64.exe
  reg add HKLM\SOFTWARE\Policies\Microsoft\EdgeUpdate /v InstallDefault /t REG_DWORD /d 0 /f
Install Microsoft Edge WebView

Step 5 – Launch the Quick Assist App once. Make sure, without launching the app, don’t move to the next step. Please wait at least 10 mins on this step or/else reboot the template VM and launch the app

explorer shell:AppsFolder\MicrosoftCorporationII.QuickAssist_8wekyb3d8bbwe!app
Launching Quick Assist

Step 6 – Create an OS scheduled task to run at every logon. This will launch the Quick Assist and register at every logon.

schtasks /create /RU "SYSTEM" /TN RegisterQuickAssist /SC ONLOGON /TR "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe Add-AppxProvisionedPackage -PackagePath C:\Temp\QuickAssist\MicrosoftCorporationII.QuickAssist_2022.614.2314.0_neutral___8wekyb3d8bbwe.AppxBundle -online -SkipLicense"
Schedule Task – Register of Microsoft Quick Assist at User Login

Step 7 – Take a snapshot of the golden image and the remaning steps will be performed in App Volumes files.

Step 8 – We need to prepare the template bundle of Writable Volumes (UIA+Profile) to contain a file startup_postsvc.bat with the following contents. Note Alter the file path if you decide to use something else.

  @echo off
  setlocal enabledelayedexpansion
  set WV_Path=none
  for /F "tokens=3" %%A in ('reg query HKLM\System\ControlSet001\Services\svservice\Parameters /v WritableVolume 2^>nul') do (
    set WV_Path=C:%%A
    fltmc attach bindflt !WV_PATH!
  )
  reg query "HKLM\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\Microsoft EdgeWebView" /v DisplayName >nul 2>&1
  if %ERRORLEVEL% NEQ 0 (
    reg add HKLM\SOFTWARE\Policies\Microsoft\EdgeUpdate /v InstallDefault /t REG_DWORD /d 1 /f
    start /wait C:\Temp\QuickAssist\MicrosoftEdgeWebView2RuntimeInstallerX64.exe
    reg add HKLM\SOFTWARE\Policies\Microsoft\EdgeUpdate /v InstallDefault /t REG_DWORD /d 0 /f
  )

Here is the procedure to update the writable volume’s file – Update Writable Volumes (vmware.com). Zip all the files and push the changes using AV Manager > Writables > Update Writable Volumes

Additional Files for writable volumes startup

Step 9 – Validation login to the virtual desktop (Make sure the updated file – startup_postsvc.bat are pushed onto the WV). After a few seconds, the Microsoft Edge WebView2 Runtime component and Quick Assist should appear in Programs and Features.

PowerShell Get-AppxPackage *assist*

Step 10 – Launch Quick Assist via Start Menu or Explorer and Voila!

explorer shell:AppsFolder\MicrosoftCorporationII.QuickAssist_8wekyb3d8bbwe!app

If you encounter a similar issue, you can follow the above solution. I hope you will find this information useful if you encounter the same issue. A big thanks to my teammate Jishan T for his continuous effort while troubleshooting with GSS over 3+ months.

Thanks,
Aresh Sarkari

Tags: , , , , , , , , ,

Forward specific logs from VMware vRealize Log Insight (vRLI) to Splunk

26 Aug

If you are not using SIEM (Security Information & Event Management) solution within your environment, you should seriously consider one. Considering the modern cyber security threat landscape, it a handy tool for all teams.

I had a bunch of VMware Workspace ONE Access (WS1) appliances already performing the Syslog action within vRealize Log Insight. However, the partner team was using a different solution Splunk. The objective here was to forward a specific log Greenbox_web.log (It holds all the user interface interactions for WS1 – This is your main log to see all internet facing activities on the appliance) to Splunk.

Luckily the Log Forwarding capability already exists within the vRLI. However, the creation of filters was a bit time consuming as it will convert the input into regex.

Configure the log forwarding in vRLI to Splunk

Go to your vRLI instance and click on Administration –> Log Management –> Log Forwarding and select New Destination

vRLI Log Management
vRLI Log Management

Configuration Details

  • Name – The Log Forwarder Destination freindly name – VDI-WS1-Logs-Splunk
  • Host – Enter the Splunk load balancing VIP address
  • Protocol – RAW
  • Transport – TCP
  • Filter
    • Hostname – starts with – WS1ManagerAppPrimary* WS1ManagerAppSecondary*
    • text – matches – *GreenBox* (Note within the log its <GreenBox> however, if you put in the greater/less than symbol, the conversion of this string into regex doesn’t work within vRLI.)
    • Please Run in the interactive Analytics query to confirm your filters are working as expected
  • Enter the custom port provided to you by the Splunk team
  • Click on Save
Destination Details
Destination Details

After a while. you will start seeing the events forwarded to Splunk, and the state will be marked as Active. You can use the same logic above to forward other specific logs to 3rd party tools (Doesn’t have to be Splunk only). I hope you will find this helpful information on your SIEM journey. Please let me know if I have missed any steps, and I will be happy to update the post.

Thanks,
Aresh Sarkari

Tags: , , , ,

VMware App Volumes – Tales of the missing Writable Volumes backup

15 Aug

You have a large VMware App Volumes environment and have backed up your writable volumes using the capabilities provided in the App Volumes Manager. (You are doing the right thing!)

AV Manager – WV Backup Config

We decided to perform an audit on the backup of the writable volumes within the App Volumes Manager 2.18.10 and the VSAN Datastore. You can export all the writable volumes to a CSV using the API. My script here will provide you with a complete outlook for conducting your analysis. Now exclude your group entitlements from the list, leaving you with the total number of writable volumes within your environment. Ideally, you are after the same number of writable volumes on the VSAN datastore. (Of course, if everything is going well in the backup world!)

In my case, we observed more than 300+ missing writable volumes between the exported CSV and the VSAN datastore. Let the investigations begin – within the production.log, we could see the backup was happening, but the challenge of a large environment is impossible to track all the backup occurring just by looking at the logs. Feature request to VMW – A dedicated backup log showcases the entire environment’s status. We eventually ended up with a GSS case after few months of back and forth and the logs exchange, we finally got a working solution. This closed the mystery of the missing backup of the writable volumes.

Solution

Go the the SQL database of the App Volumes Manager. Select the DB and New Query.

AV Database – Microsoft SQL

Enter the following query and hit execute. Now this will change the default writable volumes batch size(writables_backup_batch_size) from 5 to 25. Note the value of the batch size was tweaked multiple times, we first went with 10, which drastically reduced the missing backup. However, a few were still missing and not getting backup. The final number for our environment was 25 got all the writable volumes backup.

INSERT INTO dbo.settings ("key", value, created_at, updated_at) 
VALUES ('writables_backup_batch_size', 25, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)

Disclaimer – This tweak was required for a large App Volumes environment. Please consult with VMware Support before making any changes to your setup or Database. If it works for me doesn’t mean it will work for you. The value can differ based on the size of the enivronment.

I hope you will find this helpful information on your VMware App Volumes backup strategy. Please let me know if you have observed any issues like these, and would like to share your story?

Thanks,
Aresh Sarkari

Tags: , , , , ,

Mindmap – Part 2 – Horizon Cloud on Microsoft Azure (HCoA) – Configuration of Images – Desktops – Farms – Assignments

13 Apr

This post is in continuation of my part 1 – Mindmap – Part 1 – Horizon Cloud on Microsoft Azure (HCoA) – Quick start guide where we look at pre-requisites and the initial deployment of the HCoA solution. In this post, I want to share my learnings about the configuration of Images, Virtual Desktops, Farms and Assignments. We shall take a look into the following topics:

  • Mind map for Horizon Cloud on Microsoft Azure – Part 2 – Configuration of Images – Desktops – Farms – Assignments
    • Creating a Virtual Desktop or RDSH Image
      • Import VM
      • Create Image (Converting VM to Image)
    • Farms (Published Applications)
      • Create Desktop Farm
    • Add Applications to the Farms
      • New Applications – Auto-Scan from Farm
    • Create an Application Assignment
    • Create an Assignment for Multi-session or Hosted Shared Desktop
    • Create a Virtual Desktop Assignment (Persistent – Full Clone)
    • Create a Virtual Desktop Assignment (Non-Persistent – Floating)
    • AppStacks

In the second part of this series, the mindmap acts as an visual representation of all the configurations to be performed post the initial deployment of the Horizon Cloud Pod. It also helps during customer discussions and allows everyone to be on the same page. You can figure out in advance the pre-requisites, deployment details, and requirements for performing the next steps in your HCoA journey.

HCoA – Part 2

Disclaimer – This guide is a deployment/configuration guide, and the production settings, configuration, and use-cases might be different. Please make sure you change the settings appropriate for production workloads. Here is the PDF version if you would like to download and zoom in (Don’t stress your eyes!) –

Horizon-Cloud-on-Azure-Part-2-Twitter-@askareshDownload

Screenshots from my deployment

Horizon Cloud POD Managers + Unified Access Gateways

Note everything is deployed keeping in mind High availability.

  • 2 x Horizon Cloud Pod Managers
  • 2 x External Unified Access Gateways (Public IP)
  • 2 x Internal Unified Access Gateways (Internal on-premise network)
Azure – Virtual Machines

Azure Load Balancers

  • 1 x Horizon Cloud Pod Managers
  • 1 x Public UAG Appliances
  • 1 x Internal UAG Appliance
Azure – Load Balancers

Azure Virtual Network

I have created the vNet as part of pre-requisites in Part 1 series

  • 1 x Subnet for DMZ (Unified Access Gateway)
  • 1 x Subnet for Mgmt (Pod Managers)
  • 1 x Subnet for Workload (Desktop/Farms)
Azure – vNet

Azure Resource Groups

Note these are auto-created during the Pod deployment.

Azure – Resource Groups

I hope you will find this helpful information on your HCoA journey. Please let me know if I have missed any steps in the mindmap, and I will be happy to update the post.

Thanks,
Aresh Sarkari

Tags: , , , , , , , ,

Mindmap – Part 1 – Horizon Cloud on Microsoft Azure (HCoA) – Quick start guide

7 Apr

This will be a two part blog series on VMware Horizon Cloud on Microsoft Azure (HCoA). My aim is to get you started off the ground on HCoA, and I have a fair understanding of Azure due to my past certifications on AZ-140 and AZ-104(prep). I high recommend acquiring the Azure skills to make your life easier.

In part one, we shall take a look into the following topics:

  • Mind map for Horizon Cloud on Microsoft Azure – Part 1 – Getting started
    • Getting Started
      • Azure pre-requisites
      • Horizon Cloud Account
    • Configure the Azure Pod
      • Subscription
      • Pod Setup
      • Gateway Settings
    • General Setup
      • Domain Bind
      • Domain Join
      • Administrative Group
      • Universal Broker

The idea here is that the mindmap acts as an excellent visual representation of what to do during the end-2-end cycle of the project. It also helps during customer discussions and allows everyone to be on the same page. You can figure out in advance the pre-requisites, deployment, and requirements for the initial setup.

HCoA – Part 1

Disclaimer – This guide is a get you started guide, and the production settings, configuration and usecases might be different. Please make sure you change the settings appropriate for production workloads. Here is the PDF version if you would like to download and zoom in (Don’t stress your eyes!) –

Horizon-Cloud-on-Azure-Part-1-Twitter-@askareshDownload

The useful links to get you quickly started on Horizon Cloud on Azure:

DescriptionLinks
Horizon Cloud on Microsoft Azure Architecture (Techzone)Horizon Cloud on Microsoft Azure Architecture | VMware
Evaluation Guide for VMware Horizon Cloud Service on Microsoft Azure (Techzone)https://techzone.vmware.com/quick-start-tutorial-vmware-horizon-cloud-service-microsoft-azure
Horizon Cloud on Azure Evaluation YouTube playlist – Awesome stuff by Caroline Arakelian(14) VMware Horizon Cloud on Microsoft Azure: Deploying a Cloud Manager–Based Pod – YouTube
Our local Oz hero – Shane Fowler – 0 to hero on HCoAFrom Zero to Hero: A Step by Step Guide How To Deploy Horizon Cloud Service on Azure – YouTube
Horizon Cloud on Azure Cost CalculatorPathfinder (vmware.com)

I hope you will find this helpful information on your HCoA journey. Please let me know if I have missed any steps in the mindmap or reference links, and I will be happy to update the post.

Thanks,
Aresh Sarkari

Tags: , , , ,

Unable to use Privilege Elevation – VMware Dynamic Environment Manager

7 Mar

We were exploring the feature Privilege Elevation – VMware Dynamic Environment Manager (DEM) within our development environment, and for some odd reason, a specific feature and configuration wouldn’t work in our setup.

Disclaimer

The windows registry mentioned within this blog post is used within enterprise-grade secure environments. The hardening measure is part of CIS Benchmarks on Windows 10. If your machines aren’t hardened, the feature typically works out of the box. For example, in my home lab, I had no issues with the Privilege Elevation feature working.

Issue

Whenever we enable the feature and apply any settings, it will not work. It didn’t matter which configuration you picked. The error within the logs remains constant.

The error within the FlexEngine-ElevatedTasks.log

2022-02-21 13:02:30.122 [ERROR] Cannot launch elevated task 'TaskName01' (token infrastructure not available)
2022-02-22 11:22:02.960 [ERROR] Cannot launch elevated task 'TaskName01' (token infrastructure not available)
2022-02-28 18:23:19.736 [ERROR] Cannot launch elevated task 'TaskName01' (token infrastructure not available)

Cause

Provided by VMware – The additional configuration on LSA Protection causing issues with the VMware DEM agent (2103 Version). The windows registry key – HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa

RunAsPPL=1

Resolution

My team managed to open a VMware GSS case handled by GuruKripal (A very helpful support engineer), we had to provide numerous amount of logs, procmons and group policy export of the enivornment. After giving them the export of our CIS Benchmarked group policies, they could reproduce the issue. In the end, the VMware engineering team provided us with a newer build of DEM Agent (10.2.4.1023 x64.msi).

If you encounter a similar issue, you can raise a VMware support case to obtain the fix or/else, I was assured all future releases of DEM Agent would include the fix. I hope you will find this information useful if you encounter the issue. A big thanks to my teammate Jishan T for his continuous effort while troubleshooting with GSS over 6+ months.

Thanks,
Aresh Sarkari

Tags: , , , , ,

Azure VMware Solution – Network Connectivity Azure VNet and On-premise

15 Feb

In this blog post, we shall take a deeper look into the Azure VMware Solution network connectivity between the Azure VNet for accessing Azure native services such as Bastion, Azure AD, SQL etc. and further connectivity to the On-premise network to migrate virtual machines or hyrbid setup.

AVS Networking – Image courtesy @Microsoft

Step 1 & 2 – Connectivity between Azure VMware Solution (AVS) – Express Route to Azure VNet

  • After the deploying the AVS we need to connect it to the Azure VNet for consuming Azure Native Services such as Bastion, SQL, AAD etc.
    • Note AVS pre-deploys the ExpressRoute for you (AVS – Manage – Connectivity – Express Route).
  • We need to have a Virtual Network Gateway (VNG) existing on Azure VNet, or we need to create one. All steps to be performed under portal.azure.com
  • Deploy the Virtual Network Gateway (VNG) on Azure subscription
    • Make sure you have a VNG created on Azure VNET
    • Give it a name – AZ104-VNG01
    • Resource Group – Select New or existing
    • Location – Australia East
    • SKU – Standard (for demo and testing purposes)
    • Virtual Network – Select the existing VNET (E.g. 10.0.0.0/16) for Azure. Note it will create the Gateway Subnet automatically (10.x.x.x/24)
    • Type – ExpressRoute
    • Public IP Address – Create New (It will auto assign a public IP)
    • Optional Create Tags
    • Save and Create
  • Under AVS – Connectivity – Express Route
    • Request the Authorization key
      • Name – ToAzureVNET
      • Copy the Key and Express Route ID
  • Open the VNG (AZ104-VNG01) and Settings – Connections
    • Click on Add
    • Name – FromAVSPrivateCloud
    • Connection Type – Express Route
    • Enter the Authorization Key and Express Route ID and paste them here
    • Click OK
    • The Status will change from Updating to Succeeded
  • Now we have the connectivity between the AVS and Azure VNet.

Step 1 & 3 – Connectivity between Azure VMware Solution – ExpressRoute Global Reach to On-premise networks

  • Now we will establish the connectivity between AVS and On-premise networks
  • ExpressRoute Circuits – This is the coming from On-premise into Azure VNet
    • This will depend upon the partner network (Equinix, Telstra etc.)
    • Note there are different type of Peerings available. Select based on your design – Azure ExpressRoute Overview: Connect over a private connection | Microsoft Docs
      • Azure Private – We are going with this option at the moment
      • Azure Public (Public IP address required)
      • Microsoft (Office 365 etc.)
    • Click under Settings – Authorizations
    • Click on Add
    • Name – AuthorizationforAVS
    • Copy the Authorization Key
    • Copy the Resource ID, which is the Express Route Circuit ID
  • Under AVS – Connectivity – ExpressRoute Global Reach
    • Click on Add
    • Select the Subscription and Resource Group
    • Copy and paste the Authorization key and Express Route Circuit ID
    • Click Create
    • It will show as Connected

The intention here is to get you a few useful links on the Networking on Azure VMware Solution:

DescriptionLinks
AVS Network Setup – Video from Trevor DavisAzure VMware Solution – Network Setup – YouTube
VMware Documentation – Using Azure ExpressRoute with AVSDesignlet: Using Azure ExpressRoute with Azure VMware Solution for On-premises Connectivity | VMware
Azure ExpressRoute PricingPricing – ExpressRoute | Microsoft Azure
Previous Blog post – Mindmap AVS Networking guidanceMindmap – Azure VMware Solution – Guidance on Deployment and Networking | AskAresh
ExpressRoute Private PeetingGitHub – microsoft/Deploy-and-Optimize-Azure-ExpressRoute-Private-Peering: This deployment guide is focused on helping you deploy and optimize the Azure private peering, which enables connectivity between your private network and your Azure VNets over ExpressRoute.
Useful Links

I hope you will find this helpful information on your AVS Networking journey. Please let me know if I have missed any steps or good reference links, and I will be happy to update the post.

Thanks,
Aresh Sarkari

Tags: , , , , , ,

← Older Entries