We were exploring the feature Privilege Elevation – VMware Dynamic Environment Manager (DEM) within our development environment, and for some odd reason, a specific feature and configuration wouldn’t work in our setup.
Disclaimer
The windows registry mentioned within this blog post is used within enterprise-grade secure environments. The hardening measure is part of CIS Benchmarks on Windows 10. If your machines aren’t hardened, the feature typically works out of the box. For example, in my home lab, I had no issues with the Privilege Elevation feature working.
Issue
Whenever we enable the feature and apply any settings, it will not work. It didn’t matter which configuration you picked. The error within the logs remains constant.
Provided by VMware – The additional configuration on LSA Protection causing issues with the VMware DEM agent (2103 Version). The windows registry key – HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
RunAsPPL=1
Resolution
My team managed to open a VMware GSS case handled by GuruKripal (A very helpful support engineer), we had to provide numerous amount of logs, procmons and group policy export of the enivornment. After giving them the export of our CIS Benchmarked group policies, they could reproduce the issue. In the end, the VMware engineering team provided us with a newer build of DEM Agent (10.2.4.1023 x64.msi).
If you encounter a similar issue, you can raise a VMware support case to obtain the fix or/else, I was assured all future releases of DEM Agent would include the fix. I hope you will find this information useful if you encounter the issue. A big thanks to my teammate Jishan T for his continuous effort while troubleshooting with GSS over 6+ months.
In this blog post, we shall take a deeper look into the Azure VMware Solution network connectivity between the Azure VNet for accessing Azure native services such as Bastion, Azure AD, SQL etc. and further connectivity to the On-premise network to migrate virtual machines or hyrbid setup.
AVS Networking – Image courtesy @Microsoft
Step 1 & 2 – Connectivity between Azure VMware Solution (AVS) – Express Route to Azure VNet
After the deploying the AVS we need to connect it to the Azure VNet for consuming Azure Native Services such as Bastion, SQL, AAD etc.
Note AVS pre-deploys the ExpressRoute for you (AVS – Manage – Connectivity – Express Route).
We need to have a Virtual Network Gateway (VNG) existing on Azure VNet, or we need to create one. All steps to be performed under portal.azure.com
Deploy the Virtual Network Gateway (VNG) on Azure subscription
Make sure you have a VNG created on Azure VNET
Give it a name – AZ104-VNG01
Resource Group – Select New or existing
Location – Australia East
SKU – Standard (for demo and testing purposes)
Virtual Network – Select the existing VNET (E.g. 10.0.0.0/16) for Azure. Note it will create the Gateway Subnet automatically (10.x.x.x/24)
Type – ExpressRoute
Public IP Address – Create New (It will auto assign a public IP)
Optional Create Tags
Save and Create
Under AVS – Connectivity – Express Route
Request the Authorization key
Name – ToAzureVNET
Copy the Key and Express Route ID
Open the VNG (AZ104-VNG01) and Settings – Connections
Click on Add
Name – FromAVSPrivateCloud
Connection Type – Express Route
Enter the Authorization Key and Express Route ID and paste them here
Click OK
The Status will change from Updating to Succeeded
Now we have the connectivity between the AVS and Azure VNet.
Step 1 & 3 – Connectivity between Azure VMware Solution – ExpressRoute Global Reach to On-premise networks
Now we will establish the connectivity between AVS and On-premise networks
ExpressRoute Circuits – This is the coming from On-premise into Azure VNet
This will depend upon the partner network (Equinix, Telstra etc.)
I hope you will find this helpful information on your AVS Networking journey. Please let me know if I have missed any steps or good reference links, and I will be happy to update the post.
The MP4H (Management Pack for Horizon) 2.0 was recently released – Release Notes. It was time to give it a go, and the best place IMHO is VMware Test Drive, as they have a larger environment and there is alot of data for simulation compared to our small homelab.
Let’s take a look at the following information from the vRealize Operations – MP4H:
New engaging Dashboards and View
Observations and thoughts
Where to download the MP4H 2.0 pack?
New engaging Dashboards and Views
Service Monitoring for Unified Access Gateway (UAG) and Connection Server (CS) – Using the Telegraf Agent, you can monitor the Tomcat services and HTTP Health Checks against the UAG and CS web services. The step by step configuration details can be found here.
Connection Server certificate validation – There are new properties around CS certificate validation. This way, you can an dashboard based on the view – Availability \ Horizon Connection Server Certificate.
Unified Access Gateway Session – There are views on UAG Session Disparity amongst different UAG Appliances and session split across internal and external UAG appliances. The UAG Overall experience widget gives a high-level overview. Note the HTTP check is using the Telegraf agent to perform the checks (It’s empty, but you get the point there is capability.)
Horizon Client Versions and Horizon User Agent Version – These dashboards are not new and were present in the previous MP4H, but these are so important to get an overview of the Client and Agent versions within the environment.
Observations and thoughts
Very excited to see the UAG Appliances inclusion as a first class citizen into monitoring & alerting. One can tell looking at the MP4H 2.0 has come a long way since its previous releases. I am personally looking forward to the built-in reports to come back in future releases, along with the inclusion of VMware App Volumes and VMware Workspace ONE alerting and reporting. If the VMware product team considers my recommendation, it can start to differentiate itself against other players within the same market segment.
Features wishlist:
HTTP Check and Service monitoring for App Volumes and Workspace ONE Access
Certificate validation for UAG, App Volumes and Workspace ONE Access
Builtin Reports – Previous Horizon Reports, App Volumes – AppStacks, App Volumes – Wrtiable Volumes, Workspace ONE Access – User Sessions etc.
I can’t wait to see what the next release of the vRealize Operations – Horizon Management pack beholds.
Where to download the MP4H 2.0 pack?
You can download the pack from VMware Marketplace, and the following versions of vRealize Operations are supported.
Disclaimer – All the screenshots are from VMware Test Drive Portal. All credits to them for their hardwork.
I hope you will find this helpful post about the latest release of MP4H 2.0. I highly recommend giving it a go on Test Drive and a small request if you find anything interesting. I hope you can share it back with me?
It was an honour to come and speak at the podcast with host Eric L Nielsen and co-host Matt Langguth. A treat to watch and speak live, and catch-up on the signature Eric’s introduction of “Across the nation or around the world” and the “color of the bay”. The entire overall experience on the podcast was outstanding and I hope to come back again soon. Following is the Audio/Video version on Youtube the podcast is available on leading platform such as Apple, Google, Spotify etc.
I hope you find our conversation useful and feel free to revert if you have further questions.
Often within the VMware App Volumes Manager (AVM), Writable Volumes will show up as Status – Orphaned. Let’s take a look at the following topics:
What is Orphaned Writable Volumes?
Script to delete them from the App Volumes Managers
What is Orphaned Writable Volumes?
App Volumes Manager is integrated with Microsoft Active Directory (AD), and it’s in continuous synchronization. Whenever an end-user account gets disabled into the AD during the next sync activity of App Volumes Manager, it will mark the writable volumes with Writable Status = Orphaned.
Now in the ideal world, these accounts have been disabled and should be okay to delete? Maybe, if you don’t have the obligation of data retention, then you are ready to delete them. If you need to retain them, keep them as-is for compliance purposes.
Script to delete them for App Volumes Manager
Before we talk about the script, the deletion is very straightforward within the App Volumes Manager. Select the volumes with Status = Orphaned and select the Delete button. However, when you have to do the same against multiple POD, which becomes challenging, and as always, if it’s not automated, there is the scope of human error.
Pre-requisites
You need the App Volumes Manager URL
You need the username and password of the App Volumes Manager
You need to enter y/Y to proceed further with the deletion
The script was tested on PowerShell V5.x with App Volumes Manager version 2.18.10 (The logic will be the same however, the API call for App Volumes 4.x will be different)
###########################################################################
# Get List of Wrtiable Volumes from AppVolumes Manager for Status=Orphaned
# Delete the Orphaned Wrtiable Volumes
# You need username and password for the App Volumes Manager
# Author - Aresh Sarkari (Twitter - @askaresh)
# Version - V5.0
###########################################################################
#App Volumes Manager Name or IP Address
$AVManager = "https://avm001.askaresh.local"
# Run at the start of each script to import the credentials
$RESTAPIUser = "domain\username"
$RESTAPIPassword = "enteryourpassword"
#Ignore cert errors
add-type @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]'Ssl3,Tls,Tls11,Tls12'
#Login AV Manager Body
$body = @{
username = “$RESTAPIUser"
password = “$RESTAPIPassword”
}
#Login API call to the AV Manager
Invoke-RestMethod -SessionVariable DaLogin -Method Post -Uri "$AVManager/cv_api/sessions” -Body $body
#Get the list of Writbale Volumes from the AV Manager
$output = Invoke-RestMethod -WebSession $DaLogin -Method Get -Uri "$AVManager/cv_api/writables" -ContentType "application/json"
#Selecting the WV with status Orphaned into a variable
$WVouput = $output.datastores.writable_volumes | Select-Object id, owner_name, owner_upn, title, status | Where-Object {[string]$_.status -match "Orphaned"}
#Output on the console (Validate carefully before proceeding ahead)
$WVouput | Format-Table | Out-String | % {Write-Host $_}
#Confirmation logic to proceed with the deletion
$confirmation = Read-Host -Prompt "Are you Sure You Want To Proceed with the deletion:"
if ($confirmation -match "[yY]" ) {
# proceed
# The WV Deletion API call only looks for IDs. We are filtering the ids only
$WVOutputIDs = $WVouput.id
#Looping to delete each Writable Volumes via its ID
foreach ($WVOutputIDss in $WVOutputIDs) {
# Writable Volumes deletion Parameters body
$jsonbody = @{
bg = "0"
volumes = "$WVOutputIDss"
} | ConvertTo-Json
#API call to delete the Wrtiable Volumes
#We are using Invoke-webrequest for getting the Content of the deletion (Success) in oneline
$WVdeletecall = Invoke-WebRequest -WebSession $DaLogin -Method Post -Uri "$AVManager/cv_api/volumes/delete_writable" -Body $jsonbody -ContentType "application/json"
}
#Dig into the exception to get the Response details.
Write-Host $WVdeletecall.StatusCode
Write-Host $WVdeletecall.StatusDescription
Write-Host $WVdeletecall.Content
}
When you run the script, it will identify all the end-users with Status = Orphaned. If you like, you can copy and paste the output in an editior (Notepad++) to verify the output.
Once you press y/Y it will go ahead and delete the Orphaned writable volumes.
I hope you will find this script useful to bulk delete orphaned Writable Volumes in App Volumes Manager. A small request if you further enhance the script or make it more creative, I hope you can share it back with me?
Horizon Reach is a potent tool, and Andrew Morgan has put in a lot of blood, sweat and tears to develope it. What suprises me is why isnt this fling included into the Horizon product? We haven’t gathered here to talk about the product management and roadmap aspects 😉
Horizon Reach fling aggregates all the various Horizon POD information into its database. Typically, running Horizon API calls or Horizon Powershell modules might have to run them against individual pods to fetch information about that POD. The beauty with Horizon Reach is it aggregates all the information, we can write scripts/API calls to request information from there instead of writing Horizon POD specific scripts.
Let’s take a look at the following information from the Horizon Reach fling:
What API’s are available with Horizon Reach?
What all options are available to interact with Horizon Reach API?
Script – Get a consolidated list of Horizon Farm details (Display the Name, Base Image details, Snapshot Version, Health and If provisioning is enabled)
Note the above can also be fetched using the old Horizon Powershell modules but trust me it’s pretty tricky to run a foreach loop for every object on the SOAP method.
Script – Get a consolidated list of Horizon Desktop Pools details (Display the Name, Base Image details, Snapshot Version, Health and If provisioning is enabled)
What API’s are avilable with Horizon Reach?
After you have installed the Horizon Reach fling, go to the following URL to check out all the avilable API’s. Its the UI Swagger interface to simplify and understand each calls.
Scripts to get consolidated Horizon Farms/Desktops information
Pre-requsites:
You need the Horizon Reach Server URL
You need the password of the Horizon Reach Server
The script provides you with the details of all Horizon PODs in your setup.
The script was tested on PowerShell V5.x
#Horizon Reach Server Name or IP Address
$HZReachServer = "https://horizonreach.domain:9443"
#Ignore the self signed cert errors
add-type @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]'Ssl3,Tls,Tls11,Tls12'
#API Call to make the intial connection to the Horizon Reach Server##
$HZReachLogonAPIcall = "$HZReachServer`/api/Logon"
#The body payload that comprises of the login API request
$body = @{
username = "administrator"
password = "enteryourpassword"
} | ConvertTo-Json
$HZReachlogin = Invoke-RestMethod -Method Post -uri $HZReachLogonAPIcall -Body $body -ContentType "application/json"
#Header along with the JWT token is now saved for future API calls
#You need to call this header in all subsequent calls as it has the token
$Headers = @{ Authorization = "Bearer $($HZReachlogin.jwt)" }
#API Call to fetch the consolidated (as many pods you have) Horizon Farm information##
$HZReachFarms = Invoke-RestMethod -Method Get -uri "$HZReachServer/api/Farms" -Headers $Headers -ContentType "application/json" -UseBasicParsing | Format-Table -Property displayname, baseimage, snapshot, enabled, health, isProvisioningEnabled
Write-Output $HZReachFarms
#API Call to fetch the consolidated (as many pods you have) Horizon desktop pool information##
$HZReachPools = Invoke-RestMethod -Method Get -uri "$HZReachServer/api/pools" -Headers $Headers -ContentType "application/json" -UseBasicParsing | Format-Table -Property displayname, baseimage, snapshot, enabled, healthDetail, isProvisioningEnabled
Write-Output $HZReachPools
The following information (Display Name, Snapshot, Base Image, Health, Provisioning Mode) is pulled using the above scripts. I was much interested to see the snapshot versions of each Farms/Pools along with Health and provisioning status. Feel free to extract whatever details you are looking for there are plenty of other properties.
I hope you will find this script useful to fetch helpful information from Horizon Reach. A small request if you further enhance the script or make it more creative, I hope you can share it back with me?
It was patch Tuesday time, and we were implementing the Windows 10 1909 Oct October 12, 2021—KB5006667 (OS Build 18363.1854) patch to our base images which are used for VMware Horizon VDI. During our validations, we started noticing the strange behaviour of Explorer.exe crashing and desktop becoming completely unusable.
The Windows explorer.exe keeps crashing within the virtual desktop of Windows 10 1909. The virtual desktop is entirely unusable. The only way to see the Event Viewer or anything is by using Horizon Client – Options – Send Ctrl + Alt + Del command within the virtual desktop and then opening up the Task Manager.
Cause
Provided by Microsoft – The explorer is trying to update feeds content, and there is a NULL value due to this bug that is causing explorer to crash.
Resolution
We tried performing various steps of un-install and re-installing the patch etc.. However, nothing worked, and we ended up working with Microsoft and seemed like it was a known issue, and they provided us with the following fix:
I have been trying out the Azure VMware Solution (AVS) on the VMware HOL and going through the techzone documentation. In this post, we shall take a look into these topics:
Mindmap – Steps for AVS Deployment and Networking
Quick Start Links
Optional – Deploy AVS via Azure Resource Manager Templates
Mindmap for AVS Deployment/Networking
Managed to put together a mindmap on the AVS deployment and networking steps of the service. The idea here is the mindmap acts like an excellent visual representation of what to do during deployment and you can figure out in advance the requirements/steps and pre-requisites.
Azure VMware Solution
Here is the PDF version if you would like to download and zoom-in
Optional – Deploying AVS using Azure Resource Manager
We can also deploy the AVS solution via PowerShell and using the Azure Resource Manager (ARM) template. The advantage here is you have slightly more advanced options that are not available via the GUI. (E.g. You can set the desired password for vCenter and NSX)
I hope you will find this helpful information on your AVS journey. Please let me know if I have missed any steps in the mindmap or reference links and I will be happy to update the post.
Many blogs discuss and show in detail how to integrate the Azure Active Directory (AAD) with VMware Workspace ONE Access (WoA) as a 3rd party IDP, and the following are my top post on that topic:
However, in this blog post, I would like to shed more light on the SAML Authentication Flow between the Azure Active Directory (Identity Provider) and VMware Workspace ONE Access (Service Provider). When designing the WoA and AAD integration, the below flow helped me understand what is happening behind the scenes, and I thought of sharing my knowledge with you all.
SAML Authentication Flow
AuthnRequest
Issuer
NameIDPolicy
RequestAuthnContext
SAML Response that AAD sends to WoA
#ProTip – I use a Chrome/Edge extension called SAML-tracer to inspect the SAML responses back and forth within the browser.
SAML Authentication Flow
The diagram below describes the single sign-on sequence. The VMware Workspace ONE Access (the service provider) uses an HTTP Redirect binding to pass an AuthnRequest (authentication request) element to Azure AD (the 3rd party identity provider in case of WoA). Azure AD then uses an HTTP post binding to post a Response element to the cloud service.
SAML Authentication Flow – AAD and WoA
S. No.
DescriptionFlow
1.
End-user tries to access the VMware Workspace ONE Access portal
2.
VMware Workspace ONE Access finds the identity provider to authenticate the user
3.
VMware Workspace ONE Access generates a SAML 2.0 AuthnRequest and redirects the user’s browser to the Azure AD SAML single sign-on URL
4.
If the end-user is not signed in, Azure AD authenticates the user using multi-factor authentication & generates a SAML token
5.
Azure AD posts the SAML response to the WoA application via the user’s browser
6.
VMware Workspace ONE Access verifies the SAML Response
7.
VMware Workspace ONE Access completes the end-user sign-in and presents the desktop/app entitlements
Note – I have randomly created the GUID within the XML response just for demonstration purposes.
AuthnRequest
To request a end-user authentication, from WoA portal send an AuthnRequest element to Azure AD. Following is the SAML SAML 2.0 AuthnRequest from WoA portal:
The Issuer element in an AuthnRequest must exactly match one of the ServicePrincipalNames in the cloud service in Azure AD. Typically, this is set to the App ID URI that is specified during application registration. (When the Enterprise Application is created under AAD portal)
This element requests a particular name ID format in the response and is optional in AuthnRequest elements sent to Azure AD. A NameIdPolicy element looks like the following from WoA portal:
The RequestedAuthnContext element specifies the desired authentication methods. It is optional in AuthnRequest elements sent to Azure AD. Azure AD supports AuthnContextClassRef values snippet from WoA portal:
I hope you will find the above information helpful in your journey with AAD/WoA. A small request if you see any scope of improvisation or refinements. I hope you can share it back with me?
It was that time of the year to perform a VMware End-User Computing (EUC) stack upgrade on the environment, and I thought of sharing the overall thought process and decisions made along the way. It will be interesting to share with others who might be in a similar situation or process of developing their upgrade/migration strategies. In this post, we shall take a look into these topics:
Current versions of the VMware EUC Stack
What version numbers did we upgrade/migrated to?
Why did we migrate to these versions?
Wishlist (Someone Listening?)
Valuable links to reference during upgrade/migration
Current versions of the VMware EUC Stack
VMware Horizon 7.11 (Connection Server/Agents)
VMware Horizon Client 5.5
VMware App Volumes 2.18.1.x Manager/VMware App Volumes 2.18.5 Agent version
VMware Workspace ONE Access 20.01/Connector 1903 (Not in scope for the upgrade)
VMware Dynamic Environment Manager 9.10
VMware Unified Access Gateway 3.10
What versions numbers did we upgrade/migrated to?
VMware Horizon 7.13.1 (Connection Server/Agents)
VMware Horizon Client 5.5.2
VMware App Volumes 2.18.10.10 (Manager/Agents)
VMware Dynamic Environment Manager 2103
VMware Unified Access Gateway 2103.1
Why did we migrate to these versions?
The obvious question everyone might ask is the latest versions are Horizon 8.x and App Volumes 4.x why are you picking older versions for the upgrade? The short answer is the limitations and trade-off, and the following matrix tries to uncover in more detail.
Note – Not all customers might fall under the limitation category, or the limiting feature/functionality could be different in your case. By no means this should be your defacto reasons. Make sure to evaluate your situation and create a matrix to make a data-driven decision. If the project is greenfield/no-limitations applied, it’s a no-brainer to opt for the latest product releases.
++ We had all the boxes ticked from a feature/functionality standpoint to be able to upgrade/migrate to Horizon 8.x version. (Instant Clones, Printing, UAG etc.). Infact everything worked well in the development environment ++ The latest vROPS Horizon Adapter 1.2/Horizon 8.x version doesn’t include the built-in Horizon reports. We use the reporting feature for all sorts of custom reporting on Horizon PODs. The older version of vROPS Horizon Adapter 6.7.1/Horizon 7.x has all the existing metrics and reporting available but doesn’t include support for Horizon 8.x on the support matrix ++ The no reporting on the Horizon Adapter 1.2 + limited metrics on RDSH limited our ability to move to the latest version of Horizon 8.x. Once the built-in reports\metrics and guidance is made available, we shall jump onto the latest version (n-1)
++ Lack of Writable Volumes (UIA+Profile and UIA) migrations from 2.18.x to 4.x. Need official guidance or tool/script/guidance to upgrade all the wrtiable of the 2.18.x environment to 4.x. I am sure alot of enterprise customers will have plenty of Writable Volumes to migrate and don’t have the flexibility to start from scratch on a new version ++ VMware AppStack Migration fling is the perfect migration utility to migrate AppStacks 2.18.x to 4.x need something similar for Writable Volumes
++ The appliance has no interoperability issues with Horizon 7.13.1 or upgrade complexity. The obvious choice was to upgrade to the latest (n-1)
UAG 2103.1
Upgrade Decision Matrix
The above stack provides us with the required General Availability support until Q2 FY2022 and beyond.
Wishlist
I am looking forward to vROPS Horizon Adapter XX to include the built-in Horizon Reports/Additional Metrics for RDSH in the new version or provide detailed guidance on creating meaningful reports in future releases. Additionally, the App Volumes team releases tools and advice on migrating 4000’s+ Writable Volumes from 2.18.x to 4.x. Once the above is released, I plan to upgrade to the branch of Horizon 8.x and App Volumes 4.x releases version numbers.
Valuable links to reference during upgrades
Here is the cheat sheet for all the useful links to review and formulate an upgrade plan:
I hope you will find the above information useful in your enterprise upgrade/migrate strategy for VMware EUC Stack. I would love to hear your strategy and similar situations limiting your ability to migrate to the latest and greatest versions.
Recent Comments