Archive | August, 2022

Forward specific logs from VMware vRealize Log Insight (vRLI) to Splunk

26 Aug

If you are not using SIEM (Security Information & Event Management) solution within your environment, you should seriously consider one. Considering the modern cyber security threat landscape, it a handy tool for all teams.

I had a bunch of VMware Workspace ONE Access (WS1) appliances already performing the Syslog action within vRealize Log Insight. However, the partner team was using a different solution Splunk. The objective here was to forward a specific log Greenbox_web.log (It holds all the user interface interactions for WS1 – This is your main log to see all internet facing activities on the appliance) to Splunk.

Luckily the Log Forwarding capability already exists within the vRLI. However, the creation of filters was a bit time consuming as it will convert the input into regex.

Configure the log forwarding in vRLI to Splunk

Go to your vRLI instance and click on Administration –> Log Management –> Log Forwarding and select New Destination

vRLI Log Management
vRLI Log Management

Configuration Details

  • Name – The Log Forwarder Destination freindly name – VDI-WS1-Logs-Splunk
  • Host – Enter the Splunk load balancing VIP address
  • Protocol – RAW
  • Transport – TCP
  • Filter
    • Hostname – starts with – WS1ManagerAppPrimary* WS1ManagerAppSecondary*
    • text – matches – *GreenBox* (Note within the log its <GreenBox> however, if you put in the greater/less than symbol, the conversion of this string into regex doesn’t work within vRLI.)
    • Please Run in the interactive Analytics query to confirm your filters are working as expected
  • Enter the custom port provided to you by the Splunk team
  • Click on Save
Destination Details
Destination Details

After a while. you will start seeing the events forwarded to Splunk, and the state will be marked as Active. You can use the same logic above to forward other specific logs to 3rd party tools (Doesn’t have to be Splunk only). I hope you will find this helpful information on your SIEM journey. Please let me know if I have missed any steps, and I will be happy to update the post.

Thanks,
Aresh Sarkari

Tags: , , , ,

VMware App Volumes – Tales of the missing Writable Volumes backup

15 Aug

You have a large VMware App Volumes environment and have backed up your writable volumes using the capabilities provided in the App Volumes Manager. (You are doing the right thing!)

AV Manager – WV Backup Config

We decided to perform an audit on the backup of the writable volumes within the App Volumes Manager 2.18.10 and the VSAN Datastore. You can export all the writable volumes to a CSV using the API. My script here will provide you with a complete outlook for conducting your analysis. Now exclude your group entitlements from the list, leaving you with the total number of writable volumes within your environment. Ideally, you are after the same number of writable volumes on the VSAN datastore. (Of course, if everything is going well in the backup world!)

In my case, we observed more than 300+ missing writable volumes between the exported CSV and the VSAN datastore. Let the investigations begin – within the production.log, we could see the backup was happening, but the challenge of a large environment is impossible to track all the backup occurring just by looking at the logs. Feature request to VMW – A dedicated backup log showcases the entire environment’s status. We eventually ended up with a GSS case after few months of back and forth and the logs exchange, we finally got a working solution. This closed the mystery of the missing backup of the writable volumes.

Solution

Go the the SQL database of the App Volumes Manager. Select the DB and New Query.

AV Database – Microsoft SQL

Enter the following query and hit execute. Now this will change the default writable volumes batch size(writables_backup_batch_size) from 5 to 25. Note the value of the batch size was tweaked multiple times, we first went with 10, which drastically reduced the missing backup. However, a few were still missing and not getting backup. The final number for our environment was 25 got all the writable volumes backup.

INSERT INTO dbo.settings ("key", value, created_at, updated_at) 
VALUES ('writables_backup_batch_size', 25, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)

Disclaimer – This tweak was required for a large App Volumes environment. Please consult with VMware Support before making any changes to your setup or Database. If it works for me doesn’t mean it will work for you. The value can differ based on the size of the enivronment.

I hope you will find this helpful information on your VMware App Volumes backup strategy. Please let me know if you have observed any issues like these, and would like to share your story?

Thanks,
Aresh Sarkari

Tags: , , , , ,