Create a Memory Dump from a Suspended Virtual Machine – VMware vSAN

If you have a VMware VSAN environment and you wanted to capture a memory dump of the Virtual Machine for debugging or want to provide memory.dmp to VMware GSS or R&D for further analysis go ahead and read further!

Use Case – In our scenario had a few VDI Desktops running Windows 10 1607 + Horizon 7.3.1 + App Volumes Writable Volumes 2.13.1 + UEM 9.2.1 that were getting into unresponsive state. As a last resort we wanted to capture the memory dump to find out more what is causing the VM to get unresponsive.

Step by Step Instructions:

— Using the vCenter console select the Virtual Machine VM – Power – Suspend

— This will create the *.vmss and *.vmem file for Debugging. (Note the *.vmem file is applicable for ESXi 6.0 onwards)


— Make a note of the ESXi host Name/IP for the VM is in Suspend state

— SSH to the ESXi Host and browser to the VM Directory location:

— Now lets open the *.vmem file using “cat” command to retrieve the Object ID information. Make a note of the ObjectID

— In my scenario the Object ID was properly pre-created I didn’t have to use the objtool to find out the Object opened. However, in some cases you might have to run the following command

— Now using WINSCP login to the same ESXi Host and go the path:
Object ID – /vmfs/device/vsan/2c86055a-573b-d20a-5cdf-ecf4bbea1e48 (my scenario)
Or/else Object opened at path and download the file “2c86055a-573b-d20a-5cdf-ecf4bbea1e48” which is your ”*.vmem file and move the files to local or remote location that you are using the WINSCP tool.

— Rename the Object ID to a friendly name shown in the VM Directory Folder. I renamed it (od-av-trou-1-7622414e.vmem)

— For the *.vmss (od-av-trou-1-7622414e.vms) you can directly WINSCP to the ESXi Host and go to the location in the table and move the files to your local or remote location

— Once you have both the files *.vmem and *.vmss you can use a VMware Vmss2core Fling and convert it to a dump. Please make sure you meet the requirements and use the appropriate switches to your environment

— The above command will generate a memory.dmp file which can used in WINDBG for further analysis. If you are sending the dump file to someone make sure use *.zip and compress it before sending.

I hope you will find these steps useful and save a lot of time during daunting unresponsive VM issues. A big thanks to Frank EscarosBuechsel to helping with the entire procedure.

Thanks,
Aresh Sarkari