VMware Backup Set

Last modified: 2019/08/26 (Note: Content written for AhsayCBS v7+v8, and may generally apply to latest product release)

AhsayOBM allows you to back up individual Guest virtual machines (VMs) on your VMware hosts with our VMware Backup Set. VMware Guest VM backup module of AhsayOBM provides you with a set of tools to perform hot backup of your running VMs in VMware environment.

When you need to restore a VM, besides restoring the whole VM image, you can also utilize the built-in Run Direct feature to spin-up the VM in minutes without the need to restore the whole VM first. Moreover, our Granular Restore feature allows you to mount the virtual disks in a backed up VM and restore individual files within the VM without the need to spin-up the VM first.

Below is the system architecture diagram illustrating the major elements involved in the backup process among the VMware server, AhsayOBM and AhsayCBS. In this user guide, we will focus on the software installation, as well as the end-to-end backup and restore process using the AhsayOBM as a client backup software.

Run Direct is a feature that helps reduce disruption and downtime of your production VMs. Unlike normal VM restore procedure where a VM is extracted from backup files and copied to the production storage, which can take hours to complete. Restore with Run Direct can instantly power up a VM by running it directly from the backup files in the backup destination and the VM can be put into production.

When a Run Direct restore is performed, the backup destination is mounted as an NFS datastore from the VMware host, where the VM is run directly from the backup files.

The backup destination can either be the AhsayCBS server or a local drive that can connect with AhsayOBM. Initiating a Run Direct from the AhsayCBS (also known as agentless restore) will trigger a connection directly with the VMWare host (ESXi server and the direction shown in orange indicator below), while initiating the same action on the AhsayOBM requires the connection to route through the OBM (shown in green indication below).

The restored virtual machine, at this stage (e.g. before the restore is finalized) is in a read-only state to preserve its integrity. All changes made to the virtual disks (e.g. operation within the guest virtual machine) are stored separately in transaction logs stored on the NFS datastore or the original datastore, depending on the setting selected. These changes are discarded when Run Direct is stopped, where the restored VM will be removed and all changes will be discarded, or the changes will be consolidated with the original virtual machine data when the restore is finalized.

The restored virtual machine, at this stage (e.g. before the restore is finalized) is in a read-only state to preserve its integrity. All changes made to the virtual disks (e.g. operation within the guest virtual machine) are stored separately in transaction logs stored on the NFS datastore or the original datastore, depending on the setting selected. These changes are discarded when Run Direct is stopped, where the restored VM will be removed and all changes will be discarded, or the changes will be consolidated with the original virtual machine data when the restore is finalized.

To finalize recovery of a VM, you will still need to migrate it to a permanent location on the VMware host. The following steps are taken when you finalize a Run Direct restore:

A VMware snapshot is created for the VM

Backup files from the NFS datastore are copied to the production datastore on the VMware host.

Changes made to the VM after the snapshot creation are moved to the new location.

The VM is temporarily suspended to consolidate the changes made after the snapshot creation.

After all changes are consolidated, the VM is resumed.

The NFS datastore is dismounted.

There are two backup modes available for VM backup:

For VM on VMware host on Enterprise Standard, Enterprise and Enterprise Plus Edition, backup is performed in VDDK mode. Backup in VDDK mode produces a backup chain that consists of a full VDDK file and a set of VDDK incremental files.

• During the first backup, full files (*.F.vddk) are created in the backup destination.
• During subsequent backup, Changed Block Tracking (CBT) - a VMware native feature is employed, to identify disk sectors altered since the last backup. Altered blocks are saved as incremental VDDK file (*.I.vddk) in the backup chain.

During a subsequent backup in VDDK mode, AhsayOBM queries CBT through VADP (vSphere APIs for Data Protection) to request for transmission of all altered blocks since the last backup. As there is no need to stream the VM files to the Backup Client Computer for delta generation, backup in VDDK mode will greatly enhance the speed of subsequent backup.

For VM on VMware host on Enterprise Standard, Enterprise and Enterprise Plus Edition, backup is performed in VDDK mode. Backup in VDDK mode produces a backup chain that consists of a full VDDK file and a set of VDDK incremental files.

• During the first backup, full files (e.g. virtual disk file (*.vmdk)) are created in the backup destination.
• During subsequent backup, In-file delta - an AhsayOBM feature is employed, to track only data blocks that have change since the last backup. All changed data blocks are saved as incremental / differential delta files in the backup chain.

During a subsequent backup in non-VDDK mode, VM files are streamed to the Backup Client Computer, for delta generation:

For backup of virtual machines on vCenter / ESXi servers, make sure the following requirements are met.

For backup of virtual machines on ESXi server (with no vCenter setup), a separate Backup Client Computer must be prepared for AhsayOBM to install on.

IMPORTANT
AhsayOBM cannot be installed on an ESXi server directly.

For environment with vCenter setup, AhsayOBM is installed on the vCenter computer for best performance.

Ensure that the following requirements are met by the Backup Client Computer or the vCenter computer:

Run Direct is a feature that helps reduce disruption and downtime of your production VMs. To utilize the Run Direct feature, ensure that the following requirements are met:

` ^ ~ = ; ! / ( [ ] { } @ $ \ & # % +

For VM that cannot be backed up in VDDK mode, non-VDDK backup mode will be used instead.

• Independent Disk (Persistent or Non-persistent)
• Independent disk can only be backed up if the VM is shutdown during a backup. If the VM is started up during the backup, all independent disks selected for backup cannot be backed up.

Please consider the following recommendations for running VMware backup:

Install the latest version of AhsayOBM on the staging machine or Backup Client Computer for backup of VM hosted on a VMware ESX/ESXi server, or on the vCenter server.

Always stay up-to-date when newer version of AhsayOBM is released. To get our latest product and company news through email, please subscribe to our newsletter: https://www.ahsay.com/jsp/en/contact/ahsay-contact.jsp

For best backup and restore performance, it is highly recommended that AhsayOBM is installed on a server grade staging machine or backup client computer with sufficient memory and processing power. As guest VM can be very large, during backups and restore this may involve the compression & encryption of large amounts of data, which can be very resource intensive.

Make sure the latest version of VMware Tools is installed on each guest VM selected for backup. VMware Tools is used by AhsayOBM to quiesce the guest VMs prior to backing them up to create consistent backup for your VMs.

Quiescing is a process that ensures that the disk data is in a state suitable for backups to reduce the possibility of data corruption upon restore. This is especially important for transaction-based applications running on VMs like MS SQL Server, MS Exchange etc. There are different types of quiescing mechanisms, according to the guest operating systems (e.g. Crash-consistent, File-system-consistent and Application-consistent quiescing).

Although installing AhsayOBM on a guest VM as a staging machine is possible, the backup and restore will work as on a physical staging machine. This setup is actually inefficient and can lead to possible performance bottlenecks on the VMware host server, as in a VMware host the virtualization layer separates guest VM OS layer and the VMware host physical hardware layer. As the guest VM operating system does not have direct access to physical hardware where the data resides, a backup agent installed inside the guest VM must go through the virtualization layer to access the guest virtual machine data.

The CBT (Change Block Tracking) feature, which is required for backup in VDDK mode, is supported by VM host with VMware Essentials License (or other paid licenses). The CBT feature, which is utilized for tracking changes of data blocks since the last backup can be done quickly and directly on the VM host. Therefore, the performance of incremental backups is much faster with VDDK backup mode.

Another advantage of VDDK mode is the amount of data backed up is relatively smaller. The used data size of the guest VM is backed up instead of the provisioned size, so the storage cost of these backups will be less.

The temporary directory of a VMware VM backup set is set to a local volume, and not to a network volume (e.g. to improve I/O performance). However, the temporary directory should not be set to the system volume (e.g. where the operating system is installed). Refer to this article for how to setup the temporary directory for your VMware backup set.

To avoid concentrated disk I/O on the VMware host datastores which will have a negative performance impact on the guest VMs residing on these datastores, you should schedule your backups to limit the number of concurrent VM backups on a host and shared datastores. Hosts typically share the same datastores in virtual environments, and bottlenecks caused by too many simultaneous VM backups on a single datastore will affect all hosts that have VMs running on that datastore.

To provide maximum data protection and recovery flexibility you should consider to store your guest VM backups in multiple backup destinations, ideally both onsite and offsite locations. Onsite locations on local or network drives will enable very quick recovery even for large guest VMs. While offsite locations will ensure that if there is a site outage, the guest can be restored from another location.

Consider to increase the Java memory allocation setting for AhsayOBM (Java heap space) if you are using non-VDDK backup mode. If you are using non-VDDK mode and or Granular restore, it is recommended to increase the Java heap size space to at least 2GB or above for optimal performance. Refer to this KB article for how to modify Java heap size for your AhsayOBM client.

It is highly recommended to backup the whole VM instead of individual disk for backup of virtual machine with snapshot.

Consider to perform routine recovery test to ensure your backup is setup and performed properly.

Consider to disable the memory snapshot or quiesce guest options when taking snapshot for VMware VM backup, to shorten the time required for the process.

• Snapshot the virtual machine's memory
• Quiesce guest file system (Needs VMware Tools installed)

AhsayOBM granular restore technology enables the recovery of individual files from a guest VM without booting up or restoring the whole guest VM first. Click here to read the details.

• VMware Backup and Restore Guide

• How to create a VMware backup set to be used with the Run Direct feature?
• How to instantly startup a VMware virtual machine with the Run Direct feature (with migration)?
• How to instantly startup a VMware virtual machine with the Run Direct feature (without migration)?

• Incorrect error message 'Another backup job is still running' is displayed (VMware VM backup)
• VMware VM backup job cannot run to completion (for backup set with backup schedule disabled)
• Guest virtual machines are not listed in AhsayOBM backup source when creating a VMware Workstation backup set
• "Cannot connect SSH, please check the SSH settings" error when creating a backup set on a VMware ESXi 6.0 Update 2 host
• VMware VM cannot start up properly with Operation System not found error (Run Direct restore failing when the backup destination is located on a network drive)
• When perform a VMware Run Direct restore on AhsayOBM the following error is shown “The NFS Service on this machine is not started or not functioning properly. This service is required for VM Run Direct.
• "Cannot connect SSH, please check the SSH settings" error when creating a backup set on a VMware ESXi host
• Unable to find vmrun.exe in "‘C:\Program Files (x86)\VMware\VMware Player’ or ‘C:\Program Files (x86)\VMware\VMware VIX’” when creating a VMware Workstation BackupSet
• ISSUE:"The specified virtual machine could not be found." warning is shown on VMware Workstation scheduled backup job
• ISSUE:"Reason = "UUID conflict on Virtual Machine." error is shown on VMware ESXi backup job
• VMware ESXi scheduled backup job missed with 'Current license or ESXi version prohibits execution' error
• VMware ESXi/vCenter VDDK mode backup job ends with "Error=Snapshot not taken since the state of the virtual machine has not changed since the last snapshot operation."
• “Hostname cannot be empty” error is shown when starting VMware Run Direct restore from AhsayCBS web console
• ISSUE:"Failed to restore "New Virtual Machine". Reason = "Fail to set entity permission. Error=Unknown error from class "com.vmware.vim25.UserNotFound" when restoring a guest VM to another VMware ESXi host
• ISSUE:"No backup set can start Run Direct" message shown when initiating a VMware ESXi/vCenter Run Direct restore on AhsayCBS user web console
• "IP address "xxx.xxx.xxx.xxx" of the VirtualCenter server managing this host. Please either connect this host through it or disconnect this host from it." error is shown when creating a VMware ESXi backup set
• "[CloudException.ConnectFailedExpt[SFtpManager.login] Failed to access SFTP" error when create VMware ESXi backup set
• ISSUE:"Falied to list file (the media is write protected" when mounting a VMware ESXi/vCenter Window 10 guest VM virtual disk using Granular Restore
• "Access is denied" error when restoring files/folders using Granular Restore