Cleaning up VMware Snapshots

Posted by Keith Aasen - Consulting Systems Engineer, Virtualization

NetApp’s Snap Manager for Virtual Infrastructure (SMVI) has been very well accepted into our customer base and I believe even brought several new customers into the fold. Details on the tool have been well documented here and here as well as some tips and tricks here.

In short, SMVI leverages the Snapshot capability of the NetApp storage array to create very fast (seconds) backups of the virtual machine datastore and allows for very granular restores of the virtual machines or even the files contained within them.  SMVI can optionally also leverage the VMware snapshot process which will quiece the VM using the installed VMware tools and VMware’s VSS writer.

Users of any tool that leverages VMware snapshots know that on occasion VMware snapshots can fail to close on demand. This is usually an unforeseen spike in IO on the VM or host that causes the Snapshot delete API call to time out. Although SMVI will still successfully continue to backup the VM this orphaned snapshot can cause problems down the road if not addressed promptly. First, the longer the snapshot is in existence the harder and more disruptive it is to close. This is because all the IO that has accumulated in the snapshot needs to be replayed back into the base VMDK.  So the sooner we detect these orphans the better. Second if left long enough the VMware snapshot can fill all the free space on the VMFS data store which will take all the VMs housed in that datastore offline.  Not good.

Matt Robinson, one of NetApp’s Professional Service specialists and creator of the very popular (and contest winning) SV-SMVI application has once again come to our rescue.

Matt has developed CVMS (Clean VMware Snapshots). This tool will let you target these orphaned snapshots and take action. It also works very nicely with the support for custom scripts in SMVI 2.0. For example if you add this following script the last SMVI backup job of the night, CVMS will do a check on all your datastores and make sure no SMVI created snapshots still exist, destroying any orphaned SMVI snapshots it’s finds. The tool however will leave any user created snapshots intact. 

if %BACKUP_PHASE% == POST_BACKUP goto Cleanup

goto ende

:Cleanup

set PATH=C:\Program Files\NetApp\cvms; %PATH%

cvms.exe -vcip myvcserver -vcuser smvicleaner -vccryptpasswd 53616c7465645f5f7ae38726a98a94262c754945adee29c3d2eb126addd27eea -snapname smvi_ -allds

goto ende

:ende

In the above example;

            -vcip –is the hostname of the VCenter server

            -vcuser is a user on the VC server with limited permissions to just delete snapshots

            -vccryptpasswd is the encrypted password for that limited user so that no plain text password are required

            -snapname is the string you want to search for. Since SMVI always creates it’s snapshots with the prefix smvi_ that is the string to search with

            -allds will search all datastores in the VC server looking for snapshots with the smvi_ prefix

 

If you want to be a bit more targeted you could also specify the exact datastore to search. In the case for each SMVI job that backs up a data store, say datastore X. The script would just be;

 

if %BACKUP_PHASE% == POST_BACKUP goto Cleanup

goto ende

:Cleanup

set PATH=C:\Program Files\NetApp\cvms; %PATH%

cvms.exe -vcip myvcserver -vcuser smvicleaner -vccryptpasswd 53616c7465645f5f7ae38726a98a94262c754945adee29c3d2eb126addd27eea -snapname smvi_ -ds datastoreX

goto ende

:ende           

You may even want to run the check before the SMVI backup as well as after by removing the opening “if” statement.

Of course you will want to be sure that you are not trying to delete snapshots from running SMVI jobs to be sure that no other SMVI job is running against the targeted datastores during the cleanup process.

The tool is available on the NetApp communities site here.

Comments

[Keith]

Update; I had a co worker suggest a couple tweaks for my script so I thought I would share. Thanks Sebastian!

Use 'if NOT ... ' to avoid unnecessary jumping and avoid hardcoding
C:\Program Files\ since many of your readers might use
'internationalized' or customized (different directories) Windows
versions. So the script would look like (e.g.)

if not %BACKUP_PHASE% == POST_BACKUP goto ende

set PATH=%ProgramFiles%\NetApp\cvms; %PATH%

cvms.exe -vcip myvcserver -vcuser smvicleaner -vccryptpasswd
53616c7465645f5f7ae38726a98a94262c754945adee29c3d2eb126addd27eea
-snapname smvi_ -ds datastoreX

:ende

Keith

[Steve]

This worked well in our environment, now we don't have to remember to cleanup after SMVI. The NetApp tool chest has a compiled version that can easily be used in a scheduled task.

[Carpet Cleaning Vancouver]

Thanks for another Timely Post.
Your ability to share Information is a talent and very appreciated

Carpet Cleaning Vancouver