The Legacy Backup Situation We Faced
Recently, one of our long term clients approached us to give their backup environment an overhaul
Their current backup solution was a slightly outdated version of FalconStor, which we had deployed for them some years earlier
FalconStor was still doing an excellent job protecting their estate, but it had long since become unmanageable from an administration standpoint
To find a worthy replacement for FalconStor is a bit of a challenge given that the disk safe agent keeps a real time mirror of their data on the CDP appliance with limited or no overhead, offering a vear zero backup window. The CDP appliance also keeps very flexible point in time snapshots of their data and replicates their data offsite to another CDP appliance
The only downfall for FalconStor was that it was now unmanageable given their current environment had grown to 30 physical servers and 200 VMs (each requiring an agent) and each disk inside each system is effetely managed individually. When the solution was deployed they were mostly physical with a much smaller footprint
Their physical estate was easily protected with the File System, and application specific agents, there is also a CDP agent should they wish to protect specific servers in real time.
The vast majority of their virtual estate was also protected easily, the nightly backup of over 200 VMs consuming roughly 30TB takes approximately 6 hours with roughly 2TB of daily change
The Backup Problem
Unfortunately, there was 1 server that was causing a bit of a problem. It was a 3.5TB File Server with over 1000 users concurrently accessing it (including countless users with outlook pst files, that are highly sensitive to any unresponsiveness from the file server), with a very high rate of change. The client also had a desire to have recovery points of this server every 3 hours like they used to with FalconStor.
However, using Standard VADP Streaming backups over SAN transport from a HP 3PAR a Full Backup takes over 14 hours and an Incremental Backup takes ~5 hours
Note: While VADP streaming backups work well for the majority of servers in Virtual estates it can cause performance degradation during backup operations while the snapshot is left open, further to this if the snapshot is kept open for too long on a busy VM, when the snapshot is released it can cause the VM to go unresponsive during consolidation.
One potential solution to this is to move the VMs disks onto faster storage, in this case moving from NL-SAS onto SAS or even SSD so more IO is available to service the backups. However, given this machine does not require more IO for its production operations we opted for a different solution…
How we Solved the Problem
To create the VMware snapshot then create the hardware (array) snapshot then release VMware snapshot takes ~5 minutes giving us a recovery copy of that VM on the HP 3PAR. A separate process then runs which mounts the storage snapshot and performs the backup against that copy, with no production impact on the live production VM
The Incremental Backups from the array now takes under 2 hours. This now allows the client to protect their 3.5TB File server every 3 hours, with copies stored both on the SAN and on a separate disk system (also replicated offsite), with a 5-minute production backup window
Snapshot Backup Technology provides a useful tool for any backup architecture and implementation.
Our VM ‘Snapshot’ backup is finished in about 5 minutes (though it still resides on the primary SAN)
After hours, we can ingest that backup from the SAN without impacting End Users day time operations.
The backup is taken to the Backup Servers ‘bulk backup disk’ and then offsite replicated.
There’ still a case for true CDP technology such as FalconStor – where there is a Zero backup Window.
But in this scenario, moving to a single Unified Technology such as CommVault has been a winning move for this customer.