To Dup or not to Dup data during backup

Filed in archive Technology on May 24, 2008

© Sam UL

Durring the process of data backup we can question the validity of backing up everything or performing some form of selective backup. Duplication of data in the backup stream can be a blessing or a curse. Imagine backing up 100's of workstations. Some might of the workstations will share a common operating system while others will not. Some sharing the same operating system might be on the same version and some might not. If your IT deparment is doing there job there won't be that many different operating systems and backups would be enormous if full system backups were done for every workstation.

Enter in the age of de-duplication during the backup process. As backups are performed to the same storage device, we have the potential, and neccessity, to make sure redundant data is removed. The removal of redundant data can be enormous for large companies. Vendors take the de-duplication effort to different levels, some down to the data block level. De-duplication can take place either on the client side (where the backup is coming from) or server side (where the backup is going). Obviously there will be less network traffic if the de-duplication takes place on the client side as the data being backed up won't have to travel in network.

You can take a look at how EMC is cutting down on copies through de-duplication