✎ [V8] FAQ: How to run a Data Integrity Check for backup data stored in backup destination? [Ahsay Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== [V8] FAQ: How to run a Data Integrity Check for backup data stored in backup destination?  =====

<html>
 <b>
  Article ID:
 </b>
 8004
 <br/>
 <b>
  Reviewed:
 </b>
 2021-03-03
 <br/>
 <br/>
 <b>
  Product Version:
 </b>
 <br/>
 AhsayOBM / AhsayACB: 8.1 to 8.x
 <br/>
 OS: All platforms
</html>

=====  Description  =====
<html>
This article outlines the steps to perform a Data Integrity Check for backup data stored in the backup destination (e.g. local destination, AhsayCBS, or other cloud storage).
<br><br>
The functions of the Data Integrity Check (DIC) is to:
<ol>
  <li><font color=black> 
  Identify and remove the files and/or folders in the backup destination(s) which do not appear in the index.</font>
 </li>
 
 <li><font color=black> 
  Identify and remove the files and/or folders which appear in the index but do not actually exist in the backup destination(s).</font>
 </li>
 
 <li><font color=black> 
  Identify and remove corrupted files from the backup destination(s) when the Run Cyclic Redundancy Check (CRC) During Data Integrity Check setting is enabled.</font>
 </li>
 
 <li><font color=black> 
  Identify and remove partially uploaded (orphan) files from the backup destination(s) to free up storage space.</font>
 </li>

 <li><font color=black> 
  Identify and rebuild malfunction file index (option available for version 8.5.0.63 or above).</font>
 </li>
 
 <li><font color=black> 
  Update the storage statistics for the backup set(s).</font>
 </li>
 </ol>
<font color=red>Important:
<br>
The Data Integrity Check <u>cannot</u> fix or repair files that are already corrupted. It will only identify and remove any corrupted files from the backup destination(s), so these files can be re-uploaded again on the next backup job if they still exist on the backup source. However, any corrupted files removed from the Retention Area will be not be re-uploaded and will no longer be restorable.
<br>
<br>
Also, the Rebuild index function mentioned in point 5 above is only available for AhsayOBM / ACB version 8.5.0.63 or above.
<br>
<br>
      <img style="width:80%" src="http://wiki.ahsay.com/lib/exe/fetch.php?media=public:8004_2.png"/>





</font>
</html>

===== Steps =====
<html>
To perform a data integrity check, login to the AhsayOBM / ACB user interface:
 <br/>
 <ol>
  <li>
   <font color=black>Login to the AhsayOBM / ACB user interface, select the <b>Utilities</b> tile.</font><br><br>
      <img style="width:16%" src="http://wiki.ahsay.com/lib/exe/fetch.php?media=public:5004_0.png"/>
   <br>
   <br>
  </li>
  <li>
   <font color=black>Select the corresponding <b>backup set</b> from the dropdown menu and then select the <b>backup destination</b> from the dropdown menu
   <br>
   <br>
      <img style="width:60%" src="http://wiki.ahsay.com/lib/exe/fetch.php?media=public:8004_1.png"/>
   <br>
   <br>  
   Note:<br>
   The "All" backup sets option will enable check for all destinations of all backup sets. &nbsp;The time required to complete the integrity check will depend on the number of backup sets and destinations.</font><br><br>
  </li>
  <li>
   <font color=black>Enable the <b>Run Cyclic Redundancy Check (CRC) during data integrity check</b> option to check on integrity of the files against the checksum file generated at the time of the backup job.
   <br>
   <br>
      <img style="width:60%" src="http://wiki.ahsay.com/lib/exe/fetch.php?media=public:8004_4.png"/>
</font><br><br>
  </li>
  <li>
   <font color=black>Enable the <b>Rebuild index</b> option to rebuild possible malfunction index in the backup destination.
   <br>
   <br>
      <img style="width:60%" src="http://wiki.ahsay.com/lib/exe/fetch.php?media=public:8004_5.png"/>  
   <br>
   <br>
   <font color=red>Important:<br>
The Rebuild index option is only available for AhsayOBM / ACB version 8.5.0.63 or above.</font>    
</font><br><br>
  </li>  
   </li>
  <li>
   <font color=black>Then press the <b>Start</b> button to start the data integrity check.
   <br>
   <br>
   Note:<br>
   i.  A data integrity check can only be performed when there is no manual / scheduled backup job in progress (of the corresponding backup set). It is highly recommended to temporarily disable the backup schedule to ensure that no scheduled backup is started while the data integrity check is still running.
   <br/>
   <br/>
   The following error message will be displayed to indicate that the data integrity check had skipped a backup set with active backup job
   <br/>
   <br/>
   <i>
    Skipped Backup Set="Backup Set". Reason = "Backup Job "Backup Set" is still running."
    <br/>
    Finished data integrity check with error on backup set "Backup Set (Backup Set ID)
   </i>
   <br/>
   <br/>
    ii.  The time required to complete a Data Integrity Check depends on a number of factors, such as the number of files/folders in the backup set(s), the bandwidth available on the client computer, hardware specifications of the client computer such as the disk I/O and CPU performance, and if there are other resource intensive job running.
   <br/>
   <br/>
   <font color=red>iii.  If the CRC (Cycle Redundancy Check) option is enabled, backup data will be streamed from the backup destination (e.g. the cloud storage location or FTP location for example), to the client computer in order to perform the CRC check. This may incur additional charges from your Cloud Storage provider. For users with metered internet connection, this could result in additional charges by your ISP (Internet Service Provider).</font>
   <br/>
   <br/>
   iv.  During a data integrity check, pay attention to the resource usage on the client computer.

   <br/>
   <br/>
   v.  For backup destinations on cloud storage services, such as Amazon S3, data integrity checks are expected to be performed by the cloud storage service provider.
   <br/>
   <br/>
   According to Amazon S3 FAQs (<a href="http://aws.amazon.com/s3/faqs/">http://aws.amazon.com/s3/faqs/</a>)
   <br/>
   <br/>
   <blockquote class="uncited">
    <div>
     <span style="font-size: 80%; line-height: 116%;">
      Q: How is Amazon S3 designed to achieve 99.999999999% durability?
      <br/>
      <br/>
      Amazon S3 redundantly stores your objects on multiple devices across multiple facilities in an Amazon S3 Region. The service is designed to sustain concurrent device failures by quickly detecting and repairing any lost redundancy. When processing a request to store data, the service will redundantly store your object across multiple facilities before returning SUCCESS. Amazon S3 also regularly verifies the integrity of your data using checksums.
      <br/>
      <br/>
      Q: What checksums does Amazon S3 employ to detect data corruption?
      <br/>
      <br/>
      Amazon S3 uses a combination of Content-MD5 checksums and cyclic redundancy checks (CRCs) to detect data corruption. Amazon S3 performs these checksums on data at rest and repairs any corruption using redundant data. In addition, the service calculates checksums on all network traffic to detect corruption of data packets when storing or retrieving data.
     </span>
    </div>
   </blockquote>

   Consult with your cloud service provider to ensure that CRC checks are performed regularly for your data.</font>
  </li>
 </ol>
</html>

===== Keywords =====
dic, index, rebuilding, bdb, backupset, set, perform
User Tools

Site Tools

Page Tools
