Data Integrity Checking. Uploading and Downloading with data integrity checking.
S3 Browser
Free Windows Client for Amazon S3 and Amazon CloudFront
Follow:
Share:

Data Integrity checking

With S3 Browser you can reliably upload and download your files to and from Amazon S3. S3 Browser supports the number of features allowing you to be sure that data is not corrupted traversing the network.

Data Integrity checking for uploads

To enable data integrity checking for uploads, please open:

Tools, Options, Data Integrity

Tools, Options, Data Integrity

Click Tools, Options, Data Integrity

When this option is turned on, S3 Browser calculates the SHA256 hash for an each file you upload and sends this hash to Amazon S3. Amazon S3 calculates the hash on the server side and compares it with the hash provided by S3 Browser. If they do not match, returns an error. The file will not be written on Amazon S3 if hashes do not match.

This option is also strongly recommended if you would like to test data integrity when downloading files (see below).

Data Integrity checking for downloads

To enable data integrity checking for downloads, please open:

Tools, Options, Data Integrity

Tools, Options, Data Integrity

Click Tools, Options, Data Integrity

When this option is turned on, S3 Browser calculates the hash for downloaded file and compares it with the hash returned by the server (Amazon S3). If hashes do not match it returns the error and file is not written on local disk.

There are two modes of data integrity checking for downloaded files:

  • Flexible
  • Strict

In Flexible mode S3 Browser performs data integrity test only for files for which valid hash is provided by the server. Files with unknown or missing hashes are not checked and treated as valid.

In Strict mode S3 Browser checks all files. Files with unknown or missing hashes are treated as corrupted.

The reason for two modes of data integrity checking

Amazon S3 doesn’t provide the hash for files uploaded via the Multipart Upload feature. Thereby there is no way to get the hash to compare with.

Amazon S3 compatible storages may also use another hashing algorithms and data formats for storing file hashes or not provide file hashes at all.

You may choose how to handle the files with unknown or missing hashes.

How to avoid the ‘missing hash’ issue

First, you can enable data integrity checking when uploading files. When this option is turned on, S3 Browser writes the hash in file metadata during uploading (Amazon S3 allows you to store any custom information for an each file).

Later, when you download the file, S3 Browser can extract the hash from the metadata and compare it with the hash calculated.

Another way is to disable multipart uploads in Tools, Options, General, but this could reduce the speed of your uploads. Plus you can upload files up to the 5Gb in size only, for larger files you need to enable multipart uploads.

How to fix the hashes for already uploaded files

The method described below allows you to fix file hashes for files up to the 5Gb in size. There is no way to fix hashes for files over 5Gb in size, you can only re-upload them with data integrity checking for uploads enabled or select the Flexible mode for data integrity checking for downloads.

To fix the hashes for files up to the 5Gb in size:

  1. Start S3 Browser and choose the bucket
  2. Select one or multiple files
  3. Open the Http Headers tab and click Apply

S3 Browser will update the ETag header with the hash through the simple COPY request.

S3 Browser 8.1.5 Freeware
Powered by Amazon Web Services and Rated by CNET Editors!
Social Connection
S3 Client Logo
Like Us!
 
People like S3 Browser!
People like us
Our customers say

"S3 Browser is an invaluable tool to me as a web developer to easily manage my automated site backups" -Bob Kraft, Web Developer

"Just want to show my appreciation for a wonderful product. I use S3 Browser a lot, it is a great tool." -Gideon Kuijten, Pro User

"Thank You Thank You Thank You for this tool. A must have for anyone using S3!" -Brian Cummiskey, USA

Related Products
FastGlacier
Windows Client for Amazon Glacier - new low-cost storage for data archiving and backup.
RdpGuard
protects your Windows Server from RDP Brute-force Attacks.
"Amazon Web Services", "AWS", "Amazon S3", "Amazon Simple Storage Service", "Amazon CloudFront", "CloudFront", the "Powered by Amazon Web Services" logo are trademarks of Amazon.com, Inc. or its affiliates in the US and/or other countries.
Copyright © 2008-2019 NetSDK Software. All rights reserved.  Terms of Use.  Privacy Policy.  S3 Drive.  RDP brute-force protection.