With S3 Browser, you can reliably
upload and
download
your files to and from Amazon S3. S3 Browser supports data integrity features to ensure that
data is not corrupted while traversing the network.
Data Integrity Checking for Uploads
To enable data integrity checking for uploads:
1. Click Tools, Options
Click Tools, Options
2. The Options dialog will open, switch to the Data Integrity tab:
Click Tools, Options, Data Integrity
3. Tick the Perform data integrity check when uploading files checkbox.
4. Click Save Changes.
When this option is enabled, S3 Browser calculates an SHA256 hash for each file you upload
and sends this hash to Amazon S3. Amazon S3 calculates the hash on the server side and compares
it with the hash provided by S3 Browser. If the hashes do not match, an error is returned,
and the file will not be written to Amazon S3.
S3 Browser will also write calculated hash as a custom metadata header (x-amz-meta-sha256)
to check data integrity when you download the file.
This option is also strongly recommended if you would like to
verify data integrity when downloading files (see below).
Data Integrity Checking for Downloads
To enable data integrity checking for downloads:
1. Click Tools, Options
Click Tools, Options
2. The Options dialog will open, switch to the Data Integrity tab:
Click Tools, Options, Data Integrity
3. Tick the Perform data integrity check when downloading files checkbox.
4. Click Save Changes.
When this option is enabled, S3 Browser calculates the hash for the downloaded file and compares
it with the hash returned by the server (Amazon S3). If the hashes do not match, an error is returned,
and the file is not written to the local disk.
There are two modes of data integrity checking for downloaded files:
Flexible
Strict
In Flexible mode, S3 Browser performs a data integrity test only for files for which a valid hash is provided
by the server. Files with unknown or missing hashes are not checked and are treated as valid.
In Strict mode, S3 Browser checks all files. Files with unknown or missing hashes are treated as corrupted.
More details about data integrity checking
S3 Browser uses the following approach to get the hash of a remote file:
First, it attempts to read the hash from the x-amz-meta-sha256 custom metadata header.
If the header contains a 256-bit hash string, S3 Browser calculates an SHA256 hash of the downloaded file and compares the hashes.
If no valid SHA256 hash is found, it attempts to read the hash from the x-amz-meta-md5
custom metadata header. This header was used by older versions of S3 Browser to store an MD5 hash of the file.
If the header contains a 128-bit hash string, S3 Browser calculates an MD5 hash of the downloaded file and compares the hashes.
If the x-amz-meta-sha256 and x-amz-meta-md5 headers are missing or do not contain valid hashes,
S3 Browser looks into the ETag header. This header is used by Amazon S3 to store an MD5 hash for files
uploaded via single-part uploads. If a valid hash is found, S3 Browser calculates an MD5 hash of the downloaded file
and compares the hashes.
However, there may be situations where a valid hash of the remote file is not available, and thus a data integrity test
cannot be performed.
Amazon S3 does not write a valid MD5 hash into the ETag header for files uploaded via the Multipart Upload feature.
Amazon S3-compatible storage solutions may use different hashing algorithms or data formats for storing file hashes,
or they may not provide file hashes at all.
Additionally, the file could have been uploaded by third-party software or by S3 Browser with the data integrity feature
turned off.
You may choose how to handle files with unknown or missing hashes:
In Flexible mode, S3 Browser performs a data integrity test only for files for which a valid hash is provided
by the server. Files with unknown or missing hashes are not checked and are treated as valid.
In Strict mode, S3 Browser checks all files. Files with unknown or missing hashes are treated as corrupted.
How to Avoid the 'Missing Hash' Issue
First, you can enable data integrity checking when uploading files. When this option is turned on,
S3 Browser writes the hash in the file metadata during the upload process (Amazon S3 allows you to store any
custom information for each file).
Later, when you download the file, S3 Browser can extract the hash from the metadata
and compare it with the calculated hash.
Another option is to disable multipart uploads in Tools, Options, General. However, this may reduce
the speed of your uploads. Additionally, you can only upload files up to 5 GB in size with this option;
for larger files, multipart uploads need to be enabled.
How to Fix the Hashes for Already Uploaded Files
The method described below allows you to fix file hashes for files up to 5 GB in size.
There is no way to fix hashes for files over 5 GB in size. You can only re-upload them with
data integrity checking for uploads enabled or select the Flexible mode for data integrity checking for downloads.
To fix the hashes for files up to 5 GB in size:
Click Tools, Options and open the General tab.
Disable the Enable multipart uploads option.
Select one or multiple files.
Open the Http Headers tab and click Apply.
S3 Browser will update the ETag header with the hash through a simple COPY request.
"Amazon Web Services", "AWS", "Amazon S3", "Amazon Simple Storage Service", "Amazon CloudFront", "CloudFront",
the "Powered by Amazon Web Services" logo are trademarks of Amazon.com, Inc. or its affiliates in the US
and/or other countries.