Set Default Metadata/HTTP Header for Amazon S3 with CrossFTP

Metadata is widely used in S3 and other cloud storage systems for its storage object's description and HTTP Header's setting.

There are cases that you want to setup the default metadata for all your uploaded objects, e.g, define a common HTTP Header or expire date. To do this, you can choose Site Manager -> S3 -> Enable Default Metadata to allow the setting of the default metadata, and setup the metadata by press the "..." button.

Fig. 1. Default Metadata Setup

You can setup the default metadata in the popup dialog. After press the OK button, the metadata will be saved for future transfers.

When a file is to be transferred to the remote site, CrossFTP will firstly calculate the general metadata for the file, including the file's original modification date, the compression algorithm, the encryption algorithm, mime type, etc. After that, the default metadata will be inserted into the object's metadata table before the transfer. The file transfers together with its metadata info to avoid additional operation cost.

You can check whether the file's metadata has been correctly setup by right click on the file, chose properties, and see the metadata tab.

This feature is available for CrossFTP Pro 1.86.2 or later.

-- 

CrossFTP is a FTP, SFTP, FXP, WebDav, Amazon S3, Amazon Glacier and Google Cloud Storage client for Windows, Mac, and Linux.
CrossFTP Team
http://www.crossftp.com/

How to Sign Private Distribution's URL with CrossFTP

To sign a URL for private distribution's object, you firstly need to setup a private distribution with read permission to your S3 objects. Check this blog for how to properly setup the private distribution.

To create the sign URLs, you can select the files you want to sign, and choose "URL..." from the popup menu. The URL will be shown to assist your URL Sign tasks. Our URL generator dialog can detect whether an S3 object is associated with private Download/Stream distribution, and then assist to create the corresponding HTTP/HTTPS/RTMP/HTML URLs, as shown in Fig. 1.

 Fig. 1. URL Dialog

Before start to sign the URL, defining the secure signing policy's definition is a precondition:

Secure Signing Policy

You can setup the signing policies by choosing Tools -> Policies, or  in the URL dialog and press the Signing policy: Configure button.
The policies list dialog will be shown as Fig. 2. You can use "New" button to create the new policy, "Edit" button to edit an existing policy, "Delete" button to remove a policy, "Save as" and "Load from" button to export/import the policies.

Fig. 2. Policies Dialog

There are two types of signing policies: Canned Policy and Custom Policy. Here we introduce these policies one by one:

Canned Policy: a "simplified" policy definition that uses default options for your signed URL's behaviors, as shown in Fig. 3. Key pair ID and Private Key file are the most importation items in this dialog.

Fig. 3. Canned Policy Modification

1. Key pair ID is the ID number you will get from Amazon when you generate a Key Pair using Amazon web interface or when you upload your own key.
2. Private Key File is the file you downloaded from Amazon web interface when you generate a Key Pair, or the private key file you used to generate the Key pair for uploading to Amazon.

Custom Policy: a policy describes custom access permissions to apply via a private distribution's signed URL, as shown in Fig. 4. Compare this with the Canned policy, we can find that two important items are added here: CIDR range and Resource URL.

Fig. 4. Custom Policy Dialog

1. CIDR range: An optional range of client IP addresses that will be allowed to access the distribution, specified as a CIDR range.
2. Resource URL: An optional HTTP/S or RTMP resource path that restricts which distribution and S3 objects will be accessible in a signed URL. For standard distributions the resource URL will be "http://distributionName/objectKey" (may also include URL parameters). For distributions with the HTTPS required protocol, the resource URL must start with "https://". RTMP resources do not take the form of a URL, and instead the resource path is nothing but the stream's name. The '*' and '?' characters can be used as a wildcards to allow multi-character or single-character matches respectively.

Generate signed URL for Private Distribution

URL dialog checks the S3 objects' associated distribution, and adds the private distribution's domain name/CNAMEs in the CNAME dropdown box. The URL dialog supports to generate HTTP/HTTPS/RTMP/HTML/FTP/Path type URLs.
For the download distribution, you should choose HTTP/HTTPS radio button, choose the prefixed distribution URL in CNAME drop-down box, select the proper signing policy, and then press "Generate" button to sign the distribution, as shown in Fig. 1.
For the stream distribution, you should choose RTMP/HTML radio button, choose the prefixed distribution URL in CNAME drop-down box, select the proper signing policy, and then press Generate button to sign the distribution, as shown in Fig. 5.

Fig. 5. Sign URL for Stream Distribution

Conclusion

CrossFTP Pro makes it easy to manage your CloudFront distribution and URL signing.

This feature is available for CrossFTP Pro 1.86.2 or later.

--

CrossFTP is a FTP, SFTP, FXP, WebDav, Amazon S3, Amazon Glacier and Google Cloud Storage client for Windows, Mac, and Linux.
CrossFTP Team
http://www.crossftp.com/

How to Configure Private CloudFront Distribution with CrossFTP

Private CloudFront distribution is a good way to securely distribute the private download/streaming contents with Amazon CloudFront.

CrossFTP fully supports the private CloudFront distribution. To enable Private distribution, you can select a bucket, and right click to choose CloudFront -> Manage CloudFront Distributions..., and you can see a list of existing CloudFront distributions for the bucket, as shown in the Fig. 1.

 Fig. 1. CloudFront List Dialog

Create/Update the Private CloudFront Distribution

You can create a new CloudFront distribution by press the "New" button, or update an existing CloudFront distribution by press the "Edit" button. You can fill in the CloudFront distribution's basic information in the General Tab. 
To enable the private CloudFront distribution, you should choose the "Private Content" tab, and toggle the "Enable Private Distribution" button, as shown in Fig. 2.

 Fig. 2. Enable Private Distribution

There may be a few concepts new for you in this dialog:

• Original Access Identity: this is a "virtual" user that is used for managing permissions that represents the private CloudFront distribution.

• S3 Canonical User ID:  Associated with the Origin Access Identity, this is the user ID that you can use in ACL/permission setting dialog to grant permissions for the distribution. If you want to use distribution to share the contents, you need firstly to grant this ID the read permission in ACL/permission setting dialog.
• Trusted Signers: Only trusted signers are allowed to sign the distribution URL in S3. If you want to sign the private distribution URL for the download/streaming, you should at least add one trust signer, for example, by toggle "Add Myself as a Trusted Signer".
  To find your account number, go to Your Account | Account Activity on AWS website, and look for the Account Number.

Grant the Permission for the Original Access Identity

You have to grant Read access to CloudFront Origin Access Identity account to make the files available for Private Distribution. 
To do this, two approaches are possible:

• Grant the read permission for the S3 Canonical User ID. Copy the S3 Canonical User ID as shown in Fig. 2., and add that user the read permission for the S3 object by right click on the object, and choose "ACL Setting...", as shown in the Fig. 3.

 

Fig. 3. Add S3 Canonical User ID with Read Permission 

• Or you can configure the Bucket Policy to always grant Read access to Origin Access Identity, by right click on the Bucket, and choose "Bucket Policy..." to set up the policy.

This feature is available for CrossFTP Pro 1.86.2 or later.
CrossFTP is a FTP and Amazon S3 client for Windows, Mac, and Linux.

Setup Default Permission for File Transfer with CrossFTP

Permission is one of the most important security aspect for the files on the remote site. We pay high attention on this area, and to allow flexible permission control for file transfers, CrossFTP supports four types of permission setting policies for the upload and FXP tasks. You can find the permission setting options in the Site Manager -> Actions-> Default Permission.

 Fig. 1. Default Permission Setup

As shown in the Figure 1, the available options are:

• No Inheritance: do not setup and arbitery permissions in the transfer process.

• From Parent: the file's permission will be copied from its parent folder's permission.

• From Source: the permission will be copied from the FXP transfer's source file.

• Default Permission: you can specify the default permission at the right side's "..." button. Transfered files will be assigned with this permission. For different FTP protocols, such as FTP (Fig. 2) and S3 (Fig. 3), different UI will be provided for setting the default permission. This is the recommended choice if you don't know which option to choose.

 Fig. 2. Default Permission UI for FTP

 Fig. 3. Default Permission UI for S3

For S3 site, the permission will be assigned during the transfer, so that you don't need to pay for the permission change operation's fee.

This feature is available for CrossFTP 1.81.2 or later.
CrossFTP is a FTP and Amazon S3 client for Windows, Mac, and Linux.

Client Side Encryption Support with CrossFTP

Security is one of the top consideration when choosing FTP/Cloud storage client software and providers. CrossFTP puts security as top priority in its design. For security of FTP/Cloud storage client software, it includes 3 parts: 1,  protect the FTP/Cloud storage sites' password in local configuration; 2,  protect the password and files during the data transfer; 3, protect the files on the remote storage service provider.

In this document, we focus on point 3: how CrossFTP protect the files stored on the remote storage service provider: encrypt the files on local before sending them out.

Client side encryption for Amazon S3, Amazon Glacier, FTP, or WebDav are one important security feature CrossFTP Pro provides to enhance the data transfer security. To further enhance the transfer security, CrossFTP Pro supports client side encryption for for all FTP protocols it can handle, including FTP, SFTP, FTPS, WebDAV, Amazon S3, etc. After enable the client side encryption, in the upload process, all files will be firstly encrypted before they are transferred to the remote site.  In the download process, the encrypted file will be decrypted after the file is downloaded from the remote site to the local drive. We will add ".aes" extension to the encrypted files to stand for the encryption.

We uses one of the strongest block ciphers available, 256-bit Advanced Encryption Standard (AES-256), for the encryption. 256-bit is the largest key size defined for AES.

To enable the client side encryption, you can choose Site Manager -> Security -> Enable Local Encryption, and input the password for the encryption, as shown in the figure. The password is encrypted stored in the site bookmarks file to ensure the security.

TIP: On a default JRE/JDK installation, AES is limited to 128-bit key size. Hence you will see AES-256 encryption failed with empty files. This is a remnant of import/export laws on cryptographic algorithm. To unlock larger AES key sizes, you need to download and apply the "JCE Unlimited Strength Jurisdiction Policy Files" (for latest JRE/JDK see at the bottom of this page).
Attention: You must make sure you enter the password correctly and remember the password.  Otherwise, you cannot recover the encrypted files if you lost the configuration.

This feature is available for CrossFTP 1.86.2 or later.
CrossFTP is a FTP and Amazon S3 client for Windows, Mac, and Linux. 

CrossFTP 1.86.2 Released

CrossFTP 1.86.2 is a major update. The main changes are:
* Adds sign URL support for S3 private distribution.
* Adds local encryption support for file transfer.
* Support enqueue the file delete operation for multi-thread processing.
* Adds Mac Mountain Lion and sandbox's support.
* Adds zebra line color to match Mountain Lion's finder style.
* Better compatbility for s3 compatible protocol, such as Ceph, etc.
* Solve FTP PASV/PORT firewall issue in Windows Vista/7/2008 with JDK 7 by fallback IPV4 network stack.
* Fix the private CloudFront update bug.
* Improved file context menu for easier understanding.

This update is recommended for all CrossFTP users.

CrossFTP Team
http://www.crossftp.com/