S3 supported APIs and limitations
This page describes limitations concerning the S3 service and protocol implementation.
Last updated
This page describes limitations concerning the S3 service and protocol implementation.
Last updated
WEKA supports two URL styles for API requests to S3 buckets: path-style and virtual-hosted-style.
Style | URL format |
---|---|
The difference between the styles is subtle but significant. When using a URL to reference an object, the DNS resolution maps the subdomain name to an IP address. With the path style, the subdomain is always s3.domain-name.com
. With the virtual-hosted-style, the subdomain is specific to the bucket.
The addressing style used to construct the request is determined by the S3 client sending the request.
The following standard S3 APIs are supported:
Bucket Lifecycle APIs:
Bucket Policy APIs:
Bucket Tagging APIs:
Object APIs:
Object Tagging APIs:
For a multipart object upload, the bucket’s free capacity must be at least twice the size of the object until the upload is complete.
Bucket names must be between 3 and 63 characters long.
Bucket names can consist only of lowercase letters, numbers, dots (".
"), and hyphens ("-
").
Bucket names must begin and end with a letter or number.
Bucket names must not be formatted as IP addresses (for example, 192.168.5.4
).
Bucket names must be unique across the cluster.
Object key names may be up to 1024 characters long.
An object prefix cannot begin with a forward slash ("/
").
Adding a forward slash ("/
") in the object's prefix after the first character is interpreted as a directory. Such directory segments are limited to 255 characters.
For naming convention details, see Creating object key names (AWS portal).
Ensure the object key name is also compatible with protocols other than S3. Specifically, avoid special characters that might be unsupported in the other protocols.
The S3 protocol implementation supports the following policy actions:
s3:*
This wildcard is supported for IAM policies but not for bucket policies.
s3:AbortMultipartUpload
s3:CreateBucket
s3:DeleteBucket
s3:DeleteBucketPolicy
s3:DeleteObject
s3:GetBucketLocation
s3:GetLifecycleConfiguration
s3:PutLifecycleConfiguration
s3:ListBucketMultipartUploads
s3:ListMultipartUploadParts
s3:GetBucketPolicy
s3:GetObject
s3:ListAllMyBuckets
s3:ListBucket
s3:PutBucketPolicy
s3:PutObject
s3:GetBucketTagging
s3:PutBucketTagging
Only MD5 checksum algorithm is supported.
WEKA supports the Amazon S3 Lifecycle Configuration elements and definitions, with the limitation of only supporting the lifecycle Expiration
action.
Item | Limits |
---|---|
Path-style
https://s3.domain-name.com/bucket-name/object-name
Virtual-hosted-style
https://bucket-name.s3.domain-name.com/object-name
Maximum number of buckets
10000
Maximum object size
5 TiB
Maximum number of parts per upload
10000
Part numbers
1 to 10000 (inclusive)
Part size
5 MiB to 5 GiB. The last part can be < 5 MiB
Maximum number of parts returned for a list parts request
1000
Maximum number of multipart uploads returned in a list multipart uploads request
1000
User-defined metadata per object
2 KB
Maximum length of an S3 IAM user policy
2048
Maximum number of S3 IAM user policies
1024
Maximum number of S3 regular users
1024
Maximum number of S3 service accounts
5000
Maximum number of S3 STS credentials
5000