S3 supported APIs and limitations

This page describes limitations concerning the S3 service and protocol implementation.

Supported URL styles for API requests to S3 buckets

WEKA supports two URL styles for API requests to S3 buckets: path-style and virtual-hosted-style.

Style	URL format
Path-style	https://s3.domain-name.com/bucket-name/object-name
Virtual-hosted-style	https://bucket-name.s3.domain-name.com/object-name

The difference between the styles is subtle but significant. When using a URL to reference an object, the DNS resolution maps the subdomain name to an IP address. With the path style, the subdomain is always s3.domain-name.com. With the virtual-hosted-style, the subdomain is specific to the bucket.

The addressing style used to construct the request is determined by the S3 client sending the request.

Ensure S3 clients support HTTP 1.1 or higher. WEKA S3 requires this protocol version for compatibility.

URL-formatted object keys in WEKA S3

WEKA S3 supports non-posix compliant object keys (for example, https://company.com/id), expanding compatibility with AWS S3–native applications.

• Standard keys: Maintain full multi-protocol interoperability; keys map directly to standard posix filenames in the WEKA filesystem..

• Non-posix compliant keys: Keys cannot be represented with identical names in the filesystem view. These objects also remain fully accessible as POSIX files and other protocols, however under alternative naming convention that represents and is derived from the original S3 keys.

This allows organizations to integrate applications using non-posix compliant keys as identifiers, without requiring changes to naming conventions, while still preserving cross-protocol access.

Supported S3 APIs

The following standard S3 APIs are supported for bucket and object management:

• Bucket APIs:

  • HeadBucket

  • CreateBucket

  • DeleteBucket

  • ListObjects

  • ListObjectsV2

  • ListBuckets

• Bucket Lifecycle APIs:

  • GetBucketLifecycleConfiguration

  • PutBucketLifecycleConfiguration

  • DeleteBucketLifecycle

• Bucket Policy APIs:

  • GetBucketPolicy

  • PutBucketPolicy

  • DeleteBucketPolicy

• Bucket Tagging APIs:

  • GetBucketTagging

  • PutBucketTagging

  • DeleteBucketTagging

• Object APIs:

  • GetObject

  • PutObject

  • DeleteObject

  • DeleteObjects

  • CopyObject

  • HeadObject

• Object Tagging APIs:

  • GetObjectTagging

  • PutObjectTagging

  • DeleteObjectTagging

• Object Multipart APIs:

  • CreateMultipartUpload

  • CompleteMultipartUpload

  • ListParts

  • UploadPart

  • AbortMultipartUpload

• Bucket Notification APIs:

  • GETBucketNotification

  • GETBucketNotificationConfiguration

  • PUTBucketNotification

  • PUTBucketNotificationConfiguration

• Versioning APIs:

  • PutBucketVersioning

  • GetBucketVersioning

  • ListObjectVersions

• Object Lock APIs:

  • PutObjectRetention

  • GetObjectRetention

  • PutObjectLegalHold

  • GetObjectLegalHold

General limits

Item	Limits
Maximum number of buckets	10000
Maximum object size	5 TiB
Maximum number of parts per upload	10000
Part numbers	1 to 10000 (inclusive)
Part size	5 MiB to 5 GiB. The last part can be < 5 MiB
Maximum number of parts returned for a list parts request	1000
Maximum number of multipart uploads returned in a list multipart uploads request	1000
User-defined metadata per object	2 KB
Maximum number of tags per object	10
Maximum length of an object tag key	128 characters
Maximum length of an object tag value	256 characters
Maximum connections per server	2048
Maximum length of an S3 IAM user policy	2048
Maximum number of S3 IAM user policies	1024
Maximum number of S3 regular users	1024
Maximum number of S3 service accounts	5000
Maximum number of S3 STS credentials	5000

Bucket naming limitations

• Length: Bucket names must be between 3 and 63 characters long.

• Characters: Names can consist only of lowercase letters, numbers, dots (.), and hyphens (-).

• Structure: Names must begin and end with a letter or number.

• Format: Do not format bucket names as IP addresses, for example, 192.168.5.4.

• Uniqueness: Bucket names must be unique across the cluster.

Object naming limitations

WEKA follows AWS S3 object naming conventions with the following limitations.

• Key length: Object key names can be up to 1024 characters long.

• Nested objects: An object and a slash-extended variant of its name cannot coexist. For example, a and a/b cannot exist in the same bucket at the same time.

• For AWS naming convention details, see Creating object key names (AWS portal).

• Ensure the object key name is also compatible with protocols other than S3. Specifically, avoid special characters that might be unsupported in the other protocols.

Supported S3 policy actions

The S3 protocol implementation supports the following policy actions:

• Wildcard

  • s3:* (supported for IAM policies only)

• Bucket

  • s3:CreateBucket

  • s3:DeleteBucket

  • s3:GetBucketLocation

  • s3:ListAllMyBuckets

  • s3:ListBucket

• Object

  • s3:GetObject

  • s3:PutObject

  • s3:DeleteObject

• Multipart Upload

  • s3:AbortMultipartUpload

  • s3:ListBucketMultipartUploads

  • s3:ListMultipartUploadParts

• Bucket Policy

  • s3:GetBucketPolicy

  • s3:PutBucketPolicy

  • s3:DeleteBucketPolicy

• Lifecycle

  • s3:GetLifecycleConfiguration

  • s3:PutLifecycleConfiguration

• Bucket Tagging

  • s3:GetBucketTagging

  • s3:PutBucketTagging

• Object Tagging

  • s3:GetObjectTagging

  • s3:PutObjectTagging

  • s3:DeleteObjectTagging

• Object Lock

  • s3:PutObjectRetention

  • s3:GetObjectRetention

  • s3:PutObjectLegalHold

  • s3:GetObjectLegalHold

  • s3:BypassGovernanceRetention

  • s3:PutBucketObjectLockConfiguration

  • s3:GetBucketObjectLockConfiguration

• Versioning

  • s3:PutBucketVersioning

  • s3:GetBucketVersioning

  • s3:GetObjectVersion

  • s3:DeleteObjectVersion

  • s3:GetObjectVersionTagging

  • s3:PutObjectVersionTagging

  • s3:DeleteObjectVersionTagging

• Notifications

  • s3:GetBucketNotification

  • s3:PutBucketNotification

Supported AWS-aligned integrity algorithms

The WEKA system supports end-to-end checksum validation for S3 data integrity protections. The following AWS-aligned integrity algorithms are supported:

• CRC32

• CRC32C

• CRC64NVME

• SHA1

• SHA256

The system provides validation at the whole-object (FULL_OBJECT) and part-level, and supports trailer-based signed chunked uploads. Clients can select the desired algorithm on a per-request basis.

For multipart uploads, each algorithm supports specific checksum types:

• CRC32 and CRC32C: FULL_OBJECT or COMPOSITE.

• CRC64NVME: FULL_OBJECT. When creating the multipart upload, set the x-amz-checksum-type header to FULL_OBJECT. The default type is COMPOSITE.

• SHA1 and SHA256: COMPOSITE.

For optimal performance, CRC32 or CRC32C is recommended.

ETag handling in WEKA S3

The WEKA S3 cluster manages the S3 object ETag field using the MD5 algorithm to generate checksums based on the specific upload method. For standard single-part uploads, the returned ETag represents the MD5 checksum of the uploaded data.

In multipart uploads, the system generates an MD5 ETag for each part and ensures sequential validation, deriving the final object ETag from a checksum calculated across the individual parts' ETags.

Lifecycle configuration

WEKA supports the Amazon S3 Lifecycle Configuration elements and definitions, with the limitation of only supporting the lifecycle Expiration action.