This page describes the Weka implementation of the S3 protocol.
The S3 protocol is widely used and spans many cloud-native or cloud-ready applications.
With Weka, you can:
Ingest data with S3, and then you can access the data with either S3 or other protocols.
Expose existing data to S3, and migrate your application within the same data platform.
Burst to the cloud and use new applications without migrating your data.
While gaining Weka's scale, performance, and resiliency advantages, you can gradually move applications to S3 and access the same data through multiple protocols, POSIX, S3, SMB, NFS, and GPUDirect Storage.
The Weka S3 is a scalable and resilient service that provides multi-protocol access to data.
You implement the S3 service by specifying a set of storage hosts that run the S3 protocol and then creating a logical S3 cluster to expose the S3 service. When defining many hosts that serve the S3 protocol, the S3 cluster scales to higher performance.
By integrating a round-robin DNS or a load balancer, different S3 clients can access other hosts, allowing the Weka system to scale and service thousands of clients.
The Weka S3 service works on top of the WekaFS file service. Buckets are mapped to top-level directories, and objects are mapped to files. Then, the same data can be exposed with either Weka-supported protocols.