S3 Buckets Management
This page describes how to manage S3 buckets.

Overview

Buckets can be managed by either standard S3 API calls or by using the Weka API/CLI.
Buckets permissions are determined by the user's IAM policy for authorized access or by setting bucket policies for anonymous access.
Currently, buckets and objects created through the S3 protocol will have root POSIX permissions. In addition, all buckets are created within the filesystem specified in the S3 cluster creation. Directories (adhering to the naming limitations) within this filesystem are exposed as buckets without anonymous permissions.

Managing Buckets using the CLI

Creating a New Bucket

Command: weka s3 bucket create
Use the following command line to create an S3 bucket:
weka s3 bucket create <name> [--policy policy] [--policy-json policy-json]
Parameters in Command Line
Name
Type
Value
Limitations
Mandatory
Default
name
String
The name for the new S3 bucket
Refer to the Bucket Naming Limitations section.
Yes
policy
String
The name of a pre-defined bucket policy for anonymous access.
One of: none, download, upload, public
No
none
policy-json
String
A path to a custom policy JSON file for anonymous access.
A JSON file representing an S3 bucket policy.
No

Listing Buckets

Command: weka s3 bucket list
Use this command to list existing buckets.

Deleting a Bucket

Command: weka s3 bucket destroy
Use this command to delete an existing bucket.
Note: A bucket can only be deleted if it is empty (all its objects have been deleted).

Managing Bucket Policies

It is possible to set bucket policies for anonymous access. You can choose one of the pre-defined policies or add your own customized policies.

Setting a Pre-Defined Bucket Policy

A bucket is automatically created without any anonymous access permissions. You can use one of the pre-defined policies: download, upload, or public.
For example, for a bucket named mybucket, these will be the pre-defined policies values:
download
upload
public
1
{
2
"Statement": [
3
{
4
"Action": [
5
"s3:GetBucketLocation",
6
"s3:ListBucket"
7
],
8
"Effect": "Allow",
9
"Principal": {
10
"AWS": [
11
"*"
12
]
13
},
14
"Resource": [
15
"arn:aws:s3:::mybucket"
16
]
17
},
18
{
19
"Action": [
20
"s3:GetObject"
21
],
22
"Effect": "Allow",
23
"Principal": {
24
"AWS": [
25
"*"
26
]
27
},
28
"Resource": [
29
"arn:aws:s3:::mybucket/*"
30
]
31
}
32
],
33
"Version": "2012-10-17"
34
}
Copied!
1
{
2
"Statement": [
3
{
4
"Action": [
5
"s3:GetBucketLocation",
6
"s3:ListBucketMultipartUploads"
7
],
8
"Effect": "Allow",
9
"Principal": {
10
"AWS": [
11
"*"
12
]
13
},
14
"Resource": [
15
"arn:aws:s3:::mybucket"
16
]
17
},
18
{
19
"Action": [
20
"s3:DeleteObject",
21
"s3:ListMultipartUploadParts",
22
"s3:PutObject",
23
"s3:AbortMultipartUpload"
24
],
25
"Effect": "Allow",
26
"Principal": {
27
"AWS": [
28
"*"
29
]
30
},
31
"Resource": [
32
"arn:aws:s3:::mybucket/*"
33
]
34
}
35
],
36
"Version": "2012-10-17"
37
}
Copied!
1
{
2
"Statement": [
3
{
4
"Action": [
5
"s3:GetBucketLocation",
6
"s3:ListBucket",
7
"s3:ListBucketMultipartUploads"
8
],
9
"Effect": "Allow",
10
"Principal": {
11
"AWS": [
12
"*"
13
]
14
},
15
"Resource": [
16
"arn:aws:s3:::mybucket"
17
]
18
},
19
{
20
"Action": [
21
"s3:ListMultipartUploadParts",
22
"s3:PutObject",
23
"s3:AbortMultipartUpload",
24
"s3:DeleteObject",
25
"s3:GetObject"
26
],
27
"Effect": "Allow",
28
"Principal": {
29
"AWS": [
30
"*"
31
]
32
},
33
"Resource": [
34
"arn:aws:s3:::mybucket/*"
35
]
36
}
37
],
38
"Version": "2012-10-17"
39
}
Copied!
Command: weka s3 bucket set-policy
Use the following command line to set a pre-defined bucket policy:
weka s3 bucket set-policy <bucket-policy> <bucket-name>
Parameters in Command Line
Name
Type
Value
Limitations
Mandatory
Default
bucket-policy
String
The name of a pre-defined bucket policy for anonymous access.
One of: none, download, upload, public
Yes
bucket-name
String
The name of an existing S3 bucket
Yes

Setting a Custom Bucket Policy

To create a custom policy, you can use AWS Policy Generator and select S3 Bucket Policy as the policy type. With a custom policy, it is possible to limit anonymous access only to specific prefixes.
For example, to set a custom policy for mybucket to allow read-only access for objects with a public/ prefix, the custom policy, as generated with the calculator, is:
1
{
2
"Id": "Policy1624778813411",
3
"Version": "2012-10-17",
4
"Statement": [
5
{
6
"Sid": "Stmt1624778790840",
7
"Action": [
8
"s3:ListBucket"
9
],
10
"Effect": "Allow",
11
"Resource": "arn:aws:s3:::mybucket",
12
"Condition": {
13
"StringEquals": {
14
"s3:prefix": "public/"
15
}
16
},
17
"Principal": "*"
18
},
19
{
20
"Sid": "Stmt1624778812360",
21
"Action": [
22
"s3:GetObject"
23
],
24
"Effect": "Allow",
25
"Resource": "arn:aws:s3:::mybucket/public/*",
26
"Principal": "*"
27
}
28
]
29
}
Copied!
Command: weka s3 bucket set-custom-policy
Use the following command line to set a custom bucket policy:
weka s3 bucket set-custom-policy <policy-file> <bucket-name>
Parameters in Command Line
Name
Type
Value
Limitations
Mandatory
Default
policy-file
String
A path to a custom policy JSON file for anonymous access.
A JSON file representing an S3 bucket policy.
Wildcards (e.g., s3:*) are not allowed as an Action in the custom policy file. For supported actions, refer to the Supported Policy Actions section.
Yes
bucket-name
String
The name of an existing S3 bucket.
Yes

Viewing a Bucket Policy

Command: weka s3 bucket get-policy / weka s3 bucket get-policy-json
Use the following command line to view an S3 bucket policy name/JSON:
weka s3 bucket get-policy <bucket-name> / weka s3 bucket get-policy-json <bucket-name>
Parameters in Command Line
Name
Type
Value
Limitations
Mandatory
Default
bucket-name
String
The name of an existing S3 bucket.
Yes
Last modified 1mo ago