S3 Users and Authentication

This page describes how to gain and obtain access permissions to the S3 protocol.

Overview

S3 User Role

To access the Weka cluster through the S3 protocol, a user with an S3 user role must be created (see Managing Users for details on creating users in Weka).

Note: The S3 user name and password will serve as the S3 access key and secret key, respectively.

When accessing data with S3 and other protocols (e.g., POSIX), it is possible to control the POSIX UID/GID of the underlying file representation of objects created with a specific S3 user access/secret keys. Use --posix-uid and --posix-gid flags for a local user with an S3 user role.

IAM Policy

Once an S3 user has been created, it cannot run any S3 command or API. The Cluster Admin must attach an IAM policy to allow this user to operate (within the policy limits).

A set of pre-defined policies can be attached to an S3 user, or new custom policies can be created and attached to an S3 user. To create a custom policy you can use AWS Policy Generator, and select IAM Policy as the policy type and Amazon S3 as the AWS service.

Note: The IAM policy size is limited to 2KB. In case a larger policy is required, please contact the Weka Support Team.

IAM Temporary Credentials (STS) - Assume Role

Once an S3 user is created and an IAM policy is attached, it is possible to gain temporary credentials to access the S3 API. This is done by calling the Assume Role command.

The result of calling the API is an access key, secret key, and session token tuple that can be used to access S3 APIs. The permissions for the temporary credentials will be the permissions induced by the user's IAM policy. Furthermore, it is possible to supply a different (with reduced capabilities only) IAM policy for the temporary credentials request.

Note: some S3 clients and SDKs (e.g., boto3) support using the AssumeRole API automatically when provided with an access key and secret key pair. They will automatically generate and use new temporary credentials tuple when the previous one expires.

Manage Users and Authentication

Viewing Existing IAM Policies

Command: weka s3 policy list

Use this command to list the existing IAM policies.

The command lists both the pre-defined policies and custom policies that the Cluster Admin has added.

Command: weka s3 policy show <policy-name>

Use this command to see the JSON definition of the selected IAM policy.

The pre-defined policies value are:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketLocation",
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::*"
      ]
    }
  ]
}

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::*"
      ]
    }
  ]
}

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:*"
      ],
      "Resource": [
        "arn:aws:s3:::*"
      ]
    }
  ]
}