Weka installation on AWS
This section provides the detailed instructions on how to install a Weka system on AWS.
Last updated
This section provides the detailed instructions on how to install a Weka system on AWS.
Last updated
If you already have an AWS account and are familiar with AWS's basic concept and services, you can skip this section.
To install a Weka system in AWS, you need to .
Make sure you are familiar with the following concepts and services that are used as part of the Weka system deployment:
- Identity and access management
, , and
instances and
- Object store (to be used for tiering data)
During the deployment of the WEKA system, the EC2 instances require access to the internet to download the WEKA software. For this reason, you need to deploy the Weka system in one of the following deployment types in AWS:
Public subnet: Use a public subnet within your VPC with an internet gateway, and allow public IP addresses for your instances.
Private subnet with NAT Gateway: Create a private subnet with a route to a NAT gateway with an elastic IP in the public subnet.
Private subnet using Weka VPC endpoint: Requires the creation of a (once per VPC) that creates the necessary resources.
Private subnet using custom proxy: Requires the creation of a (once per VPC) that creates the necessary resources.
The following diagrams illustrate the components of the public subnet and private subnet with NAT gateway deployment types in AWS:
By default, AWS does not provide enough vCPUs to install a WEKA system. Use the Limits Calculator for your region from the AWS EC2 dashboard.
Procedure
On the AWS EC2 dashboard, select the Limits option from the left menu.
2. In the Limits Calculator, do the following:
In the Current Limit, set the number of vCPUs you currently have for a region.
In the vCPUs needed, set the required number of vCPUs for your specific deployment.
Select the Request on-demand limit increase link to get more vCPUs.
The following example shows the required vCPUs for a six-node cluster with two clients of type i3en.2xlarge. This example is the smallest type of instance for a WEKA system deployment.
It is advisable to use periodic (incremental) snapshots to back up the data and protect it from multiple EC2 instances failures.
Using Weka snapshots uploaded to S3 combined with S3 cross-region replication enables the protection from an AWS region failure.
For security reasons, it is advisable to rotate the SSH keys used for the EC2 instances.
To rotate the SSH keys, follow these steps:
The Weka system is a distributed cluster protected from 2 or 4 failure domain failures, providing fast rebuild times as described in the section.
If an instance failure occurs, the Weka system the data. to regain the reduced compute and storage due to the instance failure.
The recovery point objective (RPO) is determined by the cadence in which the snapshots are taken and uploaded to S3. The RPO changes between the type of data, regulations, and company policies, but it is advisable to upload at least daily snapshots () of the critical filesystems.
If a failure occurs and it is required to recover from a backup, spin up a cluster using the or , and create filesystems from those snapshots. You do not need to wait for the data to reach the EC2 volumes. It is instantly accessible through S3. The recovery time objective (RTO) for this operation mainly depends on the time it takes to deploy the CloudFormation stack and is typically below 30 min.
See .
, and
.
