During the deployment of the WEKA system, the EC2 instances require access to the internet to download the WEKA software. For this reason, you need to deploy the Weka system in one of the following deployment types in AWS:
Public subnet: Use a public subnet within your VPC with an internet gateway, and allow public IP addresses for your instances.
Private subnet with NAT Gateway: Create a private subnet with a route to a NAT gateway with an elastic IP in the public subnet.
Private subnet using Weka VPC endpoint: Requires the creation of a prerequisites stack (once per VPC) that creates the necessary resources.
Private subnet using custom proxy: Requires the creation of a prerequisites stack (once per VPC) that creates the necessary resources.
The following diagrams illustrate the components of the public subnet and private subnet with NAT gateway deployment types in AWS:
AWS subnet options for WEKA deployment
Update the number of vCPU limits in EC2
By default, AWS does not provide enough vCPUs to install a WEKA system. Use the Limits Calculator for your region from the AWS EC2 dashboard.
Procedure
1.
On the AWS EC2 dashboard, select the Limits option from the left menu.
EC2 Limits location
2. In the Limits Calculator, do the following:
In the Current Limit, set the number of vCPUs you currently have for a region.
In the vCPUs needed, set the required number of vCPUs for your specific deployment.
Select the Request on-demand limit increase link to get more vCPUs.
Note: vCPU increase is not an instant action and can take minutes to days for AWS to evaluate and approve your request.
The following example shows the required vCPUs for a six-node cluster with two clients of type i3en.2xlarge. This example is the smallest type of instance for a WEKA system deployment.
Limits Calculator
After the installation on AWS best practices
Backup and recovery
Resiliency
The Weka system is a distributed cluster protected from 2 or 4 failure domain failures, providing fast rebuild times as described in the Weka system overview section.
Instance failure
If an instance failure occurs, the Weka system rebuilds the data. Add a new instance to the cluster to regain the reduced compute and storage due to the instance failure.
Upload snapshots to S3
It is advisable to use periodic (incremental) snapshots to back up the data and protect it from multiple EC2 instances failures.
The recovery point objective (RPO) is determined by the cadence in which the snapshots are taken and uploaded to S3. The RPO changes between the type of data, regulations, and company policies, but it is advisable to upload at least daily snapshots (Snap-To-Object) of the critical filesystems.
If a failure occurs and it is required to recover from a backup, spin up a cluster using the Self-Service Portal or CloudFormation, and create filesystems from those snapshots. You do not need to wait for the data to reach the EC2 volumes. It is instantly accessible through S3. The recovery time objective (RTO) for this operation mainly depends on the time it takes to deploy the CloudFormation stack and is typically below 30 min.