WEKA installation on AWS using the Cloud Formation
Last updated
Last updated
During the deployment of the WEKA system, the EC2 instances require access to the internet to download the WEKA software. For this reason, you need to deploy the WEKA system in one of the following deployment types in AWS:
Public subnet: Use a public subnet within your VPC with an internet gateway, and allow public IP addresses for your instances.
Private subnet with NAT Gateway: Create a private subnet with a route to a NAT gateway with an elastic IP in the public subnet.
Private subnet using WEKA VPC endpoint: Requires the creation of a (once per VPC) that creates the necessary resources.
Private subnet using custom proxy: Requires the creation of a (once per VPC) that creates the necessary resources.
The following diagrams illustrate the components of the public subnet and private subnet with NAT gateway deployment types in AWS.
By default, AWS does not provide enough vCPUs to install a WEKA system. Use the Limits Calculator for your region from the AWS EC2 dashboard.
Procedure
On the AWS EC2 dashboard, select the Limits option from the left menu.
2. In the Limits Calculator, do the following:
In the Current Limit, set the number of vCPUs you currently have for a region.
In the vCPUs needed, set the required number of vCPUs for your specific deployment.
Select the Request on-demand limit increase link to get more vCPUs.
The following example shows the required vCPUs for a six servers cluster with two clients of type i3en.2xlarge. This example is the smallest type of instance for a WEKA system deployment.
If an instance failure occurs, the Weka system rebuilds the data. Add a new instance to the cluster to regain the reduced compute and storage due to the instance failure.
It is advisable to use periodic (incremental) snapshots to back up the data and protect it from multiple EC2 instances failures.
Using Weka snapshots uploaded to S3 combined with S3 cross-region replication enables protection from an AWS region failure.
For security reasons, it is advisable to rotate the SSH keys used for the EC2 instances.
To rotate the SSH keys, follow these steps:
Related topic
The Weka system is a distributed cluster protected from 2 or 4 failure domain failures, providing fast rebuild times. For details, see the section.
The recovery point objective (RPO) is determined by the cadence in which the snapshots are taken and uploaded to S3. The RPO changes between the type of data, regulations, and company policies, but it is advisable to upload at least daily snapshots of the critical filesystems. For details, see the section.
If a failure occurs and it is required to recover from a backup, spin up a cluster using the or , and create filesystems from those snapshots. You do not need to wait for the data to reach the EC2 volumes. It is instantly accessible through S3. The recovery time objective (RTO) for this operation mainly depends on the time it takes to deploy the and is typically below 30 min.
See the section.
, and
.
