Bootstrapping Private Subnet Instances In A VPC with Knife

25 Dec 2012 ~ 1 minute read

Amazon VPC

Amazon Virtual Private Cloud (VPC) is a service that allows you to define an isolated virtual network within EC2. A common scenario involves a VPC with both public and private subnets. Instances within public subnets can send and receive traffic directly to/from the Internet. On the other hand, instances within private subnets cannot receive traffic directly from the Internet and can only send outbound traffic via a NAT instance.

Bastion Host

Given a VPC setup with both public and private subnets, you’ll want at least one SSH bastion host in the public subnet. This host is needed to communicate with instances in the private subnet from your local machine. The diagram below, taken from Amazon’s documentation, helps illustrate:

SSH Bastion with VPC

Knife EC2 Example

Using a combination of Knife and the Knife EC2 plug-in, the following command connects directly to the bastion host specified by the --ssh-gateway option. From there another connection is made to the private subnet instance via its private_ip_address in order to bootstrap Chef:

knife ec2 server create --flavor hi1.4xlarge --image ami-08249861   \
  --security-group-ids [SECURITY_GROUP_ID] --tags Name=node1-dev    \
  --availability-zone us-east-1d --subnet [SUBNET_ID]               \
  --node-name node1-dev --ssh-key orgname --ssh-gateway bastion-dev \
  --server-connect-attribute private_ip_address                     \
  --ssh-user ec2-user --identity-file ~/.ec2/orgname.pem            \
  --environment development --ephemeral '/dev/sdb,/dev/sdc'         \
  --run-list 'role[base],role[solr_ssd_slave]'

Depending on how long it takes your run list to converge on a bare operating system, you should have Chef bootstrapped on an instance within the private subnet of a VPC after running only one command!