Peer VPCs to enable traffic across VPCs
Datacoral recommends that you create a dedicated VPC for Datacoral services in order to provide isolation, auditing, and network access management. However, sometimes our services will need to communicate across different VPCs. There are a few situations when this might be needed:
- When a customer is installing Datacoral in a new VPC, but they wish to use an existing Redshift cluster that is an older VPC.
- When a customer has installed Datacoral in its own VPC, and they would like Datacoral services to communicate with services/databases in a separate VPC.
In such cases, VPC Peering will have to be performed. We should make sure that the two VPCs don't have an overlapping CIDR Block.
There are three parts -- Sending a peering request, accepting it and configuring it.
Creating and sending a peering request
From the AWS account that involves the VPC that is running Datacoral, a connection request needs to be sent to the other VPC.
Step 1: Go to the VPC Console.
Step 2: Click on Create Peering Connections button
Step 3: Enter Requesting and Accepting VPC information
You can choose whether the receiving VPC is in the same or different account (account number will be needed). Press the Create Peering Connection button in the bottom-right.
After this, a success message will be shown.
For more info, and details on using the AWS CLI, see here if the VPC is in the same AWS account or here if the VPC is in a different AWS account.
AWS CLI: create-vpc-peering-connection
if requester VPC is in same account you can ignore peer-owner-id or pass your own account ID for more info, for more info see here.
Example Command:
Output:
Accepting a Peering Request
Now navigate to the AWS account to which a peering request has been sent. The request will have to be accepted here.
Step 1: Go to the VPC Console.
Here in the VPC console, you'll see the peering connection will show up with a yellow circle which means that the connection is pending.
Step 2: Click on Actions for Pending Peering Request and Accept.
Click on the pending Peering Request, and then click on Actions in the panel above it. This will open a drop-down as seen below. Click on "Accept Request".
Step 3: Accept Peering Request.
In the modal that opens, click on "Yes, Accept".
After a few seconds, we will be able to see that the peering connection is successful.
AWS CLI: accept-vpc-peering-connection
Accept a VPC peering connection request. To accept a request, the VPC peering connection must be in the pending-acceptance state, and you must be the owner of the peer VPC. Use DescribeVpcPeeringConnections to view your outstanding VPC peering connection requests.
For an inter-Region VPC peering connection request, you must accept the VPC peering connection in the Region of the accepter VPC, for more info see here.
Example Command: Using the output command from the create-vpc-peering-connection get VpcPeeringConnectionId
Output:
VPC Peering Configurations
You need to configure VPC peering connections so that your route tables have access to the entire CIDR block of the peer VPC. For more info, and details on the VPC Peering Configurations, see here
Step 1: Enable DNS resolution on VPC.
Enable a VPC to resolve public IPv4 DNS hostnames to private IPv4 addresses when queried from instances in the peer VPC. Both VPCs (Accepter and Requester) must be enabled for DNS hostnames and DNS resolution.
i. To edit DNS hostnames and DNS resolution go to VPC dashboard.
ii. Click on edit DNS hostnames and make sure the enable is checked and save.
iii. Click on edit DNS resolution and make sure the enable is checked and save.
AWS CLI: modify-vpc-attribute
Modifies the specified attribute of the specified VPC, for more info see here.
Example Command: Run these commands for both Accepter VPC and Requester VPC
Step 2: Enable DNS resolution on Peering Connection.
To enable a VPC to resolve public IPv4 DNS hostnames to private IPv4 addresses when queried from instances in the peer VPC, you must modify the peering connection.
i. To Edit DNS settings on peering connection choose Peering Connections, in the navigation pane.
ii. To ensure that queries from the peer VPC resolve to private IP addresses in your local VPC, choose the option to enable DNS resolution for queries from the peer VPC. This option is Requester DNS resolution or Accepter DNS resolution, depending on whether the VPC is the requester or accepter VPC.
Accepter AWS account
Requester AWS account
iii. If the peer VPC is in the same AWS account, you can enable DNS resolution for both VPCs in the peering connection.
Once the DNS resolution is setup, you will have a screen that looks like below on the the acceptor AWS Account.
AWS CLI: modify-vpc-peering-connection-options
Modifies the VPC peering connection options on one side of a VPC peering connection, for more info see here.
Example Commands:
Run from accepter account/region
Run from requester account/region
If both the VPC's are in same account and same region, you can run single command
iv. Choose Save.
- If the peer VPC is in a different AWS account or a different region, the owner of the peer VPC must sign into the VPC console, perform Step 4, and choose Save.
Step 3: Updating Your Route Tables for a VPC Peering Connection
To send private IPv4 traffic from your instance to an instance in a peer VPC, you must add a route to the route table that's associated with your subnet in which your instance resides. The route points to the CIDR block (or portion of the CIDR block) of the peer VPC in the VPC peering connection, and specifies the VPC peering connection as the target. for more info see here.
In VPC dashboard, click on route tables and search with VPC Resource Attributes and edit all route tables to add a route.
Add a route with destination as accepter CIDR block if you are requester, and vice versa. In Target add peering connection id and click save routes.
- Repeat the above step for all the route tables in the VPC.
- Do the same in the Accepter VPC.
AWS CLI: create-route
Creates a route in a route table within a VPC, for more info see here.
Example Command:
To get all route table ids in a VPC, for more info see here.
Add route in requester VPC route table
Add route in accepter VPC route table
Output:
Step 4: Verify VPC Peering connection
In order to test whether VPC Peering has been successful or not, the best way is to spin up an EC2 machine inside a private subnet in the account that houses the Datacoral installation and connect to the Redshift/Postgres/MySQL/MongoDB instance (dbhostname) using the telnet command.
dbport can be different for different database types. Defaults for different databases are:
- Redshift - 5439
- Postgres - 5432
- MySQL - 3306
- MongoDB - 27017