Guides

AWS Private Link

AWS PrivateLink allows two AWS environments to connect while routing traffic within AWS. This prevents traffic being exposed to the internet while also reducing egress costs. Learn more about AWS PrivateLink

AWS PrivateLink can also be used to access on-premise databases by also utilising AWS Direct Connect. Learn more about AWS Direct Connect

AWS PrivateLink can be used for all sources and destinations within AWS.

Create the Endpoint Service

Here is a link to AWS Private Link Docs on creating an Endpoint Service

  1. Create Network Load Balancer (NLB) within your VPC and configure it for each subnet/availability zone if it does not exist already
  2. Create a VPC endpoint service and choose your Network Load Balancer (NLB)
  3. Safelist Streamkap's AWS VPC Account ID (arn:aws:iam::300973880807:root). This will allow inbound access from our AWS PrivateLink
  4. Send the service name generated here to Streamkap so that we setup our AWS PrivateLink to talk to yours.
  5. Update Acceptance Settings to your preference. If you disable it, Streamkap will be able to connect automatically without permission. You have already safelisted our account in earlier steps
  6. Send the service name to Streamkap to generate the connection

IP & Port Forwarding

Network Load Balancers (NLB) can route traffic to an EC2 instance, an IP address, or a Lambda function through target groups.

AWS RDS Aurora databases give you an endpoint to access your database. This endpoint resolves to an IP address but unfortunately, this internal IP is subject to change.

There are a couple of options available to work with this.

Port Forwarder

You must deploy an EC2 instance that is configured to do port forwarding (accepting requests from the NLB and forwarding those requests to the RDS database). Here is a sample script that you can use to set up the EC2 port forwarding instance:

All Except Amazon Linux 2023

#!/bin/bash
PREVLOGFILE=/root/ip.txt # Note the below section of the code is important in the event of a server restart.
if test -f "$PREVLOGFILE"; then
  truncate -s 0 $PREVLOGFILE
  echo "State file $PREVLOGFILE has been emptied"
fi
python -m SimpleHTTPServer 801 & # NOTE: USE PORT 801 FOR <HEALTH_CHECKS> PARAMETER BELOW
echo 1 -> /proc/sys/net/ipv4/ip_forward
export RDS_ENDPOINT=<<PROSPECT RDS INSTANCE ENDPOINT>> #NOTE: DO NOT INCLUDE THE <<>> CHARACTERS, NO QUOTATION MARKS.
export RDS_PORT=<<PROSPECTS RDS INSTANCE PORT>> #NOTE: DO NOT INCLUDE THE <<>> CHARACTERS, NO QUOTATION MARKS.
iptables -t nat -A POSTROUTING -j MASQUERADE
while true
do
LOGFILE=/root/ip.txt
Current_IP=$(dig +short $RDS_ENDPOINT | tail -n1) #NOTE: THE "/ TAIL -n1" piece is critical to ensure only the IP address of the RDS instnce is picked.
if [ $LOGFILE = "" ] ; then
  iptables -I INPUT -i eth1 -s $Current_IP -j ACCEPT
  echo $Current_IP > $LOGFILE
else
  Old_IP=$(cat $LOGFILE)
  if [ "$Current_IP" = "$Old_IP" ] ; then
    echo "IP address has not changed ($Old_IP -> $Current_IP)"
  else
    iptables -t nat -D PREROUTING -p tcp --dport 80 -j DNAT --to-destination $Old_IP:$RDS_PORT
    iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination $Current_IP:$RDS_PORT
    sysctl net.ipv4.ip_forward=1
    iptables-save
    echo $Current_IP > $LOGFILE
    echo "IP address has changed ($Old_IP -> $Current_IP)"
  fi
fi
sleep 5
done

Amazon Linux 2023

#!/bin/bash
PREVLOGFILE=/root/ip.txt # Note the below section of the code is important in the event of a server restart.
if test -f "$PREVLOGFILE"; then
  truncate -s 0 $PREVLOGFILE
  echo "State file $PREVLOGFILE has been emptied"
fi
python3 -m http.server 801 & # NOTE: USE PORT 801 FOR <HEALTH_CHECKS> PARAMETER BELOW
echo 1 -> /proc/sys/net/ipv4/ip_forward
export RDS_ENDPOINT=<<PROSPECT RDS INSTANCE ENDPOINT>> #NOTE: DO NOT INCLUDE THE <<>> CHARACTERS, NO QUOTATION MARKS.
export RDS_PORT=<<PROSPECTS RDS INSTANCE PORT>> #NOTE: DO NOT INCLUDE THE <<>> CHARACTERS, NO QUOTATION MARKS.
iptables -t nat -A POSTROUTING -j MASQUERADE
while true
do
LOGFILE=/root/ip.txt
Current_IP=$(dig +short $RDS_ENDPOINT | tail -n1) #NOTE: THE "/ TAIL -n1" piece is critical to ensure only the IP address of the RDS instnce is picked.
if [ $LOGFILE = "" ] ; then
  iptables -I INPUT -i eth1 -s $Current_IP -j ACCEPT
  echo $Current_IP > $LOGFILE
else
  Old_IP=$(cat $LOGFILE)
  if [ "$Current_IP" = "$Old_IP" ] ; then
    echo "IP address has not changed ($Old_IP -> $Current_IP)"
  else
    iptables -t nat -D PREROUTING -p tcp --dport 80 -j DNAT --to-destination $Old_IP:$RDS_PORT
    iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination $Current_IP:$RDS_PORT
    sysctl net.ipv4.ip_forward=1
    iptables-save
    echo $Current_IP > $LOGFILE
    echo "IP address has changed ($Old_IP -> $Current_IP)"
  fi
fi
sleep 5
done
content_copy

Once you’ve finished setting up the port forwarding instance, configure the NLB listener and target group to route traffic to the portforwarder EC2 instance.

If you chose to use security groups with your NLB, you must allow Streamkap's IPs corresponding to the selected AWS region on Network ACLs and Security Groups. See Streamkap IP Addresses

Lambda Function

It's possible to deploy a lambda function check the current IP address and update the NLB target group accordingly.

To use the RDS IP address in your NLB target group, do the following:

  1. Run nslookup with the domain name of RDS endpoint as the input to find the IP address:
  • nslookup DNS_ENDPOINT
  1. Configure your NLB target group with the IP address from above
  2. Deploy a lambda function to periodically perform nslookup on the RDS endpoint to see if the IP address has changed and update the target group with the new IP address.

Updated 7 months ago