Hybrid connectivity
Direct Connect, Site-to-Site VPN and Client VPN, how to make them resilient, and how BGP decides which path traffic takes.
Exam tasks: 1.1 (design hybrid connectivity, evaluate resilient network designs), 4.2
The decision: how much bandwidth and consistency do you need, how fast must the link exist, and what happens when one path fails?
Choosing a link
Side by side
| Site-to-Site VPN | Direct Connect dedicated | Direct Connect hosted | |
|---|---|---|---|
| Path | Encrypted IPsec over the internet | Private fiber to a DX location | Partner's port, carved into a slice for you |
| Time to set up | Minutes to hours | Weeks (cross-connect, LOA-CFA) | Days to weeks, through the partner |
| Bandwidth | 1.25 Gbps per standard tunnel | 1, 10, 100 or 400 Gbps ports | From 50 Mbps up, set by the partner |
| Latency and jitter | Variable (internet) | Consistent | Consistent |
| Encryption | Always (IPsec) | None by default. MACsec or VPN on top | None by default. VPN on top |
| VIFs | N/A | Up to 50 public/private + 4 transit | Exactly one |
| Cost shape | Per connection-hour + normal data out | Port-hour + reduced data-transfer-out rate | Same, billed by the partner and AWS |
Direct Connect building blocks
Virtual interfaces (VIFs)
| VIF | Reaches | Terminates on | Use it for |
|---|---|---|---|
| Private | One VPC's private IPs | A virtual private gateway, or a DX gateway | A few VPCs |
| Transit | Many VPCs through a Transit Gateway or Cloud WAN | A DX gateway associated with the TGW | Dozens or hundreds of VPCs |
| Public | AWS public endpoints (S3, DynamoDB, public service APIs) in all Regions | AWS edge | Reaching public AWS services without the internet. It doesn't give internet access |
Direct Connect gateway
- A global object. One VIF to a DX gateway reaches VGWs or Transit Gateways in any Region (except China).
- Associate up to 20 VGWs, or up to 6 Transit Gateways, per DX gateway.
- It is not a transit router between VPCs. Two VGWs on the same DX gateway can't talk to each other through it.
- Allowed prefixes on each association control what gets advertised back to on-premises.
LAG, SiteLink and MACsec
- LAG (link aggregation group): bundles dedicated connections into one logical link with LACP. Set a minimum links value so the whole LAG goes down, and traffic fails over, if too few members are healthy.
- SiteLink: lets two DX locations send traffic to each other across the AWS backbone without going through a Region. Turn it on per VIF. Good for linking branch offices or data centers through DX.
- MACsec: IEEE 802.1AE encryption between your router and the AWS device, on dedicated connections at supported speeds and locations. Hosted connections can't use it.
Encrypting Direct Connect
DX isn't encrypted. For line-rate, layer 2 encryption on a dedicated 10/100/400 Gbps port, pick MACsec. For encryption on any DX (including hosted), run Site-to-Site VPN over DX: a public VIF to the VPN endpoints, or a private IP VPN over a transit VIF to a Transit Gateway so nothing uses public IP addresses.
Direct Connect resiliency models
| Model | Layout | Survives | SLA |
|---|---|---|---|
| Maximum resiliency | Two locations, two connections on separate devices in each | Losing a device and a whole location | 99.99% |
| High resiliency | One connection in each of two locations | Losing a location | 99.9% |
| Development and test | Two connections on separate devices in one location | Losing a device, not a location | None for location failure |
| Single DX + VPN backup | One DX plus a Site-to-Site VPN | Losing DX, at VPN bandwidth | VPN has no DX SLA |
Two connections, one location
Two DX connections in the same location, or a LAG, don't protect against losing the location. If a question says "critical" or asks for the highest availability, the answer uses two DX locations.
Site-to-Site VPN
- VGW termination: one VPC per connection. The VGW picks one active tunnel across all its VPNs. No ECMP.
- TGW termination: one connection serves every attached VPC. With BGP and ECMP, traffic spreads across tunnels and connections, so four connections give up to 8 tunnels of aggregate bandwidth.
- Accelerated VPN: tunnels enter the AWS network at the nearest Global Accelerator edge location instead of crossing the internet all the way to the Region. Transit Gateway or Cloud WAN only, and can't be enabled on an existing connection.
- Customer gateway: your device, or a software appliance. Use dynamic (BGP) routing when you need ECMP or automatic failover.
ECMP on a virtual private gateway
Adding more VPN connections to a VGW doesn't add bandwidth, because the VGW uses one tunnel at a time. To aggregate VPN tunnels, terminate them on a Transit Gateway with dynamic routing and ECMP turned on.
Route priority and BGP
Inside AWS: which path does AWS pick?
- Longest prefix match always wins first. The VPC's local route beats everything, even more specific propagated routes.
- For identical prefixes, static routes in the VPC route table beat propagated routes.
- For identical prefixes arriving at a VGW: DX BGP routes, then static VPN routes, then BGP VPN routes.
- Between BGP VPN routes: shortest AS_PATH, then lowest MED.
So with the same prefix on DX and VPN, AWS sends return traffic over DX automatically. A more specific prefix over VPN would pull traffic onto the VPN.
From on-premises: influencing AWS and your own routers
| Goal | Tool | How |
|---|---|---|
| Make AWS prefer one DX VIF over another for the same prefix | Local preference BGP communities | Tag routes you advertise with 7224:7300 (high), 7224:7200 (medium) or 7224:7100 (low) |
| Same, less preferred | AS_PATH prepending | Prepend your ASN on the backup path. Local preference communities are evaluated first |
| Steer traffic to AWS | Your router's local preference | Prefer routes learned on the DX session over the VPN session |
| Limit how far AWS advertises your public prefixes (public VIF) | Scope communities | 7224:9100 local Region, 7224:9200 continent, 7224:9300 global |
| Force a specific path regardless of attributes | More specific prefixes | Longest prefix match beats every BGP attribute |
Active/passive across two DX locations
Advertise the same prefixes on both VIFs, tag the primary with 7224:7300 and the secondary with 7224:7100 (or
prepend AS_PATH on the secondary). For active/active, advertise identical prefixes with identical attributes.
Client VPN
A managed OpenVPN-based service for individual users. It associates with subnets in a VPC and authenticates with Active Directory, SAML federation or mutual certificates. Authorization rules limit which CIDRs each group can reach. Split tunnel sends only AWS-bound traffic through the VPN. From that VPC, users can reach peered VPCs, a Transit Gateway or on-premises if routes and authorization rules allow it.
Scenarios
A BGP VPN on the same TGW is the cheapest backup. With identical prefixes, AWS prefers the Direct Connect path, and BGP fails over automatically if DX goes down. More specific VPN routes would pull all traffic onto the VPN, since longest prefix match wins. A second connection in the same location costs far more and shares the location's failure modes. A public VIF on the same physical connection fails with it.
The maximum resiliency model, two locations with separate devices in each, carries the 99.99% SLA. MACsec encrypts at line rate on dedicated 100 Gbps ports. A single-location LAG can't survive losing the location. VPN over a public VIF uses public IPs and each tunnel tops out far below 100 Gbps. Hosted connections don't support MACsec.
A VGW sends traffic over one tunnel at a time, whatever the routing type or number of connections. A Transit Gateway with BGP and ECMP spreads traffic across all tunnels. Acceleration only exists on TGW or Cloud WAN attachments and can't be turned on for an existing connection.