CareerByteCode
ByteBlogby CareerByteCode

Site-to-Site VPN Between Two Google Cloud Regions

What you build: HA VPN with BGP between two Google Cloud networks in two regions, failover when a tunnel dies, and a wrong-secret fix. Every step tested.

Site-to-Site VPN Between Two Google Cloud Regions

✍️ Translate this blog into Hindi, Tamil or Telugu

A site-to-site VPN joins two private networks over the internet so their machines talk on private addresses, encrypted. Companies use it to connect an office or a data centre to the cloud. In this lab you build Google Cloud's HA VPN between two separate networks in two regions (playing "head office" and "branch"), let Cloud Router learn the routes with BGP, prove traffic flows, then pull a tunnel out and give another one the wrong secret to see what high availability really means.

Who it is for: cloud and network engineers who want to see VPN and BGP work, not just read about them. Time: about 1 hour. Cost: HA VPN tunnels and two small VMs for about 20 minutes, covered by the free trial credit. Delete everything at the end: tunnels cost by the hour.

Every command below was run for real by the CareerByteCode Team, and every output is the real result we got. We ran the gcloud commands from a Cloud Shell-like Linux machine; Cloud Shell in your browser works the same.

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →

Part A: VPN, HA VPN and BGP in plain English

HA VPN between two networks
Two networks with an HA VPN gateway and a Cloud Router each, joined by four tunnels; BGP tells each side the other's ranges

Words you will use:

  • VPC network: a private network in Google Cloud. Ours: vpc-office (10.10.0.0/24, us-east1) and vpc-branch (10.20.0.0/24, us-central1). The ranges must not overlap.
  • HA VPN gateway: Google's VPN endpoint. It has two interfaces, each with its own public IP, so one can fail while the other carries traffic.
  • Tunnel: one encrypted connection between two interfaces, set up with IKE (the handshake) and a shared secret both sides know. We build four: two from each side.
  • Cloud Router + BGP: instead of typing routes by hand, each Cloud Router tells the other which ranges it can reach (BGP, the routing protocol of the internet). Each router has an ASN (a number that names it in BGP): 65001 and 65002.
  • Link addresses: each BGP session talks over a tiny private range like 169.254.0.0/30, inside the tunnel.
  • Real-life example: HA VPN is two guarded bridges between two cities. BGP is the traffic office on each side that tells drivers which streets lie across the river. If one bridge closes, traffic uses the other.

Before you start

A Google Cloud project with billing linked (the free trial works) and Cloud Shell open, with PROJECT_ID set to your project id and the Compute Engine API on.

Part B: the lab

Step 1: two separate networks

Why: the two networks play two sites. Each gets one subnet and a firewall rule that allows ping and SSH from the other site's range and from Google's IAP range (for SSH without public IPs).

# Two separate networks that play "head office" (us-east1) and "branch" (us-central1). Their ranges must not overlap
gcloud compute networks create vpc-office --subnet-mode=custom > /dev/null && gcloud compute networks subnets create office-sub --network=vpc-office --region=us-east1 --range=10.10.0.0/24 > /dev/null
gcloud compute networks create vpc-branch --subnet-mode=custom > /dev/null && gcloud compute networks subnets create branch-sub --network=vpc-branch --region=us-central1 --range=10.20.0.0/24 > /dev/null
for n in office branch; do gcloud compute firewall-rules create $n-allow-vpn --network=vpc-$n --allow=icmp,tcp:22 --source-ranges=10.10.0.0/24,10.20.0.0/24,35.235.240.0/20 > /dev/null; done
gcloud compute networks subnets list --filter="name~^(office|branch)-sub" --format="table(name,region.basename(),ipCidrRange)"
Created [https://www.googleapis.com/compute/v1/projects/your-project-id/global/networks/vpc-office].

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network vpc-office --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network vpc-office --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/your-project-id/regions/us-east1/subnetworks/office-sub].
Created [https://www.googleapis.com/compute/v1/projects/your-project-id/global/networks/vpc-branch].

Instances on this network will not be reachable until firewall rules
are created. As an example, you can allow all internal traffic between
instances as well as SSH, RDP, and ICMP by running:

$ gcloud compute firewall-rules create <FIREWALL_NAME> --network vpc-branch --allow tcp,udp,icmp --source-ranges <IP_RANGE>
$ gcloud compute firewall-rules create <FIREWALL_NAME> --network vpc-branch --allow tcp:22,tcp:3389,icmp

Created [https://www.googleapis.com/compute/v1/projects/your-project-id/regions/us-central1/subnetworks/branch-sub].
Creating firewall...
..Created [https://www.googleapis.com/compute/v1/projects/your-project-id/global/firewalls/office-allow-vpn].
done.
Creating firewall...
..Created [https://www.googleapis.com/compute/v1/projects/your-project-id/global/firewalls/branch-allow-vpn].
done.

Step 2: a test VM in each network

Why: something to ping. Neither VM has a public IP; the only way between them will be the VPN.

# One small test VM in each network, with no public IP: the only way between them will be the VPN
gcloud compute instances create office-vm --zone=us-east1-b --machine-type=e2-micro --subnet=office-sub --no-address --image-family=debian-12 --image-project=debian-cloud > /dev/null 2>&1
gcloud compute instances create branch-vm --zone=us-central1-a --machine-type=e2-micro --subnet=branch-sub --no-address --image-family=debian-12 --image-project=debian-cloud > /dev/null 2>&1
gcloud compute instances list --filter="name~-vm$" --format="table(name,zone.basename(),networkInterfaces[0].networkIP)"
NAME       ZONE           NETWORK_IP
branch-vm  us-central1-a  10.20.0.2
office-vm  us-east1-b     10.10.0.2

Step 3: before the VPN, no route

# Before the VPN: the office VM cannot reach the branch VM (different networks, no route)
until gcloud compute ssh office-vm --zone=us-east1-b --tunnel-through-iap --command="true" > /dev/null 2>&1; do sleep 10; done
gcloud compute ssh office-vm --zone=us-east1-b --tunnel-through-iap --command="ping -c 3 -W 2 10.20.0.2 | tail -2" 2>/dev/null
3 packets transmitted, 0 received, 100% packet loss, time 2054ms
Networks and VMs
Two test VMs without public IPs; no route between the networks yet

100 percent packet loss: two separate networks know nothing about each other.

Step 4: HA VPN gateways and Cloud Routers

Why: one gateway and one Cloud Router per side, each router with its own ASN. Each gateway gets two public IPs (interfaces 0 and 1).

# HA VPN: one gateway per side (each gets TWO public IPs, interface 0 and 1) and one Cloud Router per side for BGP
gcloud compute vpn-gateways create gw-office --network=vpc-office --region=us-east1 > /dev/null
gcloud compute vpn-gateways create gw-branch --network=vpc-branch --region=us-central1 > /dev/null
gcloud compute routers create rt-office --network=vpc-office --region=us-east1 --asn=65001 > /dev/null
gcloud compute routers create rt-branch --network=vpc-branch --region=us-central1 --asn=65002 > /dev/null
gcloud compute vpn-gateways list --format="table(name,region.basename(),vpnInterfaces.len():label=PUBLIC_IPS)"
gcloud compute routers list --filter="name~^rt-" --format="table(name,region.basename(),bgp.asn)"
Creating VPN Gateway...
........done.
Creating VPN Gateway...
.........done.
Creating router [rt-office]...
..........done.
Creating router [rt-branch]...
..........done.
NAME       REGION       PUBLIC_IPS
gw-branch  us-central1  2
gw-office  us-east1     2
NAME       REGION       ASN
rt-branch  us-central1  65002
rt-office  us-east1     65001

Step 5: four tunnels

Why: interface 0 to interface 0 and 1 to 1, from both sides. The shared secret is made in the lab with openssl rand and never printed. The peer gateway is given by its full resource path (projects/.../regions/.../vpnGateways/...); with the current gcloud, giving its name plus a separate region flag was refused in our first try.

# Four tunnels (two from each side, one per gateway interface) with one shared secret made in the lab, never printed
openssl rand -base64 24 > .vpn-secret
for i in 0 1; do
  gcloud compute vpn-tunnels create office-to-branch-$i --region=us-east1 --vpn-gateway=gw-office --interface=$i --peer-gcp-gateway=projects/$PROJECT_ID/regions/us-central1/vpnGateways/gw-branch --router=rt-office --ike-version=2 --shared-secret="$(cat .vpn-secret)" > /dev/null
  gcloud compute vpn-tunnels create branch-to-office-$i --region=us-central1 --vpn-gateway=gw-branch --interface=$i --peer-gcp-gateway=projects/$PROJECT_ID/regions/us-east1/vpnGateways/gw-office --router=rt-branch --ike-version=2 --shared-secret="$(cat .vpn-secret)" > /dev/null
done
gcloud compute vpn-tunnels list --format="table(name,region.basename(),status)"
Creating VPN tunnel...
..........done.
Creating VPN tunnel...
..........done.
Creating VPN tunnel...
..........done.
Creating VPN tunnel...
..........done.
NAME                REGION       STATUS
branch-to-office-0  us-central1  ESTABLISHED
branch-to-office-1  us-central1  FIRST_HANDSHAKE
office-to-branch-0  us-east1     ESTABLISHED
office-to-branch-1  us-east1     FIRST_HANDSHAKE

Tunnels show ESTABLISHED as soon as both sides of a pair exist and agree on the secret; FIRST_HANDSHAKE means the other side was still being created.

Step 6: BGP sessions

Why: each tunnel gets a router interface with a link address, and each router gets a BGP peer on the other side. Then the routers exchange routes by themselves. We wait until both sessions are UP:

# BGP: each tunnel gets a router interface with a link address (169.254.x.x/30) and a BGP peer on the other side
for i in 0 1; do
  gcloud compute routers add-interface rt-office --region=us-east1 --interface-name=if-$i --vpn-tunnel=office-to-branch-$i --ip-address=169.254.$i.1 --mask-length=30 > /dev/null
  gcloud compute routers add-bgp-peer rt-office --region=us-east1 --peer-name=peer-$i --interface=if-$i --peer-ip-address=169.254.$i.2 --peer-asn=65002 > /dev/null
  gcloud compute routers add-interface rt-branch --region=us-central1 --interface-name=if-$i --vpn-tunnel=branch-to-office-$i --ip-address=169.254.$i.2 --mask-length=30 > /dev/null
  gcloud compute routers add-bgp-peer rt-branch --region=us-central1 --peer-name=peer-$i --interface=if-$i --peer-ip-address=169.254.$i.1 --peer-asn=65001 > /dev/null
done
for t in $(seq 1 30); do U=$(gcloud compute routers get-status rt-office --region=us-east1 --format="value(result.bgpPeerStatus[].status)" | tr ';' '\n' | grep -c '^UP$'); [ "$U" = 2 ] && break; sleep 10; done
gcloud compute vpn-tunnels list --format="table(name,status,detailedStatus)"
gcloud compute routers get-status rt-office --region=us-east1 --format="table(result.bgpPeerStatus[].name,result.bgpPeerStatus[].status,result.bgpPeerStatus[].numLearnedRoutes)"
Updated [https://www.googleapis.com/compute/v1/projects/your-project-id/regions/us-east1/routers/rt-office].
Creating peer [peer-0] in router [rt-office]...
..........done.
Updated [https://www.googleapis.com/compute/v1/projects/your-project-id/regions/us-central1/routers/rt-branch].
Creating peer [peer-0] in router [rt-branch]...
..........done.
Updated [https://www.googleapis.com/compute/v1/projects/your-project-id/regions/us-east1/routers/rt-office].
Creating peer [peer-1] in router [rt-office]...
..........done.
Updated [https://www.googleapis.com/compute/v1/projects/your-project-id/regions/us-central1/routers/rt-branch].
Creating peer [peer-1] in router [rt-branch]...
..........done.
NAME                STATUS       DETAILED_STATUS
branch-to-office-0  ESTABLISHED  Tunnel is up and running.
branch-to-office-1  ESTABLISHED  Tunnel is up and running.
office-to-branch-0  ESTABLISHED  Tunnel is up and running.
office-to-branch-1  ESTABLISHED  Tunnel is up and running.
NAME                  STATUS        NUM_LEARNED_ROUTES
['peer-0', 'peer-1']  ['UP', 'UP']  [1, 1]

All four tunnels Tunnel is up and running, both BGP sessions UP, one route learned on each.

Step 7: ping through the VPN

# Through the VPN: the office VM now reaches the branch VM on its private address, and learned the route by BGP
gcloud compute routers get-status rt-office --region=us-east1 --format="value(result.bestRoutes[].destRange)" | tr ';' '\n' | sort -u | sed 's/^/route learned by BGP: /'
gcloud compute ssh office-vm --zone=us-east1-b --tunnel-through-iap --command="ping -c 4 10.20.0.2 | tail -2" 2>/dev/null
route learned by BGP: 10.20.0.0/24
route learned by BGP: 10.20.0.0/24
4 packets transmitted, 4 received, 0% packet loss, time 3002ms
rtt min/avg/max/mdev = 34.691/35.518/37.712/1.271 ms

The office router learned 10.20.0.0/24 (the branch subnet) from BGP over both tunnels, and the office VM reaches the branch VM's private address in about 35 ms (us-east1 to us-central1).

VPN up
Four tunnels established, both BGP peers up, the branch range learned, 4 of 4 pings across regions

Break it: a tunnel goes down

We delete one tunnel, as if one internet link or one gateway interface failed:

# Break it: one tunnel goes down (we delete it, like a failed internet link). HA VPN keeps traffic on the other one
gcloud compute vpn-tunnels delete office-to-branch-0 --region=us-east1 --quiet 2>&1 | tail -1; sleep 30
gcloud compute routers get-status rt-office --region=us-east1 --format="table(result.bgpPeerStatus[].name,result.bgpPeerStatus[].status)"
gcloud compute ssh office-vm --zone=us-east1-b --tunnel-through-iap --command="ping -c 4 10.20.0.2 | tail -2" 2>/dev/null
..........done.
NAME                  STATUS
['peer-0', 'peer-1']  ['DOWN', 'UP']
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 34.826/35.672/37.821/1.244 ms
📝⭐ Recommended by CareerByteCode · Certification examCIPS Level 5 Advanced Diploma in Procurement and Supply PracticeTake the examTake the exam →

One BGP session is DOWN, but the ping still gets 4 of 4 replies: traffic moved to the remaining tunnel. That is the "HA" in HA VPN, and it only works because we built two tunnels.

Break it again: the wrong shared secret

We recreate the tunnel with a different secret than the other side uses, a classic mistake when two teams configure the two ends:

# Break it again: recreate the tunnel with a WRONG shared secret. IKE cannot agree, the tunnel never comes up
gcloud compute vpn-tunnels create office-to-branch-0 --region=us-east1 --vpn-gateway=gw-office --interface=0 --peer-gcp-gateway=projects/$PROJECT_ID/regions/us-central1/vpnGateways/gw-branch --router=rt-office --ike-version=2 --shared-secret="wrong-$(openssl rand -hex 6)" > /dev/null
sleep 60; gcloud compute vpn-tunnels describe office-to-branch-0 --region=us-east1 --format="value(status,detailedStatus)"
Creating VPN tunnel...
..........done.
FIRST_HANDSHAKE	Allocating resources. VPN tunnel will start soon.

After 60 seconds the tunnel is still in FIRST_HANDSHAKE: IKE cannot finish because the two sides do not share the same key. (We did not wait longer; a tunnel with a wrong secret does not come up until the secret is fixed.)

Fix it

Recreate the tunnel with the right secret and wait for its BGP session:

# Fix it: recreate the tunnel with the right secret; the BGP peer comes back and both paths are up again
gcloud compute vpn-tunnels delete office-to-branch-0 --region=us-east1 --quiet > /dev/null 2>&1
gcloud compute vpn-tunnels create office-to-branch-0 --region=us-east1 --vpn-gateway=gw-office --interface=0 --peer-gcp-gateway=projects/$PROJECT_ID/regions/us-central1/vpnGateways/gw-branch --router=rt-office --ike-version=2 --shared-secret="$(cat .vpn-secret)" > /dev/null
for t in $(seq 1 30); do U=$(gcloud compute routers get-status rt-office --region=us-east1 --format="value(result.bgpPeerStatus[].status)" | tr ';' '\n' | grep -c '^UP$'); [ "$U" = 2 ] && break; sleep 10; done
gcloud compute routers get-status rt-office --region=us-east1 --format="table(result.bgpPeerStatus[].name,result.bgpPeerStatus[].status)"
Creating VPN tunnel...
..........done.
NAME                  STATUS
['peer-0', 'peer-1']  ['UP', 'UP']

Both BGP sessions are UP again.

Break it and fix it
One tunnel deleted: ping still 4 of 4; a wrong secret stays in FIRST_HANDSHAKE; the right secret brings both peers up

POC: prove it works

  • No route before the VPN, 4 of 4 pings after it, over private addresses (Steps 3 and 7).
  • BGP learned the branch range automatically (Step 7).
  • With one tunnel gone, traffic kept flowing on the other (Break it).
  • A wrong secret kept the tunnel from coming up; the right one fixed it (Break it again, Fix it).

Clean up (delete everything)

Why: tunnels and gateways cost by the hour. We delete in order: tunnels, routers, gateways, VMs, firewall rules, subnets, networks.

# Clean up fast (tunnels cost by the hour): tunnels, routers, gateways, VMs, firewall rules, subnets, networks
for i in 0 1; do gcloud compute vpn-tunnels delete office-to-branch-$i --region=us-east1 --quiet > /dev/null 2>&1; gcloud compute vpn-tunnels delete branch-to-office-$i --region=us-central1 --quiet > /dev/null 2>&1; done
gcloud compute routers delete rt-office --region=us-east1 --quiet > /dev/null 2>&1; gcloud compute routers delete rt-branch --region=us-central1 --quiet > /dev/null 2>&1
gcloud compute vpn-gateways delete gw-office --region=us-east1 --quiet > /dev/null 2>&1; gcloud compute vpn-gateways delete gw-branch --region=us-central1 --quiet > /dev/null 2>&1
gcloud compute instances delete office-vm --zone=us-east1-b --quiet > /dev/null 2>&1; gcloud compute instances delete branch-vm --zone=us-central1-a --quiet > /dev/null 2>&1
for n in office branch; do gcloud compute firewall-rules delete $n-allow-vpn --quiet > /dev/null 2>&1; done
gcloud compute networks subnets delete office-sub --region=us-east1 --quiet > /dev/null 2>&1; gcloud compute networks subnets delete branch-sub --region=us-central1 --quiet > /dev/null 2>&1
gcloud compute networks delete vpc-office vpc-branch --quiet > /dev/null 2>&1; rm -f .vpn-secret
echo "left: $(gcloud compute vpn-tunnels list --format='value(name)' | wc -l) tunnels, $(gcloud compute vpn-gateways list --format='value(name)' | wc -l) gateways, $(gcloud compute routers list --filter='name~^rt-' --format='value(name)' | wc -l) routers, $(gcloud compute instances list --filter='name~-vm$' --format='value(name)' | wc -l) VMs, $(gcloud compute networks list --filter='name~^vpc-' --format='value(name)' | wc -l) networks"
WARNING: The following filter keys were not present in any resource : name
left: 0 tunnels, 0 gateways, 0 routers, 0 VMs, 0 networks
Clean up
0 tunnels, 0 gateways, 0 routers, 0 VMs, 0 networks left

FAQ

Classic VPN or HA VPN? HA VPN: two interfaces, a 99.99 percent SLA when set up with two tunnels, and dynamic routing with BGP. Classic VPN is the older option; Google recommends HA VPN for new connections.

How do I connect an office router instead of a second VPC? Create an external VPN gateway object with the office router's public IP, and use --peer-external-gateway in the tunnels. The office side needs IKEv2, the same secret and BGP. We did not test a physical router here.

Why BGP and not static routes? BGP adds and removes routes by itself when a tunnel or a subnet changes, which is what made the failover in Break it automatic.

Can the two networks use the same IP range? No. Overlapping ranges cannot be routed between each other. Plan ranges before you connect sites.

What about Cloud Interconnect? A private physical connection with more bandwidth and no internet in between. VPN is the quick, cheap start; Interconnect is for big, steady traffic.

Troubleshooting

You seeWhyFix
Exactly one of (--peer-address ... --peer-gcp-gateway-region) must be specifiedCurrent gcloud refuses name plus region for the peer gatewayGive the full path: projects/PROJECT/regions/REGION/vpnGateways/NAME
Tunnel stays in FIRST_HANDSHAKEThe other side is missing, or the shared secrets differCreate both sides; use the same secret
Tunnel up, BGP DOWNLink addresses or peer ASN do not matchEach side's peer IP is the other side's interface IP; ASNs must match the routers
BGP up, ping failsFirewall rules do not allow the other site's rangeAllow the remote subnet (and ICMP) in each network
Ping works one way onlyRoutes or firewall rules on one side onlyCheck learned routes on both routers

What next

Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.

Read next →Cilium CNI: eBPF Networking and Security for Kubernetes⏱ 13 min

⭐ Keep learning with CareerByteCode

Hand-picked to go deeper on this topic
News, guides and updates from the CareerByteCode team
🏅 Expert writer

The CareerByteCode team writes about what is new on careerbytecode.in: courses, labs, exams, events and the community, which is free to join for influencers and content creators in tech.

Get CareerByteCode Team's next blog by email

One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.

We email you once to confirm. By subscribing you agree to the privacy policy.

💬 Comments (0)

Top · Newest

Sign in to comment

Related blogs

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →