I have been experimenting with ZeroTier and wanted to share how I was able to setup site-to-site connectivity between a home network and AWS to avoid installing the ZT app on every system in my network. It is easier than you might think. This is a quick how-to on setting up site-to-site connectivity from a private network to a basic AWS VPC.

Prerequisites:

A quick note for those not familiar with ZeroTier. ZeroTier is a secure Software Defined Network(SDN) solution that allows connectivity to other devices over the internet like they are connected to the same switch. The ZeroTier Virtual networks are fully encrypted with keys only the owner controls and has access to.

What you need:

• A working AWS account
• A working home network
• A working ZeroTier account
• At least one ZeroTier network setup
• A Linux instance in each network to act as a ZT gateway
• At least one other instance for testing in each network

Note: If you don’t have an AWS account, you can follow this guide on how to do it https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/.

Note: Following that guide is a bare minimum setup and you may want to do further research on how to secure your new AWS account properly before using it.

Note: You may need to setup a ZeroTier account at https://my.zerotier.com if you do not already have one. You will also need to configure at least one network https://zerotier.atlassian.net/wiki/spaces/SD/pages/8454145/Getting+Started+with+ZeroTier.

Install the zerotier-cli client on the instances that will act as the ZT gateway devices and join them to the ZT network you just created. Once joined, you will need to authorize your new nodes to be able to use them. Keep track of the assigned ZeroTier IP’s for each of your devices.

Verify you can send ping traffic between each node using the ZeroTier assigned IP.

Before proceeding, make sure you have met the above requirements.

Enabling site-to-site traffic:

Enable IP forwarding so packets destined for the remote network are forwarded from one network to another. There is no need for port forwarding rules or NAT here because we are just routing packets between networks. Perform this action on each node acting as a gateway.

echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
sysctl -p

You can verify this by running sysctl net.ipv4.ip_forward from the command line.

We will want to disable the firewall on the node running in your home network for the time being. It is a good idea to come back and enable this, after adjusting the rules for your specific traffic requirements.

systemctl stop firewalld
systemctl disable firewalld

We are now ready to add some static routes in the ZeroTier web UI, our home router, and in AWS. This is the bit that can be a little tricky. However, we have good notes on our network details, which will be helpful.

Using the following network information, we can create specific route rules to allow forwarding traffic from each network to the proper destination network.

Network details:

ZeroTier: 192.168.191.0/24
Home(Site A): 192.168.2.0/24
AWS(Site B): 172.31.0.0/16
Site A’s VM is 192.168.2.237 on the Local Network
Site A’s VM is 192.168.191.107 on the ZT Network
Site B’s VM is 172.31.17.63 on the Local Network
Site B’s VM is 192.168.191.152 on the ZT Network

In AWS, perform the following actions:

Disable the source/destination check on the ZT gateway instance from the ec2 instance’s action menu.

• When logged in to your AWS account, go to Services->EC2->Instances
• Select the instance acting as the ZeroTier gateway then Actions->Networking->Change Source/Dest Check
• Disable the setting if enabled(Default is Enabled)

Add the route for sending traffic to your home network via the devices acting as the ZeroTier gateway.

• When logged in to your AWS account, go to Services->VPC->Route Tables
• Select the route you want to make changes to then Actions->Edit routes

• Add routes to allow traffic destined for your home network to use the ZT gateway device

Add the cidr or specific device IP’s you wish to allow access from like your home network and/or the ZeroTier networks, to a Security Group. Assign that SG to each instance you wish to allow access to.

In your home network router:

You will need to add static route information so your home router knows the proper pathway for sending packets to your AWS account via the ZeroTier gateway device. Each router is going to be different here so I can only give you an example of the routes used for my setup. You will have to figure out where the settings can be changed on your own device.

In the ZeroTier Web console:

Add the routes you wish to be propagated by the ZeroTier cli client.

That’s it. You should now have a working site-to-site connection between your home network and your AWS VPC. There are a lot of steps that need to be done correctly for this to work, but with a little patience, you can set this up in less than an hour.

Additional thoughts:

Some other ideas that have not been fully thought out yet, which are adding DNS support and additional routing capabilities based on different client networks might make this a cool tool to explore further. There are many overlaps with using a VPN but from a client perspective, one doesn’t need a lot of details to get connected. I see this may serve as the basis for more robust setups in the future. Something I ran across after I wrote this, which may be useful for further exploration as well https://blog.reconinfosec.com/locking-down-zerotier/.

REF: https://zerotier.atlassian.net/wiki/spaces/SD/pages/8454145/Getting+Started+with+ZeroTier
REF: https://zerotier.atlassian.net/wiki/spaces/SD/pages/6848513/Join+a+Network
REF: https://zerotier.atlassian.net/wiki/spaces/SD/pages/7503890/ZeroTier+to+Amazon+VPC+Gateway
REF: https://mangolassi.it/topic/19493/zerotier-site-to-site