You can view all the demo code here on Github

Wireguard is a highly efficient and secure VPN protocol that has been gaining popularity for its simplicity and performance. Lately, I’ve been increasingly interested in building projects that leverage Wireguard’s capabilities. Recently, I started working on a proof-of-concept system with a friend that would relay packets over secure tunnels via an authentication proxy/relay server, as illustrated in the diagram below:

Click to enlarge

The core idea is that a user points their DNS record for their domain name (e.g., example.com) to the relay server (relay-server.com) using a CNAME record. When someone requests example.com, the request reaches the relay server, which then checks for an authentication token. Once access is granted, the packets are forwarded over the tunnel registered with the domain. There are of course caveats here with different protocols, but you get the idea.

This flow shares some similarities with Cloudflare Tunnels and Tailscale, although it is a more basic implementation. However, the purpose of this article is not to compare commercial tunneling software but rather to walk through the process of building a simple tunneling system from scratch.

Wireguard & Go

wireguard-go is super duper because it implements the complete Wireguard protocol in Go, but it runs in user space. User space? This ultimately means that we can run Wireguard without needing root permissions. There is a fat caveat here which is network configuration such as TUN device creation or virtual network interface creation (👋 hello wg0) requires root permissions still.

// Snippet w/ wireguard-go for creating a TUN device
tun, err := tun.CreateTUN("wg0", device.DefaultMTU)
if err != nil {
    log.Fatal(err)
}

That little guy up there ☝️ requires root permissions. This is something you never want to just hand out for free. If you just willy nilly give any random CLI tool root permissions with sudo ./run-sketchy-cli-binary then you put your systems and networks at massive risk.

Netstack to the rescue

After mulling over this for a few days and reading around, I have to say, I’m pretty thankful for the fly.io blog. They have a fun way of discussing all their networking woes and also heavily discuss Wireguard within their tooling and infrastructure. More specifically, they caught my interest with these 2 articles:

• Our User-Mode WireGuard Year
• SSH and User-mode IP WireGuard

The tl;dr here is they essentially bypassed this escalation problem by importing an entire TCP/IP stack into their client application, circumventing the need to create actual network interfaces against the operating system kernel. They did this with a combined effort of using gVisor’s Netstack which has a complete TCP/IP implementation written in Go along with wireguard-go adding native bindings for Netstack. This allows us to do all the cool TCP/IP things we typically rely on the kernel for… but also in user space!

Confused yet? Let’s walk through the flow. Let’s assume we have a client application written in Go that connects to a remote Wireguard tunnel. Additionally, the tunnel uses this virtualized network via Netstack to facilitate routing before any actual transmission over the network.

Packet flow

1. User makes a request to something like https://google.com
2. Instead of the kernel’s TCP/IP stack getting ahold of the packet immediately, the request is passed to the virtual stack using Netstack (user-space implementation of kernel TCP/IP stack that provides network virtualization).
3. Netstack encapsulates the HTTP request in a TCP segment, which is wrapped in an IP packet + any additional headers necessary.
4. For routing, the packet is forwarded to the virtual Wireguard interface before transmission.
5. Wireguard encrypts the IP packet and encapsulates it in a UDP packet with the appropriate Wireguard headers before handing it back through Netstack.
6. At this point, Netstack just needs to encapsulate the UDP packet in an IP packet with a source IP being the outgoing network interface and the destination IP being the public IP of the Wireguard peer.

Wala! No root permissions required. At the end of the day, we’re just manipulating packets so the TCP/IP implementation provided by the OS is more or less just a convenience (also a bit more performant!). Nothing is preventing us from just firing off UDP packets over our default network interface. Netstack provides the benefit of packet encapsulation. We’re speaking UDP with Wireguard but we can’t just write arbitrary data to the tunnel by saying wireguardUDPSocket.write('hi mom'). We need to encapsulate and send valid IP packets before sending them over, this is something a network stack thrives at.

Toy implementation

As always, talk is cheap. Fly.io described the problem well, but the code was sparse and my limited knowledge of user land TCP/IP stacks wasn’t getting me too far, so I began tinkering. Let’s build a user space tunnel using Wireguard, but proxy HTTP connections made from our browser over it. This allows websites to see our origin IP as the IP of our VM and not the IP assigned to us by our ISP.

Click to enlarge

Wireguard configuration

I’m using and importing https://git.zx2c4.com/wireguard-go in my project, which is written in Go.

To start, I’m adding a tiny bit of abstraction to keep things separated in the code

type WireguardConfig struct {
	PrivateKey string
	PublicKey  string
	AllowedIPs string
	Endpoint   string
	Port       int
}
 
type Wireguard struct {
	config WireguardConfig
}

This allows me to just quickly load up any useful Wireguard specific variables into the struct if I need them later.

The first step when connecting to the tunnel is to create a TUN device that uses the virtualized gVisor Netstack.

type WireguardConfig struct {
	PrivateKey string
	PublicKey  string
	AllowedIPs string
	Endpoint   string
	Port       int
}
 
type Wireguard struct {
	config WireguardConfig
}
 
func (w *Wireguard) GenerateTUN(localAddresses []netip.Addr, dnsAddresses []netip.Addr, mtu *int) (tun.Device, *netstack.Net, error) {
	defaultMtu := 1500
	if mtu == nil {
		mtu = &defaultMtu
	}
 
	tun, tnet, err := netstack.CreateNetTUN(
		localAddresses,
		dnsAddresses,
		*mtu,
	)
	return tun, tnet, err
}

I like to think of the TUN device in this case as a kind of middleware. The device is seen as a virtual network interface, but you can intercept packets off of it before any actual transmission happens. In the Wireguard case, we would be encrypting and encapsulating our packets into a UDP packet before sending them over the network. It’s important to note that the TUN device gets its own IP address as well (just like any other network interface you would have!).

The next step is to associate the TUN device with Wireguard. This is the device responsible for receiving packets off the above TUN device and encrypting them before transmission.

type WireguardConfig struct {
	PrivateKey string
	PublicKey  string
	AllowedIPs string
	Endpoint   string
	Port       int
}
 
type Wireguard struct {
	config WireguardConfig
}
 
func (w *Wireguard) GenerateTUN(localAddresses []netip.Addr, dnsAddresses []netip.Addr, mtu *int) (tun.Device, *netstack.Net, error) {
	defaultMtu := 1500
	if mtu == nil {
		mtu = &defaultMtu
	}
 
	tun, tnet, err := netstack.CreateNetTUN(
		localAddresses,
		dnsAddresses,
		*mtu,
	)
	return tun, tnet, err
}
 
func (w *Wireguard) CreateDevice(tunDevice tun.Device, logLevel int) (*device.Device, error) {
	dev := device.NewDevice(
		tunDevice,
		conn.NewDefaultBind(),
		device.NewLogger(logLevel, ""),
	)
	if dev == nil {
		return nil, fmt.Errorf("Failed to create device")
	}
	return dev, nil
}

Notice we bind the device together with the tunDevice to establish the link. Utilizing these methods would look something like this (ignore the fact that the struct is empty).

preferredMTU := 1500
wg := Wireguard{}
tun, tnet, err := wg.GenerateTUN(
    []netip.Addr{netip.MustParseAddr("10.0.0.3")},
    []netip.Addr{netip.MustParseAddr("1.1.1.1")},
    &preferredMTU)
if err != nil {
    log.Panic("Failed to create TUN device:", err)
}
 
dev, err := wg.CreateDevice(tun, device.LogLevelVerbose)
if err != nil {
    log.Panic("Failed to create WireGuard device")
}

• 10.0.0.3 is the IP of the TUN device
• 1.1.1.1 is using Cloudflare as the DNS resolver

From here, we just need to bring up the device so we can start sending traffic over it.

preferredMTU := 1500
wg := Wireguard{}
tun, tnet, err := wg.GenerateTUN(
    []netip.Addr{netip.MustParseAddr("10.0.0.3")},
    []netip.Addr{netip.MustParseAddr("1.1.1.1")},
    &preferredMTU)
if err != nil {
    log.Panic("Failed to create TUN device:", err)
}
 
dev, err := wg.CreateDevice(tun, device.LogLevelVerbose)
if err != nil {
    log.Panic("Failed to create WireGuard device")
}
 
err = dev.IpcSet(fmt.Sprintf(`private_key=%s
public_key=%s
allowed_ip=0.0.0.0/0
endpoint=<PUBLIC_IP_ADDRESS>:51820
`, base64ToHex("<PRIVATE_KEY>"), base64ToHex("<PUBLIC_KEY>")))
if err != nil {
    log.Panic("Failed to set WireGuard configuration:", err)
}
 
// bring up the Wireguard device
err = dev.Up()
if err != nil {
    log.Panic("Failed to bring up WireGuard device:", err)
}
 
fmt.Println("Connected to WireGuard server")

In my case, I’m running a Wireguard peer over on Digital Ocean and I’ve opened up port 51820/UDP using ufw. As a side-note you’ll notice base64ToHex. After debugging for a bit, I realized when setting the credentials it was expected that the keys were in decoded hex format, so I wrote a little utility for that.

func base64ToHex(base64Key string) string {
    decodedKey, err := base64.StdEncoding.DecodeString(base64Key)
    if err != nil {
        log.Panic("Failed to decode base64 key:", err)
    }
    hexKey := hex.EncodeToString(decodedKey)
    return hexKey
}

Verifying the connection

As a very straight forward test, I wanted to prove the IP was the IP of my VM and not my home ISP IP, so I made a request over the tunnel.

client := http.Client{
	Transport: &http.Transport{
		DialContext:     tnet.DialContext,
		TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
	},
	Timeout: 30 * time.Second,
}
 
resp, err := client.Get("https://api.ipify.org?format=json")
if err != nil {
	log.Panic(err)
}
 
body, err := io.ReadAll(resp.Body)
if err != nil {
	log.Panic(err)
}
 
fmt.Printf("Connected to remote host! Using IP address for proxy: %s\n", string(body))

When you run the program, this should be sufficient to see the IP of the VM get printed to the console. If you run into failures, it’s most likely a configuration or firewall issue you need to debug.

HTTP proxying over a Wireguard tunnel

To extend this idea a bit further, I wanted to traffic web requests from my browser over the tunnel. For this, I created a basic HTTP relay server and sent requests over the tunnel. I won’t cover all the code, but I’ll show the core parts.

type proxy struct {
	Tunnel *netstack.Net
}
 
handler := &proxy{
	Tunnel: tnet,
}
 
log.Println("Starting proxy server on", "127.0.0.1:8080")
if err := http.ListenAndServe("127.0.0.1:8080", handler); err != nil {
	log.Fatal("ListenAndServe:", err)
}

Within the handler itself, I’m able to leverage the Netstack TUN device DialContext like so:

client := &http.Client{
	Transport: &http.Transport{
		DialContext:     p.Tunnel.DialContext,
		TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
	},
	Timeout: 30 * time.Second,
}

You can see how nicely this plays with Go’s std library.

From here, I just downloaded a little proxy switcher extension from the Chrome store and set it to proxy everything to localhost:8080 and now all my traffic is flowing through the Digital Ocean VM!

Conclusion

There are lots of goodies packed into this library and I’m excited to explore. Let me know if you have any questions and definitely check out the fly.io blog.

You can view all the demo code here