Taking My Home IPv6-Most

This is going to be a long one, so definitely take advantage of the Table of Contents here if you’re looking for something specific. You can skip to the exciting Conclusion for just the result. (Though admittedly it’s less long than originally anticipated because it’s taken so long to write that I’m forgetting what happened 😅)

Background

I’ve had IPv6 enabled on my home network for ages, and slowly but surely more of the devices I buy are adding support. I recently did a bit of a network infrastructure refresh (see the entry on Incus OS) and was reminded of how frustrated I am by the sort of disjointed nature that a dual-stack network brings. E.g. I can easily partition all my dockers with ::d0cc:* IPv6 addresses, but struggle to do any sort of partitioning with my /24 subnet without running out of space for DHCP.

DNS is also funky because DHCP easily resolves the IPv4 of any clients, but IPv6 (when it’s supported) isn’t consistently mapped.

Open Table of contents

Goals and Topology
- VLANs
  - LAN
  - Cameras
  - IoT/Guest
- DNS
Initial Setup
Quirks
- Cameras
- LAN
- IoT
DNS
Conclusion

Goals and Topology

VLANs

I have 3 VLANs on my network, and each has its own requirements for what kind of networking I need:

LAN

Contains my home server and trusted devices like my PC and cellphone. Has Internet access. This VLAN contains mostly modern devices, so should be able to handle IPv6-only.

Cameras

For security cameras; no internet access. Should be able to handle IPv6 only as long as I can get all the cameras to support it.

IoT/Guest

This is where guest devices and all my IoT devices go. I already know that e.g. my Kasa Smart Plugs can’t do IPv6, and I don’t know for sure that guest devices will, so there will need to be some amount of IPv4 support here.

DNS

I want to be able to connect to servers using a name instead of an IP address, and I’d like to be able to use rDNS to figure out the name of IP addresses that show up in logs and analysis tools.

Initial Setup

I checked the core devices on the network (IncusOS, OpenWRT router, Switch, and Home Server) to make sure they had a static IPv4 and static IPv6 address assigned. This would ensure that even if things went sideways, I could still at least adjust or rollback the core components. I also dumped any records I had of MAC/IP/hostname mappings (like the dhcp config file from OpenWRT) so I could find and identify devices later.

I created a second insance of OpenWRT on Incus, and configured it for the new network. I connected it to fake ports and used a proxy to connect to the UI so I could pre-configure as much as possible without interrupting the family’s Internet access. I left DHCP on for the IoT/Guest VLAN, but set up Cameras and LAN to use only SLAAC. I knew I eventually might need to add NAT64 for full functionality, but figured it was easiest to start with SLAAC-only.

After things were sufficiently set up, I switched the OpenWRT instances and… there were some additional challenges. Each VLAN had its own quirks, and DNS was a whole chapter of its own…

Quirks

Cameras

It turns out most of my cameras (Dahua) don’t support SLAAC, so I intended to just use static addresses, but: my interior cameras running Thingino don’t easily support static IPv6 or SLAAC. Also using static addresses would mean I would also need to manually configure DNS and NTP, so DHCPv6 it is! There’s a lot of reasons not to use DHCPv6, but for this well-controlled VLAN it made sense. It also made DNS a little easier, but we’ll get to that later.

I have a single old camera that doesn’t support IPv6. Since I was already doing DHCPv6, I just set up DHCPv4 too, but unchecked the “Dynamic DHCP” option in OpenWRT so only this camera would get an IPv4 address from DHCP, the other cameras should only have IPv6.

ℹ️ Don’t forget to add Firewall rules for DHCP ports in OpenWRT. This is also another way to ensure only specific devices can get e.g. a DHCPv4 address (by allowing only packets from their MAC address to port 67).

LAN

The LAN VLAN was pretty straight forward as the devices all fell into one of three categories:

Modern enough to do SLAAC well (Windows PCs and Android phones)
Easy to manually configure with “static” addresses (docker containers and a Brother laser printer)
Too old for IPv6, but easy to set a static IPv4 (an old Zebra label printer)

No DHCPv6 needed, and almost no DHCPv4 needed except:

IncusOS is running a very up-to-date Linux kernel (6.17 at the time of writing), and because OpenWRT is running as a system container, it shares its kernel with IncusOS. Unfortunately, all the kmod packages for OpenWRT 24 are compiled for version 6.6. This explains why the “Realtime Graphs” page wasn’t working, but also means I cannot install Jool to handle IPv6 to IPv4-only connections.

For fun I tried IPv6-only on my desktop, but basically the first application I opened failed because it tried to connect to an IPv4 only server (not to mention, e.g. Reddit which is IPv4-only). So DHCPv4 got turned on for LAN, but again without the “Dynamic DHCP” option so that only devices I know will be accessing the Internet (desktops and phones) will get an IPv4 address.

Our NVIDIA Shield was again almost (are you sensing a pattern?) fine without an IPv4 address (all the major streaming services seem to support IPv6 just fine), but the NVIDIA Shield app for remote controlling SHIELD from my phone could only do IPv4 ☹️.

IoT

IoT ended up probably the most straight forward with just DHCPv4 and SLAAC across the board. Initially I had planned to try to push as many of these devices to IPv6 as possible (maybe even do a similar only-static-reservations-get-IPv4 strategy), but ended up stymied by lackluster IPv6 support.

We have quite a collection of various IoT devices, but the three biggest groups are:

Google Nest speakers/displays that support SLAAC but don’t do DHCPv6.
Kasa smart plugs that don’t do IPv6 at all.
ESPHome devices, which do support IPv6 but only if they also have an IPv4 address which is required for boot.

So while the Google devices would probably be okay IPv6-only, they were the only ones (and I’m not about to make a VLAN just for them). For this VLAN there’s nothing I need to connect to by hostname, but I still do want rDNS so that’s something that needs to be solved.

DNS

DNS was a big consideration when I started out, and the solution is… multifaceted. So far this all seems to be working save the last section that I haven’t implemented yet.

mDNS

I’ve seen it suggested that mDNS is a good solution to the lack of DHCPv6 for connecting to local devices by host name, but my brief journey went like this:

Only a few devices actually natively support mDNS (ESPHome, some printers, WLED, Nvidia Shield).
I could add mDNS support to the dozens of docker containers on my server, but there’s no universal way to do this so I’d be looking at potentially custom Dockerfiles for each service, etc.
There are some solutions that allow me to have a single device that responds with mDNS info for all the docker containers (I think some that even read this info from the labels set in the compose files), but this would require me to maintain that mapping in the format that service requires. Which, isn’t too bad except…
Outside of the docker host, there’s no easy way to automatically detect non-mDNS devices, look up their information in a map, and then create mDNS entries for them (potentially dynamically if they’re using DHCP). And…
This doesn’t help with e.g. all the cameras that I’ll need to just manually maintain a map for anyway.

So mDNS as a catch-all solution is off the table, but I still need mDNS support for casting to devices (and some other magical detection stuff) so I added this to a file I created at /usr/share/nftables.d/ruleset-pre:

ℹ️ Below are some configs with actual IP addresses. I’m using addresses like fd10::/64 and 10.1.0.0/23 for the LAN VLAN, so you’ll need to substitute with your own if you copy any of this.

table ip relay4
flush table ip relay4

table ip relay4 {
        chain prerouting_mangle_mdns4 {
                type filter hook prerouting priority mangle; policy accept;
                ip daddr 224.0.0.251 iifname "lan" ip saddr set 10.2.0.1 dup to 224.0.0.251 device "iot" notrack
                ip daddr 224.0.0.251 iifname "iot" ip saddr set 10.1.0.1 dup to 224.0.0.251 device "lan" notrack
        }
}

table ip6 relay6
flush table ip6 relay6

table ip6 relay6 {
        chain prerouting_mangle_mdns6 {
                type filter hook prerouting priority mangle; policy accept;
                ip6 daddr ff02::fb iif "lan" ip6 saddr set fd10::1 dup to ff02::fb device "iot" notrack
                ip6 daddr ff02::fb iif "iot" ip6 saddr set fd10::1 dup to ff02::fb device "lan" notrack
        }
}

ℹ️ Make sure you add /usr/share/nftables.d/ to the lists of directories to be included during a sysupgrade so you don’t lose this with a backup or upgrade.

This just basically rebroadcasts mDNS packets between the IoT and LAN VLANs. From a security standpoint this does technically leak a little bit of information (the mDNS responses) between VLANs, but connections between the devices still go through the firewall, so even if a device knows something exists on another VLAN it can’t connect without permission.

DHCP

As discussed above, quite a few collections of devices ended up using DHCP. For these devices, their own provided hostname or a hostname in the /etc/config/dhcp file can easily be used for (r)DNS lookups. I’ve given OpenWRT a subdomain of *.dhcp.mydomain.cloud to give these devices. Then in AdGuard Home, I have the following lines at the beginning of my “Upstream DNS Servers” configuration

[//][fd10::1]
[/dhcp.mydomain.cloud/][fd10::1]

This sends any lookups for bare domains or domains at dhcp.mydomain.cloud to OpenWRT for resolution.

I also added [fd10::1]:53 to “Private reverse DNS servers” to handle rDNS requests for any private IP ranges.

Docker Containers

The majority of my docker containers are behind Traefik to provide some security, but mostly just to avoid me having to worry about http vs. https and different port numbers for each service. Traefik requires (or at least does for the easiest routing) that services are accessed by hostname rather than IP, so I need a DNS entry for every container. But since all these hostnames just need to point to Traefik’s IP, I was able to just create a rewrite rule in AdGuard Home that directs *.container.mydomain.cloud to fd10::d0cc:60 (the Traefik container’s IP address). And then added [/container.mydomain.cloud/][::1] to the Upstream DNS servers to prevent queries for that subdomain from being forwarded to the external DNS servers.

This means that for any new container, I only need to add the appropriate Traefik labels in the compose file and I can immediately connect to the container at https://containername.container.mydomain.cloud.

The only downside to this scheme is that any containers that make outgoing connections will use an IP address not associated with a hostname, and incoming connections are all to the same IP so rDNS can’t really help determine anything about that traffic. For the most part though the docker containers aren’t making any outgoing connections, and if they are, hopefully I can guess what’s going on by where the connection is going.

Any containers that need to be connected directly to the network (e.g. the NTP server or Home Assistant) can be handled as with “Everything Else” below.

Static IPs

For anything statically assigned I’ve reserved the lan.mydomain.cloud subdomain, and can make static rewrite entries in AdGuard Home. These are mostly just for the network equipment and Traefik. I added [/lan.mydomain.cloud/][::1] to the Upstream DNS Servers config in AdGuard Home so that queries for this local domain wouldn’t be forwarded externally.

Everything Else (SLAAC et al.)

For everything else I have two challenges:

For high-availability devices (mostly the security cameras and printers), DHCP can be sometimes unreliable e.g. after a router restart before the device has renewed it DHCP lease. For DHCPv4, there’s a dns option that will create a static DNS entry for the reserved IP so it can be served even when the device isn’t online or hasn’t renewed since reboot. However because DHCPv6 doesn’t specify full IP addresses (only preferred suffixes), there can be no static DNS entry. (Now, this clearly isn’t technically true because the local IPv6 /64 prefix is a fixed value, so combining that at the suffix as a static entry would be fine, but I can at least understand why OpenWRT doesn’t actually do that.) So for these devices I’ve created a separate hosts file /etc/hosts.manual to explicitly list out the IPv6 addresses for DHCPv6 servers, and IPv4 addresses for statically-assigned devices (like the Zebra printer from above).
SLAAC is famously incompatible with discovering hostnames. There are some workarounds, but they’re pretty reliant on the client and I can’t modify most of the client networking code on the devices I’m concerned about (mostly on IoT). So I’m going to write my own service to do the discovery and mapping.

There are still details to be worked out, but essentially it boils down to using ip -6 neigh show or a similar networking library to gather a list of neighbors, and then using entries in /etc/config/dhcp or /etc/ethers to amp the MAC addresses from the neighbor to DNS entries in /etc/hosts.automatic.

This will obviously only work for devices that communicate to/via the router (anything that only communicates internally won’t be discovered), but that should pretty much cover most devices I think. And to reiterate, this is primarily for rDNS; it’s unlikely I’ll ever need to open a connection to my Android phone, but I would like to know which connections it’s been opening itself.

Conclusion

There were probably a lot of little fiddly detail bits that I’ve forgotten to include, but that’s the general gist of my attempts to go as-IPv6-as-possible and the end result is: 🤷🏻‍♂️ fine.

In each area of the network, whether it be IoT smart plugs, Jool for backward compatibility, or getting (r)DNS working for router clients, there was always something that needed IPv4 or DHCP; there’s really no feasible way to accomplish the dream of “just using SLAAC”.

If I ever manage to get Jool working, I can potentially start to build islands of IPv6-only devices (translating via NAT64 on the router to talk to IPv4 devices), but I still need DHCPv6 for devices that won’t do SLAAC, and need some magical (custom) service to associate discovered devices with a hostname. And even then that only works if they send traffic through the router— anything else will need manual DNS records.

Potentially if I was starting entirely from scratch (and solved the NAT64 issue), I could ensure I only added new devices that fully supported DHCPv6 (so nothing Google/Android) and it could work. Or only devices that supported SLAAC and a custom discovery tool, and that would also work.

Until then though, I feel like things are at least a little more organized even if they’re pretty heterogenous.