I was running into problems with BitTorrent file sharing on my old 2015 model TP-Link WDR4300 router; too much upload traffic (above 800KB/s) or too many connections would cause it to drop all connections to the BitTorrent computer (though other systems still were connected). I thought a newer router with more CPU power and memory (to handle more connections) would help, the 2018 ASUS RT-AC86u router seemed to fit the bill, and was available locally at Staples.
The new ASUS RT-AC86u worked, but with a flaw. The one annoyance was that it didn't do NAT Reflection aka NAT Hairpinning, so that local computers connecting to a local server showed up on that server as a connection coming from the router's internal address. The old WDR4300 (and most of my previous routers) would show the external Internet address for local computers. Normally this wouldn't be important, unless the server needs to show the client address to the rest of the world. But as it happens, I'm running a BeShare server and a BitTorrent tracker, both of which publish a registry of computer addresses for other computers on the Internet to connect to. The result was that when my home BitTorrent client registered with the Tracker, the Tracker would see and register it under the internal address and nobody outside my home network would be able to connect to that.
Fortunately there was some third party more open firmware for the AC86u, the Asuswrt-Merlin firmware project. Since the router software was reusing GPL licensed software (based on Linux), they had to make public their changes to it. The Merlin people used the source code from ASUS with their own changes to make it more open (such as running user provided scripts when certain events happen). Fortunately Merlin had recently added a build for the AC86u, which I downloaded and tried out. It didn't do NAT reflection either. I found out that the "Asus NAT Loopback" function had done it, but had been removed because the firewall rules were getting too complex and it wasn't working reliably.
I gave up for a while on that and let it run. BitTorrent worked mostly
okay, since it has other ways of finding the addresses of clients on the
Internet. However, there were many log messages on the router (viewable by
doing tail -F /tmp/syslog.log) complaining about "nf_conntrack:
expectation table full". After much research on the forums, I found the
appropriate settings and it turns out the default size of the table is around
50. I was able to bump it up by writing this /jffs/scripts/services-start
file:
#!/bin/sh # AGMS20180615 Avoid those expectation table overflows when running BitTorrent. OldMax=`cat /proc/sys/net/netfilter/nf_conntrack_expect_max` echo 1000 > /proc/sys/net/netfilter/nf_conntrack_expect_max logger "Changing nf_conntrack_expect_max from $OldMax to `cat /proc/sys/net/netfilter/nf_conntrack_expect_max`"
Then today I decided to dig in and research the iptables command and NAT reflection. Took all afternoon. The Wikipedia article is a good overview. The iptables man page has all the details, including the -m match extensions (like "-m tcp" which lets you test TCP packet contents for various properties) and -j jump/target extensions that do special actions (like SNAT which modifies the source address of a packet). The existing table entries for doing port forwarding (automatically created by using the router's web interface) look like this:
-A PREROUTING -d 206.47.13.115/32 -j VSERVER -A VSERVER -p tcp -m tcp --dport 2960 -j DNAT --to-destination 192.168.174.36 -A VSERVER -p tcp -m tcp --dport 6969 -j DNAT --to-destination 192.168.174.36They modify the packets which are going to the external IP address to change the destination from that external address to be the appropriate local server computer's address, using that DNAT target operation.
The hairpin trick is simply to hack up the source address in packets going from a local computer to the web server to use the external IP address rather than the local computer's address; SNAT does that. There's some iptables NAT connection tracking magic (see nf_conntrack) that unhacks it for the reply packets. So it turns out that just one new rule is needed to do that. I wrote a /jffs/scripts/nat-start file for the AC86u-Merlin to add that rule for my servers:
#!/bin/sh # AGMS20180812 Add Hairpin NAT rules, so that accessing some internal servers # uses the Internet external IP address as far as the server is concerned. # That way a local BitTorrent client will register in a local Tracker server as # using the external IP address, so that other BitTorrent clients can actually # connect to it. ExternalIP=`ip address show ppp0 | grep peer | sed -e 's/^.*inet \([0-9.]*\) peer.*$/\1/'` logger "Changing firewall to use NAT Hairpinning with external IP $ExternalIP for certain servers." # Start a new chain for our NAT reflection or hairpinning rules. if ! iptables --table nat --new HairPin ; then logger "Hairpin rules already in place, doing nothing." exit fi # The web server at 192.168.174.36 on port 80 # iptables --table nat --append HairPin -d 192.168.174.36 -p tcp -m tcp --dport 80 -j SNAT --to-source $ExternalIP # The BeShare server at 192.168.174.36 on port 2960 iptables --table nat --append HairPin -d 192.168.174.36 -p tcp -m tcp --dport 2960 -j SNAT --to-source $ExternalIP # The BitTorrent Tracker server at 192.168.174.36 on port 6969 iptables --table nat --append HairPin -d 192.168.174.36 -p tcp -m tcp --dport 6969 -j SNAT --to-source $ExternalIP # Hook the Hairpinning rules into the main POSTROUTING table just after the # policy rule. Also do some common prerequisite tests for hairpinning (has # to be from LAN to LAN). iptables --table nat --insert POSTROUTING 2 -o br0 -s 192.168.174.0/24 -d 192.168.174.0/24 -j HairPin logger "Hairpin rules added."The output from iptables-save looks like this:
-A POSTROUTING -s 192.168.174.0/24 -m policy --dir out --pol ipsec -j ACCEPT -A POSTROUTING -s 192.168.174.0/24 -d 192.168.174.0/24 -o br0 -j HairPin -A POSTROUTING -o ppp0 -j PUPNP -A POSTROUTING ! -s 206.47.13.115/32 -o ppp0 -j MASQUERADE -A POSTROUTING ! -s 169.254.210.181/32 -o eth0 -j MASQUERADE -A POSTROUTING -s 192.168.174.0/24 -d 192.168.174.0/24 -o br0 -j MASQUERADE -A HairPin -d 192.168.174.36/32 -p tcp -m tcp --dport 2960 -j SNAT --to-source 206.47.13.115 -A HairPin -d 192.168.174.36/32 -p tcp -m tcp --dport 6969 -j SNAT --to-source 206.47.13.115
Finally it's all working! Yay. Time for some ice cream.
Copyright © 2018 by Alexander G. M. Smith.