As to the most recent escapade, I recently found myself fighting with Shorewall.  A hastily hand-drawn, but informative, diagram of the network:

184highnet

odin acts as bridge, router, and single point of catastrophic failure.  artemis and cobblebox have enough services running that I want them to have public IPs, so they are bridged to the cable modem segment.  Everything is also connected to the LAN, which odin NATs from its bridge interface to eth2.  odin provides DNS, DHCP, etc. to the LAN.  So far, so good (except that dhcpd still won’t update bind properly…).

Yes, this is far more complex than it strictly needs to be, but it means I can make pretty graphs and figure out how close I am to making Comcast angry.

mrtg eth0-week

I soon found that I need to be able to forward ports on odin’s external interface (br0) to machines on the LAN.  It’s easy with a two-interface machine: just say, for example, “any traffic in zone net with a destination port of 5121/udp goes to 192.168.10.2:5121″, or, in shorewall-speak (/etc/shorewall/rules):

#ACTION        SOURCE  DEST                    PROTO   DESTPORT        SRCPORT ORIGINAL_DEST
DNAT           net     loc:192.168.10.2:5121   udp     5121

Let’s give it a shot on the three-interface setup.  Oops, still can’t connect from dmz (the zone with artemis and cobblebox).  How about adding this, then?

DNAT           dmz     loc:192.168.10.2:5121   udp     5121

Great, now it works!  Except… oh shit, I can’t connect to anything else on port 5121 now.  WTF?

Turns out that those rules apply to any traffic in those zones with a destport of 5121, meaning all my outgoing traffic on 5121 was getting rewritten to go to 192.168.10.2.  To stop this, we use the original destination in the rules.  Because the original destination (read: the address of br0) is obtained via DHCP, though, it’s bad mojo to specify it in the file.  The solution to that is to modify /etc/shorewall/params.  The net result is:

/etc/shorewall/rules:
#ACTION         SOURCE  DEST                    PROTO   DESTPORT        SRCPORT ORIGINAL_DEST
DNAT            net     loc:192.168.10.2        udp     5121            -       $BR0_IP
DNAT            dmz     loc:192.168.10.2        udp     5121            -       $BR0_IP
/etc/shorewall/params:
BR0_IP=$(find_first_interface_address br0)

Yes, all this information was clearly in the FAQ if I had bothered to read, but my attention span is oh hey, I didn’t know WordPress could do that