[Firehol-support] Help Needed - Debugging Implicitly Dropped Forwarded Packets

[Costa Tsaousis]

Hi Federico,

Federico Sevilla III wrote:
> Hi,
>
> I hope someone can help me debug a situation regarding some implicitly
> dropped forwarded packets. Here is one sample log message:
>
>         Dec 29 12:44:52 firewall 'PASS-unknown:' IN=eth0 OUT=eth2 MAC=00:15:17:bf:70:64:00:11:d8:b1:c2:ba:08:00  SRC=172.16.3.174 DST=172.16.0.1 LEN=52 TOS=10 PREC=0x00 TTL=63 ID=16479 DF PROTO=TCP SPT=48985 DPT=80 SEQ=2105617222 ACK=3098975575 WINDOW=46 ACK URGP=0 
>
> On the outset, this looks really simple. For what should be a very
> "open" setup, just add:
>
>         router foo inface eth0 outface eth2
>                 server all accept
>
> But the above message about this implicitly dropped forwarded packet
> appeared with exactly that in the Firehol configuration file (properly
> restarted after the modification). Also to note that when Firehol is
> stopped, things work just fine. Unfortunately that is not how we want
> the firewall to operate. ;)
>
>   
This packet is dropped by the iptables connection tracker, not firehol 
rules.

The iptables connection tracker believes this packet is not part of an 
existing connection; it is not a new connection (new TCP connections 
have the SYN bit set), so it drops it. The connection tracker has 
several sysctl parameters that can affect its operation (mainly 
timeouts), however the defaults are pretty good for general use.

Does this dropped packet appear every time you access the web server?
Does this dropped packet affect the client experience accessing the web 
server?

> Now to describe the network, which may be a bit weird. These are two
> separate /24 networks with the firewall in the middle. 172.16.3.174 is
> the workstation and 172.16.0.1 is the server. The firewall has IP
> addresses and physical interfaces on each subnet. The firewall's
> 172.16.3.0/24 address is the default gateway of 172.16.3.174.
>
> 172.16.0.1 is a virtual server, using OpenVZ. Its default gateway is the
> OpenVZ internal address, using the default venet. The hardware node,
> with multiple network cards, sits on both the 172.16.0.0/24 and
> 172.16.3.0/24 networks because of some other servers it hosts which
> shouldn't go through the firewall anymore to remove a bottleneck.
>
> Because of the routing tables on the OpenVZ hardware node, traceroutes
> from 172.16.0.1 to 172.16.3.174 no longer pass through the firewall. The
> traceroutes work when Firehol on the firewall is stopped, but no longer
> work when Firehol on the firewall is started.
>
> A trace from 172.16.3.174 to 172.16.0.1 would be:
>
> 1. 172.16.3.254 (firewall)
> 2. 172.16.3.251 (hardware node)
> 3. 172.16.0.1
>
> Meanwhile, a trace from 172.16.0.1 to 172.16.3.174 would be:
>
> 1. 172.16.0.251 (hardware node)
> 2. 172.16.3.174
>
> When Firehol is stopped, both ping and HTTP works fine. With Firehol
> running:
>
> 1. 172.16.3.174 is able to ping 172.16.0.1.
> 2. 172.16.0.1 is NOT able to ping 172.16.3.174.
> 3. 172.16.3.174 is able to connect to 172.16.0.1 port 80 (eg: via
> telnet) but no responses are received to any data sent.
>
> Would appreciate advise on what Firehol rules should be able to solve
> this particular scenario.
>
>   
FireHOL is stateful, meaning it allows packets to pass if they are:

1. allowed NEW connections, or
2. part of ESTABLISHED connections

This means that if a connection has parts of it passing through the 
firewall and parts of it not passing through the firewall, the iptables 
connection tracker cannot have the whole picture and packets may be dropped.
Even in this scenario though, you should have logs of the packets 
dropped. Do you?

Regards,

Costa