[Firehol-devs] code generation

Thu Feb 12 21:03:37 CET 2015

Hi Costa

I think the current ESTABLISHED + RELATED rule only matches e.g. ftp data
connections and other things that are tracked specially. This rule
does not match normal established traffic.

For your proposal we will need 2 rules, for ESTABISHED and RELATED
separately, so that the majority of traffic matches early.

However I would sooner see this as an option e.g.
  FIREHOL_RULESET=optimal
v.s.
  FIREHOL_RULESET=complete

I can see that many people might prefer the more efficient and shorter
firewall. For me at least, though, the current behaviour is worth the extra
rules. I very much appreciate that traffic is checked in both directions
and pre-established connections will be severed if I update my firewall
to deny them.

I am assuming that since the change mostly just stops the generation of
reverse rules, it should not be too hard to put these behind a test?

In optimal mode we could just re-enable the reverse behaviour automatically
and temporarily when the user creates a stateless rule (and add the
prerouting rules). If the user tries using accounting in this mode,
then move the ESTABLISHED accept to the end and warn the user the
firewall is no longer fully optimal.

Cheers
Phil

On Thu, Feb 12, 2015 at 02:24:11PM +0200, Tsaousis, Costa wrote:
> There is another major change I would like to consider:
> 
> Today, FireHOL generates stateful rules for both request and replies, like this:
> 
> rule in ... state NEW,ESTABLISHED ... action ACCEPT
> rule out ... state ESTABLISHED ... action ACCEPT
> 
> and at the end of the firewall there is a rule that accepts all
> ESTABLISHED and RELATED traffic.
> 
> I think the optimal should be:
> 
> 1. accept ESTABLISHED and RELATED traffic at the top of the firewall
> (instead of the bottom); given that most of the traffic is replies,
> this will significantly reduce the iptables rules the packets have to
> traverse in order to be accepted.
> 
> 2. each stateful rule generates code only for state NEW packets (the
> requests), taking of course into account client ports (as it already
> does);  this will cut half of the rules generated today (no need for
> matching per statement ESTABLISHED traffic).
> 
> It will also make certain features (such as logging and ipset counters
> update) be executed only for requests.
> 
> At the same time, it will allow us to incorporate into firehol new and
> exciting features like SYNPROXY (check
> http://people.netfilter.org/hawk/presentations/devconf2014/iptables-ddos-mitigation_JesperBrouer.pdf
> and https://r00t-services.net/knowledgebase/14/Homemade-DDoS-Protection-Using-IPTables-SYNPROXY.html)
> 
> Unfortunately however, this will break:
> 
> a. accounting, since we will be unable to track accounting for the
> replies; the same will happen for ipset counters updates that the user
> wants to update for the whole traffic.
> 
> Both of these features are relatively new. My understanding is that
> optimal iptables rules generation should be of higher priority. We
> could just say, this is the way these features work in firehol and
> possibly create a few helpers that will allow the users do these
> functions the way they want.
> 
> b. stateless rules. I believe this is faulty already. stateless rules
> should have disabled connection tracking for their traffic at the raw
> table and then apply their rules outside the stateful filtering rules.
> We could have a helper to do it right like this:
> 
> stateless server smtp accept inface eth0 ...
> 
> The above should be given before all interfaces and routers. It is a
> helper and should generate rules before the stateful packet filtering.
> We could allow 'group with' work also outside the interfaces and
> routers, like this:
> 
> group with inface eth0
>    stateless server smtp accept
>    stateless server http accept
> group end
> 
> interface eth0 internet
>    server imapd accept
>    ...
> 
> What do you say?
> Is this the firehol we want?
> Do you find any issues I haven't looked at?
> 
> Costa
> 
> 
> On Thu, Feb 12, 2015 at 2:17 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
> > Hi all, I made a few more changes in rule().
> >
> > I have commented the code to let you know what I have done.
> >
> > The improvement is significant in certain cases. I optimized branching
> > (now it is avoided unless it is absolutely necessary to have it), made
> > certain feature (such as limit and ipset) be executed only once per
> > rule, etc. As I see it, rule() cannot be made to generate better
> > iptables code. It is optimal now.
> >
> > I think I have tested it a lot.
> > If you can test it, please let me know if you find any issues.
> >
> > Overall, FireHOL needs another optimization when grouping multiple
> > services together, like this:
> >
> > server "smtp ftp http ssh" accept src "10.0.0.0/8" log "My Servers"
> >
> > This should internally be interpreted as:
> >
> > group with src "10.0.0.0/8" log "My Servers"
> >       server "smtp ftp http ssh" accept
> > group end
> >
> > The above will provide optimal iptables code.
> >
> > PS: the new commit is 10-20% slower. check the code. it does a lot more.
> >
> > Examples:
> >
> > Command  : server smtp accept src "1.1.1.1 2.2.2.2 3.3.3.3" log "SMTP TRAFFIC"
> >
> > OLD
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 1.1.1.1 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j NFLOG
> > --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 1.1.1.1 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> > ACCEPT
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 2.2.2.2 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j NFLOG
> > --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 2.2.2.2 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> > ACCEPT
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 3.3.3.3 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j NFLOG
> > --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 3.3.3.3 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> > ACCEPT
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j NFLOG
> > --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> > ACCEPT
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j NFLOG
> > --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> > ACCEPT
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j NFLOG
> > --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> > ACCEPT
> >
> > NEW
> > /sbin/iptables -t filter -N in_world_smtp_s1.1
> > /sbin/iptables -t filter -A in_world_smtp_s1.1 -j NFLOG
> > --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A in_world_smtp_s1.1 -j ACCEPT
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 1.1.1.1 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> > in_world_smtp_s1.1
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 2.2.2.2 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> > in_world_smtp_s1.1
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 3.3.3.3 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> > in_world_smtp_s1.1
> > /sbin/iptables -t filter -N out_world_smtp_s1.2
> > /sbin/iptables -t filter -A out_world_smtp_s1.2 -j NFLOG
> > --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A out_world_smtp_s1.2 -j ACCEPT
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> > out_world_smtp_s1.2
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> > out_world_smtp_s1.2
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> > out_world_smtp_s1.2
> >
> >
> > ---
> >
> > Command  : server smtp accept src not 1.1.1.1 dst not 2.2.2.2 log "HELLO"
> >
> > OLD
> > /sbin/iptables -t filter -N in_world_smtp_s3.1
> > /sbin/iptables -t filter -A in_world_smtp_s3.1 -s 1.1.1.1 -j RETURN
> > /sbin/iptables -t filter -A in_world_smtp_s3.1 -d 2.2.2.2 -j RETURN
> > /sbin/iptables -t filter -A in_world_smtp_s3.1 -j NFLOG --nflog-prefix=HELLO:
> > /sbin/iptables -t filter -A in_world_smtp_s3.1 -p tcp -j ACCEPT
> > /sbin/iptables -t filter -A in_world_smtp_s3 -p tcp --sport 1024:65535
> > --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> > in_world_smtp_s3.1
> > /sbin/iptables -t filter -N out_world_smtp_s3.2
> > /sbin/iptables -t filter -A out_world_smtp_s3.2 -s 2.2.2.2 -j RETURN
> > /sbin/iptables -t filter -A out_world_smtp_s3.2 -d 1.1.1.1 -j RETURN
> > /sbin/iptables -t filter -A out_world_smtp_s3.2 -j NFLOG --nflog-prefix=HELLO:
> > /sbin/iptables -t filter -A out_world_smtp_s3.2 -p tcp -j ACCEPT
> > /sbin/iptables -t filter -A out_world_smtp_s3 -p tcp --sport 25
> > --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> > out_world_smtp_s3.2
> >
> > NEW
> > /sbin/iptables -t filter -A in_world_smtp_s2 -p tcp \! -s 1.1.1.1
> > --sport 1024:65535 \! -d 2.2.2.2 --dport 25 -m conntrack --ctstate
> > NEW\,ESTABLISHED -j NFLOG --nflog-prefix=HELLO:
> > /sbin/iptables -t filter -A in_world_smtp_s2 -p tcp \! -s 1.1.1.1
> > --sport 1024:65535 \! -d 2.2.2.2 --dport 25 -m conntrack --ctstate
> > NEW\,ESTABLISHED -j ACCEPT
> > /sbin/iptables -t filter -A out_world_smtp_s2 -p tcp \! -s 2.2.2.2
> > --sport 25 \! -d 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate
> > ESTABLISHED -j NFLOG --nflog-prefix=HELLO:
> > /sbin/iptables -t filter -A out_world_smtp_s2 -p tcp \! -s 2.2.2.2
> > --sport 25 \! -d 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate
> > ESTABLISHED -j ACCEPT
> >
> >
> >
> > Command  : server smtp accept src "1.1.1.1 2.2.2.2 3.3.3.3" ipset trap
> > src log "SMTP TRAFFIC"
> >
> > OLD
> > # Rules for smtp server, with server port(s) 'tcp/25' and client
> > port(s) 'default'
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 1.1.1.1 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> > --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 1.1.1.1 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> > --match-set trap src -j ACCEPT
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 2.2.2.2 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> > --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 2.2.2.2 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> > --match-set trap src -j ACCEPT
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 3.3.3.3 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> > --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 3.3.3.3 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> > --match-set trap src -j ACCEPT
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> > --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> > --match-set trap src -j ACCEPT
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> > --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> > --match-set trap src -j ACCEPT
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> > --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> > 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> > --match-set trap src -j ACCEPT
> >
> > NEW - check that the ipset command that updates ipset counters is
> > executed only once
> > /sbin/iptables -t filter -N in_world_smtp_s3.3
> > /sbin/iptables -t filter -A in_world_smtp_s3.3 -j NFLOG
> > --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A in_world_smtp_s3.3 -m set --match-set trap
> > src -j ACCEPT
> > /sbin/iptables -t filter -A in_world_smtp_s3 -p tcp -s 1.1.1.1 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> > in_world_smtp_s3.3
> > /sbin/iptables -t filter -A in_world_smtp_s3 -p tcp -s 2.2.2.2 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> > in_world_smtp_s3.3
> > /sbin/iptables -t filter -A in_world_smtp_s3 -p tcp -s 3.3.3.3 --sport
> > 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> > in_world_smtp_s3.3
> > /sbin/iptables -t filter -N out_world_smtp_s3.4
> > /sbin/iptables -t filter -A out_world_smtp_s3.4 -j NFLOG
> > --nflog-prefix=SMTP_TRAFFIC:
> > /sbin/iptables -t filter -A out_world_smtp_s3.4 -m set --match-set
> > trap src -j ACCEPT
> > /sbin/iptables -t filter -A out_world_smtp_s3 -p tcp --sport 25 -d
> > 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> > out_world_smtp_s3.4
> > /sbin/iptables -t filter -A out_world_smtp_s3 -p tcp --sport 25 -d
> > 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> > out_world_smtp_s3.4
> > /sbin/iptables -t filter -A out_world_smtp_s3 -p tcp --sport 25 -d
> > 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> > out_world_smtp_s3.4