[Firehol-devs] code generation

[Tsaousis, Costa]

There is another major change I would like to consider:

Today, FireHOL generates stateful rules for both request and replies, like this:

rule in ... state NEW,ESTABLISHED ... action ACCEPT
rule out ... state ESTABLISHED ... action ACCEPT

and at the end of the firewall there is a rule that accepts all
ESTABLISHED and RELATED traffic.

I think the optimal should be:

1. accept ESTABLISHED and RELATED traffic at the top of the firewall
(instead of the bottom); given that most of the traffic is replies,
this will significantly reduce the iptables rules the packets have to
traverse in order to be accepted.

2. each stateful rule generates code only for state NEW packets (the
requests), taking of course into account client ports (as it already
does);  this will cut half of the rules generated today (no need for
matching per statement ESTABLISHED traffic).

It will also make certain features (such as logging and ipset counters
update) be executed only for requests.

At the same time, it will allow us to incorporate into firehol new and
exciting features like SYNPROXY (check
http://people.netfilter.org/hawk/presentations/devconf2014/iptables-ddos-mitigation_JesperBrouer.pdf
and https://r00t-services.net/knowledgebase/14/Homemade-DDoS-Protection-Using-IPTables-SYNPROXY.html)

Unfortunately however, this will break:

a. accounting, since we will be unable to track accounting for the
replies; the same will happen for ipset counters updates that the user
wants to update for the whole traffic.

Both of these features are relatively new. My understanding is that
optimal iptables rules generation should be of higher priority. We
could just say, this is the way these features work in firehol and
possibly create a few helpers that will allow the users do these
functions the way they want.

b. stateless rules. I believe this is faulty already. stateless rules
should have disabled connection tracking for their traffic at the raw
table and then apply their rules outside the stateful filtering rules.
We could have a helper to do it right like this:

stateless server smtp accept inface eth0 ...

The above should be given before all interfaces and routers. It is a
helper and should generate rules before the stateful packet filtering.
We could allow 'group with' work also outside the interfaces and
routers, like this:

group with inface eth0
   stateless server smtp accept
   stateless server http accept
group end

interface eth0 internet
   server imapd accept
   ...

What do you say?
Is this the firehol we want?
Do you find any issues I haven't looked at?

Costa


On Thu, Feb 12, 2015 at 2:17 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
> Hi all, I made a few more changes in rule().
>
> I have commented the code to let you know what I have done.
>
> The improvement is significant in certain cases. I optimized branching
> (now it is avoided unless it is absolutely necessary to have it), made
> certain feature (such as limit and ipset) be executed only once per
> rule, etc. As I see it, rule() cannot be made to generate better
> iptables code. It is optimal now.
>
> I think I have tested it a lot.
> If you can test it, please let me know if you find any issues.
>
> Overall, FireHOL needs another optimization when grouping multiple
> services together, like this:
>
> server "smtp ftp http ssh" accept src "10.0.0.0/8" log "My Servers"
>
> This should internally be interpreted as:
>
> group with src "10.0.0.0/8" log "My Servers"
>       server "smtp ftp http ssh" accept
> group end
>
> The above will provide optimal iptables code.
>
> PS: the new commit is 10-20% slower. check the code. it does a lot more.
>
> Examples:
>
> Command  : server smtp accept src "1.1.1.1 2.2.2.2 3.3.3.3" log "SMTP TRAFFIC"
>
> OLD
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 1.1.1.1 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j NFLOG
> --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 1.1.1.1 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> ACCEPT
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 2.2.2.2 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j NFLOG
> --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 2.2.2.2 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> ACCEPT
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 3.3.3.3 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j NFLOG
> --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 3.3.3.3 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> ACCEPT
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j NFLOG
> --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> ACCEPT
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j NFLOG
> --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> ACCEPT
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j NFLOG
> --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> ACCEPT
>
> NEW
> /sbin/iptables -t filter -N in_world_smtp_s1.1
> /sbin/iptables -t filter -A in_world_smtp_s1.1 -j NFLOG
> --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A in_world_smtp_s1.1 -j ACCEPT
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 1.1.1.1 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> in_world_smtp_s1.1
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 2.2.2.2 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> in_world_smtp_s1.1
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 3.3.3.3 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> in_world_smtp_s1.1
> /sbin/iptables -t filter -N out_world_smtp_s1.2
> /sbin/iptables -t filter -A out_world_smtp_s1.2 -j NFLOG
> --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A out_world_smtp_s1.2 -j ACCEPT
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> out_world_smtp_s1.2
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> out_world_smtp_s1.2
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> out_world_smtp_s1.2
>
>
> ---
>
> Command  : server smtp accept src not 1.1.1.1 dst not 2.2.2.2 log "HELLO"
>
> OLD
> /sbin/iptables -t filter -N in_world_smtp_s3.1
> /sbin/iptables -t filter -A in_world_smtp_s3.1 -s 1.1.1.1 -j RETURN
> /sbin/iptables -t filter -A in_world_smtp_s3.1 -d 2.2.2.2 -j RETURN
> /sbin/iptables -t filter -A in_world_smtp_s3.1 -j NFLOG --nflog-prefix=HELLO:
> /sbin/iptables -t filter -A in_world_smtp_s3.1 -p tcp -j ACCEPT
> /sbin/iptables -t filter -A in_world_smtp_s3 -p tcp --sport 1024:65535
> --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> in_world_smtp_s3.1
> /sbin/iptables -t filter -N out_world_smtp_s3.2
> /sbin/iptables -t filter -A out_world_smtp_s3.2 -s 2.2.2.2 -j RETURN
> /sbin/iptables -t filter -A out_world_smtp_s3.2 -d 1.1.1.1 -j RETURN
> /sbin/iptables -t filter -A out_world_smtp_s3.2 -j NFLOG --nflog-prefix=HELLO:
> /sbin/iptables -t filter -A out_world_smtp_s3.2 -p tcp -j ACCEPT
> /sbin/iptables -t filter -A out_world_smtp_s3 -p tcp --sport 25
> --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> out_world_smtp_s3.2
>
> NEW
> /sbin/iptables -t filter -A in_world_smtp_s2 -p tcp \! -s 1.1.1.1
> --sport 1024:65535 \! -d 2.2.2.2 --dport 25 -m conntrack --ctstate
> NEW\,ESTABLISHED -j NFLOG --nflog-prefix=HELLO:
> /sbin/iptables -t filter -A in_world_smtp_s2 -p tcp \! -s 1.1.1.1
> --sport 1024:65535 \! -d 2.2.2.2 --dport 25 -m conntrack --ctstate
> NEW\,ESTABLISHED -j ACCEPT
> /sbin/iptables -t filter -A out_world_smtp_s2 -p tcp \! -s 2.2.2.2
> --sport 25 \! -d 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate
> ESTABLISHED -j NFLOG --nflog-prefix=HELLO:
> /sbin/iptables -t filter -A out_world_smtp_s2 -p tcp \! -s 2.2.2.2
> --sport 25 \! -d 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate
> ESTABLISHED -j ACCEPT
>
>
>
> Command  : server smtp accept src "1.1.1.1 2.2.2.2 3.3.3.3" ipset trap
> src log "SMTP TRAFFIC"
>
> OLD
> # Rules for smtp server, with server port(s) 'tcp/25' and client
> port(s) 'default'
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 1.1.1.1 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 1.1.1.1 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> --match-set trap src -j ACCEPT
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 2.2.2.2 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 2.2.2.2 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> --match-set trap src -j ACCEPT
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 3.3.3.3 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A in_world_smtp_s1 -p tcp -s 3.3.3.3 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -m set
> --match-set trap src -j ACCEPT
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> --match-set trap src -j ACCEPT
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> --match-set trap src -j ACCEPT
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> --match-set trap src -j NFLOG --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A out_world_smtp_s1 -p tcp --sport 25 -d
> 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -m set
> --match-set trap src -j ACCEPT
>
> NEW - check that the ipset command that updates ipset counters is
> executed only once
> /sbin/iptables -t filter -N in_world_smtp_s3.3
> /sbin/iptables -t filter -A in_world_smtp_s3.3 -j NFLOG
> --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A in_world_smtp_s3.3 -m set --match-set trap
> src -j ACCEPT
> /sbin/iptables -t filter -A in_world_smtp_s3 -p tcp -s 1.1.1.1 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> in_world_smtp_s3.3
> /sbin/iptables -t filter -A in_world_smtp_s3 -p tcp -s 2.2.2.2 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> in_world_smtp_s3.3
> /sbin/iptables -t filter -A in_world_smtp_s3 -p tcp -s 3.3.3.3 --sport
> 1024:65535 --dport 25 -m conntrack --ctstate NEW\,ESTABLISHED -j
> in_world_smtp_s3.3
> /sbin/iptables -t filter -N out_world_smtp_s3.4
> /sbin/iptables -t filter -A out_world_smtp_s3.4 -j NFLOG
> --nflog-prefix=SMTP_TRAFFIC:
> /sbin/iptables -t filter -A out_world_smtp_s3.4 -m set --match-set
> trap src -j ACCEPT
> /sbin/iptables -t filter -A out_world_smtp_s3 -p tcp --sport 25 -d
> 1.1.1.1 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> out_world_smtp_s3.4
> /sbin/iptables -t filter -A out_world_smtp_s3 -p tcp --sport 25 -d
> 2.2.2.2 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> out_world_smtp_s3.4
> /sbin/iptables -t filter -A out_world_smtp_s3 -p tcp --sport 25 -d
> 3.3.3.3 --dport 1024:65535 -m conntrack --ctstate ESTABLISHED -j
> out_world_smtp_s3.4