[Firehol-support] blocklists

Wed May 27 16:17:19 BST 2015

live not leave! ok! :-)

On Wed, May 27, 2015 at 5:02 PM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
> I have to add that at the end of the day, what you need is lists, i.e.
> patterns the bad guys use:
>
> 1. IP blocklists to stop the bad guys at the network level
> 2. Application layer blocklists to stop known application level attacks
> 3. URL or domain blocklists to prevent your systems from contacting the bad guys
> 4. Intrusion detection lists that are capable of analyzing all the
> layers of communication and alert you when something is wrong
>
> Most security companies this is what they do. They maintain such kinds
> of lists. Even an anti-virus is a list of patterns and heuristics.
>
> Of course security companies also provide customized services, they
> collect your logs to correlate the traces of the attackers and figure
> out what the attackers did or trying to do. But at the end of day,
> they will block them at the firewall. When you are under an attack you
> may not afford to let the application level security handle it. If the
> attack is massive, these layers will go out of resources quickly. So
> what you need is a way to block them at the firewall.
>
> This is why I think ipsets are a key component (and I push it so
> much). They really share key knowledge between all of us.
>
> Of course, there are more to be learned. What will happen if a key
> blocklist most people trust is poisoned? What if somehow they have a
> large number of random false positives? Do they rank dynamic IPs
> properly?
>
> What seems missing at my view, is that the security teams around the
> world have not yet found a way to exchange their knowledge is a
> structured way, quickly. Each team, each company does whatever sees
> fit. A few release IP lists, others have DNSBLs, others provide their
> lists with a very well defined retention policy, others do not say
> anything about their policies, how they remove false positives, how
> they rank the attacks, etc. The InfoSec community seems very cryptic
> at my eyes... or probably it is still at its very first steps.
>
> At the other hand, global fraud is a 30bn USD business. The fraudsters
> and attackers are pros...
>
> Anyway, as I see it, the firewall builds walls. Blocklists are like
> doors on these walls, they allow some, they block some.
> Would you leave in a home without doors?
>
> Costa
>
> On Wed, May 27, 2015 at 10:24 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>> Hi Rick.
>>
>> They are right. C&C was not just a promo. Even the attack I faced,
>> from the outside, was mainly originating from C&C hosts around the
>> world.
>>
>> Protecting a LAN of personal computers needs a different strategy
>> compared to protecting a LAN of servers.
>>
>> For the servers for example a basic strategy should be something like this:
>>
>> 1. A good layer 3 firewall in the front, with stateful connection
>> tracking, possibly with synproxy to help against SYN floods and a few
>> basic traps to prevent port scans. FireHOL v3+ is good at this.
>> synproxy support is excellent and its traps are like layer-3
>> honeypots.
>>
>> 2. A few blocklists to keep the well known bad guys out of the
>> services your firewall allows to pass. FireHOL v3+ is also good at
>> this. Of course, the blocklists will not help you much if you get a
>> zero-day attack (you are first to be attacked on the net).
>>
>> 3. An application level security layer (e.g. modsecurity for http).
>> This will detect known attacks at the application layer (sql
>> injection, known application vulnerabilities, etc). This also works
>> with lists (pattern matching rules) that should be updated frequently.
>>
>> 4. fail2ban on all services where possible (you should even contribute
>> to the blocklists with this). This will protect you against brute
>> force attacks.
>>
>> 5. all outgoing http/https requests the servers make should go through
>> a proxy, so that you can monitor what your servers are requesting from
>> the internet and possibly add some security there (url blocklists -
>> the 3rd set of lists you will need, anti-malware and anti-virus
>> checks). Also, tt the firewall level, you should not have "client all
>> accept" for servers. You should allow very specific things, probably
>> with 'src' and 'dst' on every client rule.
>>
>> Some guys also add a snort just behind the firewall that sniffs all
>> traffic and logs anomalies. In this case you will need a 4th type of
>> lists, the snort alert rules. If you do this, then most probably you
>> should add an ssl accelerator somewhere, so that it can read https
>> traffic (this is also required if you add modsecurity).
>>
>> You can also contribute to the overall internet security by installing
>> application honeyports - this will be a contribution to blocklists.
>>
>> Finally, you can design your application to handle security
>> information, such as IP reputation data. Project Honey Pot provides
>> such a mechanism through DNS (httpbl.org).
>>
>> On a LAN of personal computers, things are different. In this case the
>> blocklists will help you after you are infected (there are a few lists
>> that will block viruses, like zeus, feodo, pelevo, etc), but generally
>> your public IPs will appear on the blocklists... On LAN PCs a proxy
>> with anti-virus and anti-malware is required and a good anti-virus
>> anti-malware is required on every PC, especially Windows. A snort
>> installation can also help to detect anomalies on a LAN of PCs.
>>
>> Costa
>>
>>
>> On Wed, May 27, 2015 at 2:51 AM, Rick Marshall <rjm at zenucom.com> wrote:
>>> I haven't looked at this closely yet :( Been rather busy :(:(
>>>
>>> However, I did have time to attend a security seminar headed by Trend Micro.
>>> Obviously this had a large promotion of their idea of security and how
>>> things are happening.
>>>
>>> All good.
>>>
>>> Question is that it seems the biggest threat now is from infected devices
>>> inside the network talking on their own - viruses/trojans etc to what Trend
>>> is calling command and control (C&C) servers that are directing the deep
>>> attacks.
>>>
>>> So, my question, we have traditionally focused on external threats in to the
>>> organisation. Can I now turn around and track internal going out (ie packets
>>> originated inside the network) and going to known C&C servers then
>>> log/report the originating machine and the destination C&C. Would identify
>>> infected machines and devices - phones etc - without having to worry about
>>> up to date anti-virus software for some of the more serious threats.
>>>
>>> I suspect this is the basis of many of the devices sold by companies like
>>> Trend and it is certainly a strengthening of firewalls.
>>>
>>> This would be valuable even if I have to subscribe to a frequently (hourly?)
>>> updated list of C&Cs.
>>>
>>> Please feel free to ell me this is already well covered. If not perhaps we
>>> could consider it.
>>>
>>> Thanks Costa.
>>>
>>>
>>> On 27/05/2015 8:59 am, Tsaousis, Costa wrote:
>>>>
>>>> It seems I am right.
>>>>
>>>> This says that the number of entries in an ipset does not affect its
>>>> performance:
>>>>
>>>> http://people.netfilter.org/kadlec/nftest.pdf
>>>>
>>>> So, only the number of individual ipsets will make a difference and of
>>>> course if you run blacklist(s) in 'full' or 'input'.
>>>>
>>>> Costa
>>>>
>>>>
>>>>
>>>> On Wed, May 27, 2015 at 1:26 AM, Tsaousis, Costa <costa at tsaousis.gr>
>>>> wrote:
>>>>>
>>>>> Hi Whit,
>>>>>
>>>>> You are right to be concerned. I have not done any full tests on
>>>>> performance. However this is my experience so far:
>>>>>
>>>>> I believe there is no problem with big ipsets. Actually the number of
>>>>> IPs in an ipset might not be so relevant. The ipsets are sized so that
>>>>> one lookup per packet will do the job, even if there are hundreds of
>>>>> thousands of IPs in the ipset (I have not read the kernel source, but
>>>>> from the ipset configuration I see that there is a hashing algorithm
>>>>> that is sized properly to accomplish this and in a few cases when an
>>>>> ipset is updated from the command line, the kernel logs that re-hashed
>>>>> it). update-ipsets.sh forces a rehash by creating an new ipset and
>>>>> swapping the active with the new during the update.
>>>>>
>>>>> There are also a few very simple ways to speed it up, if you are so
>>>>> concerned:
>>>>>
>>>>> 1. If you need to use 10 blacklists, create one bigger, so that
>>>>> instead of 10 lookups, only 1 will be made. There is a tool (iprange)
>>>>> included in the contrib dir of firehol that can aggregate ipsets,
>>>>> giving an optimized netset result (example: cat *.ipset *.netset |
>>>>> grep -v "^#" | ./iprange -J). This can also be done with the standard
>>>>> aggregate tool on most distros (although it is significantly slower
>>>>> than iprange). This is a nice feature for update-ipsets.sh too. I'll
>>>>> add it...
>>>>>
>>>>> 2. Use the blacklist firehol helper with 'input' not 'full'. This will
>>>>> only check the FIRST packet of each connection, NOT ALL the packets.
>>>>> This limits the packets to be checked significantly.
>>>>>
>>>>> I have a few demanding installations:
>>>>>
>>>>> a. I use a few ipsets with tens of thousands of IPs each, on a
>>>>> firewall server that has sustained traffic from 30 to 50 Mbps, with
>>>>> one blacklist per ipset (so that I get a log with the ipset name that
>>>>> matched it), each of them running in 'full' mode (all packets
>>>>> checked). The overhead is ignorable. I cannot tell the difference with
>>>>> and without the blacklists active...
>>>>>
>>>>> b. I have another server/gateway with 4 VDSLs and 4 ADSLs (more than
>>>>> 250Mbps at peak) running again several ipsets in a similar fashion.
>>>>> Again I cannot tell the difference...
>>>>>
>>>>> Even on my home gateway, with a simple J1900 machine (a celeron quad
>>>>> core cpu) running ALL the ipsets in 'full' mode on my ADSL link, with
>>>>> the link at 100% utilization (20 Mbps in total: up + down), I cannot
>>>>> tell the difference with and without the blacklists...
>>>>>
>>>>> Keep in mind that I have not measured it. This is just the "feeling" I
>>>>> have by observing 'top' and 'vmstat'...
>>>>>
>>>>> Costa
>>>>>
>>>>>
>>>>> On Wed, May 27, 2015 at 12:09 AM, Whit Blauvelt <whit at transpect.com>
>>>>> wrote:
>>>>>>
>>>>>> Hi Costa,
>>>>>>
>>>>>> Since you've pushed ipsets so far, have you seen any noticable system
>>>>>> performance issues when there are thousands, or tens of thousands, of
>>>>>> IPs in
>>>>>> your ipsets? On the one hand I'm thinking, "What a great idea. I'll just
>>>>>> blacklist all the IPs on the blacklists." On the other hand, there must
>>>>>> be
>>>>>> some threshold where it becomes computationally expensive, and I don't
>>>>>> have
>>>>>> the measure of that.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Whit
>>>>
>>>> _______________________________________________
>>>> Firehol-support mailing list
>>>> Firehol-support at lists.firehol.org
>>>> http://lists.firehol.org/mailman/listinfo/firehol-support
>>>
>>>
>>> --
>>> *Rick Marshall*
>>> Director
>>> Zenucom Pty Ltd
>>> 0411 287530
>>> http://www.zenucom.com
>>> _______________________________________________
>>> Firehol-support mailing list
>>> Firehol-support at lists.firehol.org
>>> http://lists.firehol.org/mailman/listinfo/firehol-support