[Firehol-devs] blocklists

[Tsaousis, Costa]

Hi all,

If you are using update-ipsets.sh, I have renamed a few ipsets to have
a better naming scheme (there are more than 50 blacklists supported so
far, so this was required).

Next time you update, please make sure your firehol.conf refers to the
proper names.

The script will rename the files on disk automatically and create a
link from the new name to the old, so that if you restart the
firewall, or reboot the machine, it will not complain, but keep in
mind that the old set will not be updated by update-ipsets.sh anymore.
Just run it and look for links in /etc/firehol/ipsets (it will also
print a message on console the first time it renames something).

Also, for those of you having a public VoIP server, update-ipsets.sh
now supports VoIPBL.org.

Last, I consider adding these features to update-ipsets.sh:

1. Check if IPs in an ipset you create are part of any blacklist. For
example, let's say that you have the subnets 1.2.3.0/24 and
4.5.6.0/24. On every update update-ipsets.sh will check if any of your
IPs are blacklisted by the updated ipsets and send you an email about
the fact.

2. Check if a considerable change is made to any ipset. For example,
let's say that you trust blocklist.de, but you want to know if they
make a considerable update (e.g. +20%) to it.

3. Check and possibly block the update of an ipset if it matches a
number of IPs on one or more other ipsets, For example, let's say that
you trust blocklist.de but you want to block its automatic updates if
it matches more than 1000 IPs of your country (since update-ipsets.sh
already has geoip data, this is relatively easy).

Let me know what you think.

Costa


On Mon, May 25, 2015 at 2:25 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
> Hi again,
>
> update-ipsets.sh is now able to compare the blocklists with each other.
> Check the comparison results at the bottom of
> https://github.com/ktsaou/blocklist-ipsets
>
> Costa
>
>
> On Sat, May 23, 2015 at 7:08 PM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>> Hi all,
>>
>> update-ipsets.sh is now able to download, parse and update (while the
>> firewall is running), the free MaxMind Geolite2 Country Database.
>>
>> I have also included it in the https://github.com/ktsaou/blocklist-ipsets repo.
>>
>> Direct link to geolite2 ipsets here:
>> https://github.com/ktsaou/blocklist-ipsets/tree/master/geolite2_country
>>
>> Costa
>>
>>
>> On Sun, May 17, 2015 at 11:16 PM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>>> Hi all,
>>>
>>> Recently I faced quite a challenge: 37.500 IPs from all over the world
>>> were attacking my servers for 2 weeks. It was a challenge because all
>>> the requests these IPs did were legitimate. They were not trying to
>>> damage or take control of anything. Each IP was used just a few times
>>> per day, to remain unnoticed. It was very hard to pinpoint them, to
>>> separate the attack from the normal traffic.
>>>
>>> Anyway, I managed to block them. Actually I had them blocked for 4
>>> days and then, suddenly they stopped...
>>>
>>> What I found in the process, is that the attackers were using open
>>> proxies, command and control compromised hosts, and who knows what
>>> else, to synchronize the attack.
>>>
>>> Another interesting observation is that their IPs seem to have a large
>>> overlap with anti-spam blacklists. They seem to be using the same
>>> hosts for both spamming and web attacks.
>>>
>>> In the last few days, I tried to extend update-ipsets.sh a lot. I
>>> think I have now included in it, all the freely available IP
>>> blocklists. If you find any missing, please send me a note to add it.
>>>
>>> I have also created a new github repo at
>>> https://github.com/ktsaou/blocklist-ipsets which is automatically
>>> updated by my update-ipsets.sh. This repo mirrors all the blocklists I
>>> found and also generates a nice table at the bottom of the page, with
>>> some facts and info about each list.
>>>
>>> Normally, as a FireHOL v3 user you don't need to use this repo.
>>> update-ipsets.sh generates all the ipsets from scratch, so it can do
>>> it for you too, on your servers.
>>>
>>> Unfortunately, there are a lot of very useful blacklists that are only
>>> available as a DNSBL, not as a data feed. DNSBL is mainly for
>>> anti-spam, but as I said above, web attackers are using the exact same
>>> hosts for web attacks and forum spam. I tried contacting several
>>> DNSBLs for releasing their IP lists, without a positive response so
>>> far.
>>>
>>> Anyway, I hope you will find all these useful. If you have any
>>> suggestions, please let me know.
>>>
>>> Costa