[Firehol-support] blocklists

Thu May 28 08:58:58 BST 2015

Of course, update-ipsets.sh does not have such issues...

On Thu, May 28, 2015 at 10:55 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
> An example of childish blocklist operations:
>
> I was searching for a VoIP blacklist and found voipbl.org.
> Ok. Good. I added it to update-ipsets.sh.
>
> Look how these guys suggest we would update their list:
>
> wget -qO - http://www.voipbl.org/update/ |\
>   awk '{print "if [ ! -z \""$1"\" -a \""$1"\" !=  \"#\" ]; then
> /sbin/iptables -A BLACKLIST-INPUT -s \""$1"\" -j DROP;fi;"}' | sh
>
> or
>
> wget -qO - $URL -O /tmp/voipbl.txt
> cat /tmp/voipbl.txt |\
>   awk '{ print "if [ ! -z \""$1"\" -a \""$1"\"  != \"#\" ]; then
> /usr/sbin/ipset  -A voipbl_temp \""$1"\" ;fi;"}' | sh
>
> I sent them an email (actually there is no way to send an email to
> them - I filled a form at the company site that created voipbl.org)
> saying: Hey guys, your script allows YOU to execute any command you
> like as root on all systems your script runs! Example: Add to your
> list this: $(echo>/tmp/voipbl-did-this). All the systems your script
> runs will have the file /tmp/voipbl-did-this created as root!
>
> Its been days now, No response! No fix!
>
> Hm....
>
> On Thu, May 28, 2015 at 9:32 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>> ssh-faker is a kind of honeypot. You can contribute your logs to
>> blocklist.de if you like (I also added all their lists in
>> update-ipsets.sh yesterday, check
>> https://github.com/ktsaou/blocklist-ipsets - I also added a section on
>> this page for the lists I trust
>> https://github.com/ktsaou/blocklist-ipsets#which-ones-to-use).
>>
>> FireHOL v3 also supports knocks, so instead of approving IPs for ssh
>> access, you can enable ssh for an IP, only if you first get a connect
>> on port A, then B, the C within 10 seconds, in which case ssh is open
>> for this IP for 30 seconds.
>>
>> You can also stop port scans with FireHOL v3. You block an IP if you
>> get more than X attempts to connect on not-allowed different IPs or
>> different ports within a few seconds. This is very effective. You can
>> honeypot any port you like. You can honeypot IPs too. If you use
>> synproxy with this, the attacker will believe there is a server
>> listening there.
>>
>> Costa
>>
>>
>> On Thu, May 28, 2015 at 1:10 AM, Rick Marshall <rjm at zenucom.com> wrote:
>>> Thanks for that Costa. I need to think some more on this, but I agree it's
>>> all in the lists - ingress yes, but egress seems to be the big one for
>>> serious fraud.
>>>
>>> Just to update according to Trend cybercrime last year worldwide was $450b
>>> and in the next 2-3 years is expected to reach $3t ! It's more than big
>>> business.
>>>
>>> I think your most interesting point was the poisoned list problem. The way
>>> these things work >50,000 new addresses/urls are identified every day
>>> (mostly urls I expect) which also means a lot disappear as well. Maintaining
>>> good lists without poisoning is difficult enough.
>>>
>>> We also use ssh-faker to block access to ssh in the first place - except for
>>> approved IPs. Sort of a 2-phase access and it's been very effective.
>>>
>>> Still no good answer for social engineering :(
>>>
>>>
>>> On 28/05/2015 1:17 am, Tsaousis, Costa wrote:
>>>
>>> live not leave! ok! :-)
>>>
>>> On Wed, May 27, 2015 at 5:02 PM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>>>
>>> I have to add that at the end of the day, what you need is lists, i.e.
>>> patterns the bad guys use:
>>>
>>> 1. IP blocklists to stop the bad guys at the network level
>>> 2. Application layer blocklists to stop known application level attacks
>>> 3. URL or domain blocklists to prevent your systems from contacting the bad
>>> guys
>>> 4. Intrusion detection lists that are capable of analyzing all the
>>> layers of communication and alert you when something is wrong
>>>
>>> Most security companies this is what they do. They maintain such kinds
>>> of lists. Even an anti-virus is a list of patterns and heuristics.
>>>
>>> Of course security companies also provide customized services, they
>>> collect your logs to correlate the traces of the attackers and figure
>>> out what the attackers did or trying to do. But at the end of day,
>>> they will block them at the firewall. When you are under an attack you
>>> may not afford to let the application level security handle it. If the
>>> attack is massive, these layers will go out of resources quickly. So
>>> what you need is a way to block them at the firewall.
>>>
>>> This is why I think ipsets are a key component (and I push it so
>>> much). They really share key knowledge between all of us.
>>>
>>> Of course, there are more to be learned. What will happen if a key
>>> blocklist most people trust is poisoned? What if somehow they have a
>>> large number of random false positives? Do they rank dynamic IPs
>>> properly?
>>>
>>> What seems missing at my view, is that the security teams around the
>>> world have not yet found a way to exchange their knowledge is a
>>> structured way, quickly. Each team, each company does whatever sees
>>> fit. A few release IP lists, others have DNSBLs, others provide their
>>> lists with a very well defined retention policy, others do not say
>>> anything about their policies, how they remove false positives, how
>>> they rank the attacks, etc. The InfoSec community seems very cryptic
>>> at my eyes... or probably it is still at its very first steps.
>>>
>>> At the other hand, global fraud is a 30bn USD business. The fraudsters
>>> and attackers are pros...
>>>
>>> Anyway, as I see it, the firewall builds walls. Blocklists are like
>>> doors on these walls, they allow some, they block some.
>>> Would you leave in a home without doors?
>>>
>>> Costa
>>>
>>> On Wed, May 27, 2015 at 10:24 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>>>
>>> Hi Rick.
>>>
>>> They are right. C&C was not just a promo. Even the attack I faced,
>>> from the outside, was mainly originating from C&C hosts around the
>>> world.
>>>
>>> Protecting a LAN of personal computers needs a different strategy
>>> compared to protecting a LAN of servers.
>>>
>>> For the servers for example a basic strategy should be something like this:
>>>
>>> 1. A good layer 3 firewall in the front, with stateful connection
>>> tracking, possibly with synproxy to help against SYN floods and a few
>>> basic traps to prevent port scans. FireHOL v3+ is good at this.
>>> synproxy support is excellent and its traps are like layer-3
>>> honeypots.
>>>
>>> 2. A few blocklists to keep the well known bad guys out of the
>>> services your firewall allows to pass. FireHOL v3+ is also good at
>>> this. Of course, the blocklists will not help you much if you get a
>>> zero-day attack (you are first to be attacked on the net).
>>>
>>> 3. An application level security layer (e.g. modsecurity for http).
>>> This will detect known attacks at the application layer (sql
>>> injection, known application vulnerabilities, etc). This also works
>>> with lists (pattern matching rules) that should be updated frequently.
>>>
>>> 4. fail2ban on all services where possible (you should even contribute
>>> to the blocklists with this). This will protect you against brute
>>> force attacks.
>>>
>>> 5. all outgoing http/https requests the servers make should go through
>>> a proxy, so that you can monitor what your servers are requesting from
>>> the internet and possibly add some security there (url blocklists -
>>> the 3rd set of lists you will need, anti-malware and anti-virus
>>> checks). Also, tt the firewall level, you should not have "client all
>>> accept" for servers. You should allow very specific things, probably
>>> with 'src' and 'dst' on every client rule.
>>>
>>> Some guys also add a snort just behind the firewall that sniffs all
>>> traffic and logs anomalies. In this case you will need a 4th type of
>>> lists, the snort alert rules. If you do this, then most probably you
>>> should add an ssl accelerator somewhere, so that it can read https
>>> traffic (this is also required if you add modsecurity).
>>>
>>> You can also contribute to the overall internet security by installing
>>> application honeyports - this will be a contribution to blocklists.
>>>
>>> Finally, you can design your application to handle security
>>> information, such as IP reputation data. Project Honey Pot provides
>>> such a mechanism through DNS (httpbl.org).
>>>
>>> On a LAN of personal computers, things are different. In this case the
>>> blocklists will help you after you are infected (there are a few lists
>>> that will block viruses, like zeus, feodo, pelevo, etc), but generally
>>> your public IPs will appear on the blocklists... On LAN PCs a proxy
>>> with anti-virus and anti-malware is required and a good anti-virus
>>> anti-malware is required on every PC, especially Windows. A snort
>>> installation can also help to detect anomalies on a LAN of PCs.
>>>
>>> Costa
>>>
>>>
>>> On Wed, May 27, 2015 at 2:51 AM, Rick Marshall <rjm at zenucom.com> wrote:
>>>
>>> I haven't looked at this closely yet :( Been rather busy :(:(
>>>
>>> However, I did have time to attend a security seminar headed by Trend Micro.
>>> Obviously this had a large promotion of their idea of security and how
>>> things are happening.
>>>
>>> All good.
>>>
>>> Question is that it seems the biggest threat now is from infected devices
>>> inside the network talking on their own - viruses/trojans etc to what Trend
>>> is calling command and control (C&C) servers that are directing the deep
>>> attacks.
>>>
>>> So, my question, we have traditionally focused on external threats in to the
>>> organisation. Can I now turn around and track internal going out (ie packets
>>> originated inside the network) and going to known C&C servers then
>>> log/report the originating machine and the destination C&C. Would identify
>>> infected machines and devices - phones etc - without having to worry about
>>> up to date anti-virus software for some of the more serious threats.
>>>
>>> I suspect this is the basis of many of the devices sold by companies like
>>> Trend and it is certainly a strengthening of firewalls.
>>>
>>> This would be valuable even if I have to subscribe to a frequently (hourly?)
>>> updated list of C&Cs.
>>>
>>> Please feel free to ell me this is already well covered. If not perhaps we
>>> could consider it.
>>>
>>> Thanks Costa.
>>>
>>>
>>> On 27/05/2015 8:59 am, Tsaousis, Costa wrote:
>>>
>>> It seems I am right.
>>>
>>> This says that the number of entries in an ipset does not affect its
>>> performance:
>>>
>>> http://people.netfilter.org/kadlec/nftest.pdf
>>>
>>> So, only the number of individual ipsets will make a difference and of
>>> course if you run blacklist(s) in 'full' or 'input'.
>>>
>>> Costa
>>>
>>>
>>>
>>> On Wed, May 27, 2015 at 1:26 AM, Tsaousis, Costa <costa at tsaousis.gr>
>>> wrote:
>>>
>>> Hi Whit,
>>>
>>> You are right to be concerned. I have not done any full tests on
>>> performance. However this is my experience so far:
>>>
>>> I believe there is no problem with big ipsets. Actually the number of
>>> IPs in an ipset might not be so relevant. The ipsets are sized so that
>>> one lookup per packet will do the job, even if there are hundreds of
>>> thousands of IPs in the ipset (I have not read the kernel source, but
>>> from the ipset configuration I see that there is a hashing algorithm
>>> that is sized properly to accomplish this and in a few cases when an
>>> ipset is updated from the command line, the kernel logs that re-hashed
>>> it). update-ipsets.sh forces a rehash by creating an new ipset and
>>> swapping the active with the new during the update.
>>>
>>> There are also a few very simple ways to speed it up, if you are so
>>> concerned:
>>>
>>> 1. If you need to use 10 blacklists, create one bigger, so that
>>> instead of 10 lookups, only 1 will be made. There is a tool (iprange)
>>> included in the contrib dir of firehol that can aggregate ipsets,
>>> giving an optimized netset result (example: cat *.ipset *.netset |
>>> grep -v "^#" | ./iprange -J). This can also be done with the standard
>>> aggregate tool on most distros (although it is significantly slower
>>> than iprange). This is a nice feature for update-ipsets.sh too. I'll
>>> add it...
>>>
>>> 2. Use the blacklist firehol helper with 'input' not 'full'. This will
>>> only check the FIRST packet of each connection, NOT ALL the packets.
>>> This limits the packets to be checked significantly.
>>>
>>> I have a few demanding installations:
>>>
>>> a. I use a few ipsets with tens of thousands of IPs each, on a
>>> firewall server that has sustained traffic from 30 to 50 Mbps, with
>>> one blacklist per ipset (so that I get a log with the ipset name that
>>> matched it), each of them running in 'full' mode (all packets
>>> checked). The overhead is ignorable. I cannot tell the difference with
>>> and without the blacklists active...
>>>
>>> b. I have another server/gateway with 4 VDSLs and 4 ADSLs (more than
>>> 250Mbps at peak) running again several ipsets in a similar fashion.
>>> Again I cannot tell the difference...
>>>
>>> Even on my home gateway, with a simple J1900 machine (a celeron quad
>>> core cpu) running ALL the ipsets in 'full' mode on my ADSL link, with
>>> the link at 100% utilization (20 Mbps in total: up + down), I cannot
>>> tell the difference with and without the blacklists...
>>>
>>> Keep in mind that I have not measured it. This is just the "feeling" I
>>> have by observing 'top' and 'vmstat'...
>>>
>>> Costa
>>>
>>>
>>> On Wed, May 27, 2015 at 12:09 AM, Whit Blauvelt <whit at transpect.com>
>>> wrote:
>>>
>>> Hi Costa,
>>>
>>> Since you've pushed ipsets so far, have you seen any noticable system
>>> performance issues when there are thousands, or tens of thousands, of
>>> IPs in
>>> your ipsets? On the one hand I'm thinking, "What a great idea. I'll just
>>> blacklist all the IPs on the blacklists." On the other hand, there must
>>> be
>>> some threshold where it becomes computationally expensive, and I don't
>>> have
>>> the measure of that.
>>>
>>> Thanks,
>>>
>>> Whit
>>>
>>> _______________________________________________
>>> Firehol-support mailing list
>>> Firehol-support at lists.firehol.org
>>> http://lists.firehol.org/mailman/listinfo/firehol-support
>>>
>>> --
>>> *Rick Marshall*
>>> Director
>>> Zenucom Pty Ltd
>>> 0411 287530
>>> http://www.zenucom.com
>>> _______________________________________________
>>> Firehol-support mailing list
>>> Firehol-support at lists.firehol.org
>>> http://lists.firehol.org/mailman/listinfo/firehol-support
>>>
>>>
>>> --
>>> Rick Marshall
>>> Director
>>> Zenucom Pty Ltd
>>> 0411 287530
>>> http://www.zenucom.com