[Firehol-support] Dynamic hosts (eg no-ip) and fail2ban

[Jason Harris]

I just spun up a new instance as an example on here. There is nothing else but a stock install of debian 8.2 hosted on digital ocean.

I apt-get installed: git sudo python-pip autoconf build-essential curl ipset
Then ran the script at: https://github.com/firehol/blocklist-ipsets/wiki/Installing-update-ipsets

I’ll send you the login detail in a separate email. But here is the output of `update-ipsets -v` with the contents of the files following:

                                   | 
                  firehol_anonymous|  DISABLED  
                                   | To enable run: update-ipsets enable firehol_anonymous
Loading ipset definitions from: '/etc/firehol/ipsets.d'
Loading ipset definition file: '/etc/firehol/ipsets.d/mywhitelist.conf'
                                   | 
                        mywhitelist| parsing attributes: 
                                   | converting with 'hostname_resolver'
                                   |  ERROR  converted file is empty.
 ERROR : '/etc/firehol/ipsets.d/mywhitelist.conf' failed
Supplied ipsets directory '/usr/share/firehol/ipsets.d' does not exist. Ignoring it.
Supplied ipsets directory '/root/.update-ipsets/ipsets.d' does not exist. Ignoring it.

Cleaning up temporary files in /tmp/update-ipsets-O3Z0xZFOWe.
Completed successfully.
root at testfirehol:/etc/firehol/ipsets# cd ..
root at testfirehol:/etc/firehol# ls
firehol.conf.example  fireqos.conf.example  ipsets  ipsets.d  services
root at testfirehol:/etc/firehol# more ipsets/*

*** ipsets/errors: directory ***


*** ipsets/history: directory ***

::::::::::::::
ipsets/mywhitelist.source
::::::::::::::
google.com
yahoo.com
cnn.com
namecheap.com
root at testfirehol:/etc/firehol# more ipsets.d/*
# update its timestamp, to force reprocessing
touch /etc/firehol/ipsets/mywhitelist.source

# configuration about the list
update mywhitelist 1 0 ipv4 ip "" hostname_resolver "category" "a whitelist for me" "Jason Harris" "a url for info for the list"
root at testfirehol:/etc/firehol# 

Thanks!
   Jason


> On Dec 13, 2015, at 5:19 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
> 
> to disable a list in update-ipsets, just delete its .source file.
> The enable command just touches it.
> 
> Regarding the conversion error, could you please post
> /etc/firehol/ipsets/whitelist.source?
> 
> If you don't want to post your whitelist, this should work:
> 
> iprange </etc/firehol/ipsets/whitelist.source
> 
> it should give you the IPs of your hostnames.
> 
> Costa
> 
> On Sat, Dec 12, 2015 at 8:02 PM, Jason Harris <jason at unifiedthought.com> wrote:
>> 
>>> On Dec 8, 2015, at 1:48 PM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>>> 
>>>> Ok. but I can use hostnames like eg sub.mydomain.com with ipsets?
>>> 
>>> Yes, you have to resolve them first though. iprange does this.
>>> 
>>> 
>>>> The link: https://github.com/ktsaou/firehol/blob/master/contrib/update-ipsets.sh on the page: https://github.com/firehol/firehol/wiki/Working-with-IPSETs is dead. I google around a bit and am sure I am just missing this but am having trouble finding this script.
>>> 
>>> Thanks! I fixed the link.
>>> However, it is installed with firehol v3 (the github version).
>>> 
>>> 
>>>> So I am not sure how to actually update the ipset I have dynamically. Maybe I could build a second ipset and using 'ipset swap’? But it seems to be from the instructions below that I should use update-upsets?
>>> 
>>> ok.
>>> 
>>> 1. Install firehol v3 (this will also require from you to install
>>> iprange). If you don't know how to do it, follow this procedure:
>>> https://github.com/firehol/blocklist-ipsets/wiki/Installing-update-ipsets
>> 
>> Ok. I got around to having some time this weekend. To build this (on latest debian jessie) in addition to your listed build steps you also need:
>> 
>>   apt-get install autoconf build-essential curl ipset
>> 
>> This is kind of disappointing since it loads a bunch of gunk onto a production node, (i.e. some 200MB’s of stuff just to get the small firehol firewall. I guess I could remove most of this after the build process… Still this is not so nice for eg ansible,chef, puppet, saltstack, etc which are used to provision vm’s.)
>> 
>>> 2. Create a new file called /etc/firehol/ipsets/myhostsnames.source
>>> Put there any hostnames you like.
>>> 
>>> 3. To resolve its contents to IPs you have to configure update-ipsets
>>> (https://github.com/firehol/blocklist-ipsets/wiki/Extending-update-ipsets).
>>> Briefly:
>>> 
>>> a. create the file  /etc/firehol/ipsets.d/myhostname.conf
>>> b. using this content (copy and paste it):
>>> 
>>> # update its timestamp, to force reprocessing
>>> touch /etc/firehol/ipsets/myhostsnames.source
>>> 
>>> # configuration about the list
>>> update myhostnames 1 0 ipv4 ip "" hostname_resolver "category" "some
>>> info about the list" "your name" "a url for info for the list"
>>> 
>>> c. run:
>>> 
>>> update-ipsets enable myhostnames
>> 
>> Ok. So I followed these instructions. First there appears to be no update-ipsets disable myhostnames? (I made a mistake in one of the configurations and it would be nice to undo it…)
>> 
>>> d. check it with (this is also the command you need to put at cron):
>>> 
>>> update-upsets
>> 
>> For me this fails with the following message (using update-upsets -v)
>> 
>>                  firehol_anonymous|  DISABLED
>>                                   | To enable run: update-ipsets enable firehol_anonymous
>> Loading ipset definitions from: '/etc/firehol/ipsets.d'
>> Loading ipset definition file: '/etc/firehol/ipsets.d/whitelist.conf'
>>                                   |
>>                          whitelist| parsing attributes:
>>                                   | converting with 'hostname_resolver'
>>                                   |  ERROR  converted file is empty.
>> ERROR : '/etc/firehol/ipsets.d/whitelist.conf' failed
>> Supplied ipsets directory '/usr/share/firehol/ipsets.d' does not exist. Ignoring it.
>> Supplied ipsets directory '/root/.update-ipsets/ipsets.d' does not exist. Ignoring it.
>> 
>> Cleaning up temporary files in /tmp/update-ipsets-9B34pYTy0N.
>> Completed successfully.
>> [root at tester:/etc/firehol/ipsets] $ ls
>> 
>> Any hints on what went wrong? The errors directory is empty...
>> 
>> Thanks!
>>   Jason
>> 
>>> If successful, the file /etc/firehol/ipsets/myhostnames.ipset should
>>> be there with all the IPs.
>>> 
>>> 4. In firehol.conf use
>>> 
>>> ipset4 MYHOSTNAMES addfile ipsets/myhostnames.ipset
>>> 
>>> and later in server/client/nat statements: src ipset:MYHOSTNAMES
>>> 
>> 
>