[Firehol-support] Dynamic hosts (eg no-ip) and fail2ban
Jason Harris
jason at unifiedthought.com
Wed Dec 23 18:52:33 GMT 2015
It didn’t work with the version I compiled using the sources of around 10 to 15 days ago.
root at testfirehol:~# printf "www.google.com\nfirehol.org\niplists.firehol.org\n" | iprange
root at testfirehol:~# man iprange
No manual entry for iprange
See 'man 7 undocumented' for help when manual pages are not available.
root at testfirehol:~# iprange --version
1.0.2_master
root at testfirehol:~#
I’ll try again now.
Ahh… That new change fixed the single test you gave above. After rebuilding the droplet and reinstalling from git sources I get the following:
root at testfirehol:~# printf "www.google.com\nfirehol.org\niplists.firehol.org\n" | iprange
5.196.125.115
23.235.43.133
173.194.65.99
173.194.65.103
173.194.65.104/31
173.194.65.106
173.194.65.147
root at testfirehol:~# iprange --version
1.0.3_master
One question the fix that you put in, did it make it into 3.0.0?
In any case the instructions now work as you described them. I get the following:
...
whitelistednames| parsing attributes:
| source file has been updated
| converting with 'hostname_resolver'
iprange: DNS: 'bob.robsfdsd.com' failed permanently: Name or service not known
| parsing attributes:
| SAVED no need to load ipset in kernel
| version 3, 1 unique IPs
'/etc/firehol/ipsets.d/whitelistednames.conf' completed
Supplied ipsets directory '/usr/share/firehol/ipsets.d' does not exist. Ignoring it.
Supplied ipsets directory '/root/.update-ipsets/ipsets.d' does not exist. Ignoring it.
Cleaning up temporary files in /tmp/update-ipsets-BX6iJZO6d6.
Completed successfully.
root at testfirehol:/etc/firehol/ipsets# more whitelistednames.source
bob.robsfdsd.com
namecheap.com
root at testfirehol:/etc/firehol/ipsets# more whitelistednames.ipset
#...
199.59.161.100
root at testfirehol:/etc/firehol/ipsets#
So this looks like it is really working correctly!
Thank you so much!
Cheers,
Jason
> On Dec 23, 2015, at 10:35 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>
> I thought you solved it.
>
> I see it says: converting with 'hostname_resolver'
> But then: ERROR converted file is empty.
>
> The key question is if hostname_resolver works for your list.
>
> What hostname_resolver does is essentially this:
>
> iprange </etc/firehol/ipsets/mywhitelist.source
>> /etc/firehol/ipsets/mywhitelist.ipset
>
> Does it work if you run it by hand?
>
> For example:
>
> printf "www.google.com\nfirehol.org\niplists.firehol.org\n" | iprange
>
> Keep in mind I found a bug, in iprange where the last few hostnames in
> the source set, in certain cases were resolved, but were not included
> in the result. Try the github version of iprange.
>
> Costa
>
>
> On Wed, Dec 23, 2015 at 7:31 AM, Jason Harris <jason at unifiedthought.com> wrote:
>> Hey Tsaousis,
>>
>> Did you get a chance to see what was wrong with the configuration, below? I am still really keen to get something going here. (I really like firehol and have just been using it for some of the networking on some LXC containers. It really allows the vey nice specification of firewall / routing rules.)
>>
>> Along with Phil’s comments I can now create .deb’s for firewall and deploy them so everything is set up except for these ipsets…
>>
>> Thanks for any help!
>> Jas
>>
>>> I just spun up a new instance as an example on here. There is nothing else but a stock install of debian 8.2 hosted on digital ocean.
>>>
>>> I apt-get installed: git sudo python-pip autoconf build-essential curl ipset
>>> Then ran the script at: https://github.com/firehol/blocklist-ipsets/wiki/Installing-update-ipsets
>>>
>>> I’ll send you the login detail in a separate email. But here is the output of `update-ipsets -v` with the contents of the files following:
>>>
>>> |
>>> firehol_anonymous| DISABLED
>>> | To enable run: update-ipsets enable firehol_anonymous
>>> Loading ipset definitions from: '/etc/firehol/ipsets.d'
>>> Loading ipset definition file: '/etc/firehol/ipsets.d/mywhitelist.conf'
>>> |
>>> mywhitelist| parsing attributes:
>>> | converting with 'hostname_resolver'
>>> | ERROR converted file is empty.
>>> ERROR : '/etc/firehol/ipsets.d/mywhitelist.conf' failed
>>> Supplied ipsets directory '/usr/share/firehol/ipsets.d' does not exist. Ignoring it.
>>> Supplied ipsets directory '/root/.update-ipsets/ipsets.d' does not exist. Ignoring it.
>>>
>>> Cleaning up temporary files in /tmp/update-ipsets-O3Z0xZFOWe.
>>> Completed successfully.
>>> root at testfirehol:/etc/firehol/ipsets# cd ..
>>> root at testfirehol:/etc/firehol# ls
>>> firehol.conf.example fireqos.conf.example ipsets ipsets.d services
>>> root at testfirehol:/etc/firehol# more ipsets/*
>>>
>>> *** ipsets/errors: directory ***
>>>
>>>
>>> *** ipsets/history: directory ***
>>>
>>> ::::::::::::::
>>> ipsets/mywhitelist.source
>>> ::::::::::::::
>>> google.com
>>> yahoo.com
>>> cnn.com
>>> namecheap.com
>>> root at testfirehol:/etc/firehol# more ipsets.d/*
>>> # update its timestamp, to force reprocessing
>>> touch /etc/firehol/ipsets/mywhitelist.source
>>>
>>> # configuration about the list
>>> update mywhitelist 1 0 ipv4 ip "" hostname_resolver "category" "a whitelist for me" "Jason Harris" "a url for info for the list"
>>> root at testfirehol:/etc/firehol#
>>>
>>> Thanks!
>>> Jason
>>>
>>>
>>>> On Dec 13, 2015, at 5:19 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>>>>
>>>> to disable a list in update-ipsets, just delete its .source file.
>>>> The enable command just touches it.
>>>>
>>>> Regarding the conversion error, could you please post
>>>> /etc/firehol/ipsets/whitelist.source?
>>>>
>>>> If you don't want to post your whitelist, this should work:
>>>>
>>>> iprange </etc/firehol/ipsets/whitelist.source
>>>>
>>>> it should give you the IPs of your hostnames.
>>>>
>>>> Costa
>>>>
>>>> On Sat, Dec 12, 2015 at 8:02 PM, Jason Harris <jason at unifiedthought.com> wrote:
>>>>>
>>>>>> On Dec 8, 2015, at 1:48 PM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>>>>>>
>>>>>>> Ok. but I can use hostnames like eg sub.mydomain.com with ipsets?
>>>>>>
>>>>>> Yes, you have to resolve them first though. iprange does this.
>>>>>>
>>>>>>
>>>>>>> The link: https://github.com/ktsaou/firehol/blob/master/contrib/update-ipsets.sh on the page: https://github.com/firehol/firehol/wiki/Working-with-IPSETs is dead. I google around a bit and am sure I am just missing this but am having trouble finding this script.
>>>>>>
>>>>>> Thanks! I fixed the link.
>>>>>> However, it is installed with firehol v3 (the github version).
>>>>>>
>>>>>>
>>>>>>> So I am not sure how to actually update the ipset I have dynamically. Maybe I could build a second ipset and using 'ipset swap’? But it seems to be from the instructions below that I should use update-upsets?
>>>>>>
>>>>>> ok.
>>>>>>
>>>>>> 1. Install firehol v3 (this will also require from you to install
>>>>>> iprange). If you don't know how to do it, follow this procedure:
>>>>>> https://github.com/firehol/blocklist-ipsets/wiki/Installing-update-ipsets
>>>>>
>>>>> Ok. I got around to having some time this weekend. To build this (on latest debian jessie) in addition to your listed build steps you also need:
>>>>>
>>>>> apt-get install autoconf build-essential curl ipset
>>>>>
>>>>> This is kind of disappointing since it loads a bunch of gunk onto a production node, (i.e. some 200MB’s of stuff just to get the small firehol firewall. I guess I could remove most of this after the build process… Still this is not so nice for eg ansible,chef, puppet, saltstack, etc which are used to provision vm’s.)
>>>>>
>>>>>> 2. Create a new file called /etc/firehol/ipsets/myhostsnames.source
>>>>>> Put there any hostnames you like.
>>>>>>
>>>>>> 3. To resolve its contents to IPs you have to configure update-ipsets
>>>>>> (https://github.com/firehol/blocklist-ipsets/wiki/Extending-update-ipsets).
>>>>>> Briefly:
>>>>>>
>>>>>> a. create the file /etc/firehol/ipsets.d/myhostname.conf
>>>>>> b. using this content (copy and paste it):
>>>>>>
>>>>>> # update its timestamp, to force reprocessing
>>>>>> touch /etc/firehol/ipsets/myhostsnames.source
>>>>>>
>>>>>> # configuration about the list
>>>>>> update myhostnames 1 0 ipv4 ip "" hostname_resolver "category" "some
>>>>>> info about the list" "your name" "a url for info for the list"
>>>>>>
>>>>>> c. run:
>>>>>>
>>>>>> update-ipsets enable myhostnames
>>>>>
>>>>> Ok. So I followed these instructions. First there appears to be no update-ipsets disable myhostnames? (I made a mistake in one of the configurations and it would be nice to undo it…)
>>>>>
>>>>>> d. check it with (this is also the command you need to put at cron):
>>>>>>
>>>>>> update-upsets
>>>>>
>>>>> For me this fails with the following message (using update-upsets -v)
>>>>>
>>>>> firehol_anonymous| DISABLED
>>>>> | To enable run: update-ipsets enable firehol_anonymous
>>>>> Loading ipset definitions from: '/etc/firehol/ipsets.d'
>>>>> Loading ipset definition file: '/etc/firehol/ipsets.d/whitelist.conf'
>>>>> |
>>>>> whitelist| parsing attributes:
>>>>> | converting with 'hostname_resolver'
>>>>> | ERROR converted file is empty.
>>>>> ERROR : '/etc/firehol/ipsets.d/whitelist.conf' failed
>>>>> Supplied ipsets directory '/usr/share/firehol/ipsets.d' does not exist. Ignoring it.
>>>>> Supplied ipsets directory '/root/.update-ipsets/ipsets.d' does not exist. Ignoring it.
>>>>>
>>>>> Cleaning up temporary files in /tmp/update-ipsets-9B34pYTy0N.
>>>>> Completed successfully.
>>>>> [root at tester:/etc/firehol/ipsets] $ ls
>>>>>
>>>>> Any hints on what went wrong? The errors directory is empty...
>>>>>
>>>>> Thanks!
>>>>> Jason
>>>>>
>>>>>> If successful, the file /etc/firehol/ipsets/myhostnames.ipset should
>>>>>> be there with all the IPs.
>>>>>>
>>>>>> 4. In firehol.conf use
>>>>>>
>>>>>> ipset4 MYHOSTNAMES addfile ipsets/myhostnames.ipset
>>>>>>
>>>>>> and later in server/client/nat statements: src ipset:MYHOSTNAMES
>>>>>>
>>>>>
>>>>
>>>
>
More information about the Firehol-support
mailing list