[Firehol-support] bursting by IP

[Tsaousis, Costa]

Tommy, read this: http://firehol.org/guides/firehol-welcome/

It will have you started with firehol quickly...

Costa


On Sun, Aug 9, 2015 at 9:08 PM, Tommy Butler <networkspeedy at gmail.com> wrote:
> This seems like a really neat approach.  I'm going to go over it paying
> close attention to the interaction between firehol and fireqos.  It shows a
> lot of promise, and I really appreciate your reply!  I'm having some issues
> with firehol I'll have to work out first though.  The auto-setup feature of
> firehol completely takes out all but my primary IP address, and blocks all
> other traffic.  Any other IP addresses bound to eth0 get blocked.  So I'm
> going to have to sort through that first.
>
> On Fri, Aug 7, 2015 at 7:45 PM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
>>
>> Hi Tommy,
>>
>> tc is a complex thing. I find it difficult to understand (or even use)
>> certain advanced features.
>>
>> From what I have seen so far, I believe there is no way to apply per
>> client policing or shaping. The most advanced examples I have seen are
>> kinds of policing per match (like the one I give you below).
>>
>> Unfortunately, tc documentation is very criptic and sparse.
>>
>> Anyway, here are a few ideas:
>>
>>
>> 1. I you have many services, you can favour a percentage of HTTP to a
>> higher class, like this:
>>
>> interface ... rate 100Mbps ...
>>
>>    class high commit 50%
>>       match proto tcp sport 80,443 limit 10Mbps
>>
>>    class low commit 50%
>>       match proto tcp sport 80,443
>>
>> The above will allow the first 10Mbps of HTTP/HTTPS traffic to be
>> classified in 'high' class and the rest to 'low' class.
>> Of course this has only meaning when you have many services and you
>> want to favour a service over another.
>>
>>
>> 2. You can favour small packets over the bigger ones:
>>
>> interface ... rate 100Mbps ...
>>
>>    class high commit 50%
>>       match proto tcp sport 80,443 police "mtu 512 continue"
>>
>>    class low commit 50%
>>       match proto tcp sport 80,443
>>
>> The above will put all HTTP/HTTPS packets that are smaller than 512
>> bytes to 'high' class, all other to 'low' class.
>> Again, this has only meaning when you have many services and you want
>> to favour a service over another.
>>
>>
>> 3. You can use a combination of firehol and fireqos like this: in
>> firehol use ipset A to store all new sockets for 30 mins. If a packet
>> matches a socket in ipset A, mark it with 1, otherwise with 2. At
>> fireqos, classify packets marked 1 to 'high' and packets marked 2 to
>> 'low'. This will allow your site to be speedy for 30 mins for each
>> client.
>> (this will only work for outgoing traffic - marks do not work on
>> inbound traffic).
>>
>> You can extent the above with 2 ipsets A and B. A stores the new
>> sockets for 30 mins, B stores them for 60 mins. So you can have 3
>> speeds: A = high, B = middle, no mark = low.
>>
>> If you want to try this, there are many ipset examples in the github
>> wiki. Read them before asking for help please...
>>
>>
>> 4. Use codel or fq_codel (this is used by default by fireqos when
>> available). Codel favours new sockets over sockets that are making big
>> downloads. So surfing is snappier.
>>
>>
>> 5. Divide the days to zones, each a few hours long. Put a cron job to
>> apply a different fireqos.conf for every zone. This is not dynamic,
>> but if you know when you need the traffic, it will just do the job.
>>
>>
>> Costa
>>
>>
>> On Thu, Aug 6, 2015 at 11:04 PM, Tommy Butler <networkspeedy at gmail.com>
>> wrote:
>> > I'd like to allow traffic for a given class to burst up to a higher
>> > rate,
>> > but only within a sane timeframe, and /probably/ based on remote IP.
>> >
>> > The reason is because I'd like someone to be able to hit my web pages
>> > and
>> > get content displayed quickly without allowing a constant, sustained
>> > amount
>> > of high throughput to that destination IP.  In cases where sustained
>> > unlimited throughput takes place, my hosting provider sends me a monthly
>> > bill that is $200 higher than normal.  *Surprise*
>> >
>> > The easiest solution was to take my monthly outgoing allotment of 250GB
>> > and
>> > divide it by 30, and see how much bandwidth I had per day, then convert
>> > it
>> > to kbits, and convert that down to seconds.  At this point I know I have
>> > X
>> > amount of kbits to allow per second all month long...
>> >
>> > ...but that is sub-optimal.  I don't have a constant influx of traffic
>> > all
>> > month long.  It's spikey--I get a few hundred unique visits per day,
>> > sometimes more, if only looking at legitimate traffic.  During this
>> > time,
>> > it would be fine to let those visitors see pages at full speed.  But it
>> > would NOT be fine to let visitor(s) continuously pull the maximum amount
>> > of
>> > bandwidth possible for a sustained period of time.
>> >
>> > So I'm kind of half-way to a solution.  I'm shaping my traffic (yay!)
>> >  Everything is under control and I know I won't get any more surprise
>> > bills--call them the "overdraft fees" of the web hosting industry.  But
>> > there is more to be desired: I'd like to allow legitimate visitors (if
>> > there is any way to classify that kind of traffic at all), to view pages
>> > at
>> > maximum throughput while still keeping an eye on my usage so that there
>> > aren't overages.  Bursting seems like a way to do this, but I'm not
>> > clear
>> > on just how to make it work.
>> >
>> > Any feedback is appreciated =)
>> >
>> > Thanks!
>> > _______________________________________________
>> > Firehol-support mailing list
>> > Firehol-support at lists.firehol.org
>> > http://lists.firehol.org/mailman/listinfo/firehol-support
>
>