[Firehol-support] bursting by IP

[Tsaousis, Costa]

Hi Tommy,

tc is a complex thing. I find it difficult to understand (or even use)
certain advanced features.

>From what I have seen so far, I believe there is no way to apply per
client policing or shaping. The most advanced examples I have seen are
kinds of policing per match (like the one I give you below).

Unfortunately, tc documentation is very criptic and sparse.

Anyway, here are a few ideas:


1. I you have many services, you can favour a percentage of HTTP to a
higher class, like this:

interface ... rate 100Mbps ...

   class high commit 50%
      match proto tcp sport 80,443 limit 10Mbps

   class low commit 50%
      match proto tcp sport 80,443

The above will allow the first 10Mbps of HTTP/HTTPS traffic to be
classified in 'high' class and the rest to 'low' class.
Of course this has only meaning when you have many services and you
want to favour a service over another.


2. You can favour small packets over the bigger ones:

interface ... rate 100Mbps ...

   class high commit 50%
      match proto tcp sport 80,443 police "mtu 512 continue"

   class low commit 50%
      match proto tcp sport 80,443

The above will put all HTTP/HTTPS packets that are smaller than 512
bytes to 'high' class, all other to 'low' class.
Again, this has only meaning when you have many services and you want
to favour a service over another.


3. You can use a combination of firehol and fireqos like this: in
firehol use ipset A to store all new sockets for 30 mins. If a packet
matches a socket in ipset A, mark it with 1, otherwise with 2. At
fireqos, classify packets marked 1 to 'high' and packets marked 2 to
'low'. This will allow your site to be speedy for 30 mins for each
client.
(this will only work for outgoing traffic - marks do not work on
inbound traffic).

You can extent the above with 2 ipsets A and B. A stores the new
sockets for 30 mins, B stores them for 60 mins. So you can have 3
speeds: A = high, B = middle, no mark = low.

If you want to try this, there are many ipset examples in the github
wiki. Read them before asking for help please...


4. Use codel or fq_codel (this is used by default by fireqos when
available). Codel favours new sockets over sockets that are making big
downloads. So surfing is snappier.


5. Divide the days to zones, each a few hours long. Put a cron job to
apply a different fireqos.conf for every zone. This is not dynamic,
but if you know when you need the traffic, it will just do the job.


Costa


On Thu, Aug 6, 2015 at 11:04 PM, Tommy Butler <networkspeedy at gmail.com> wrote:
> I'd like to allow traffic for a given class to burst up to a higher rate,
> but only within a sane timeframe, and /probably/ based on remote IP.
>
> The reason is because I'd like someone to be able to hit my web pages and
> get content displayed quickly without allowing a constant, sustained amount
> of high throughput to that destination IP.  In cases where sustained
> unlimited throughput takes place, my hosting provider sends me a monthly
> bill that is $200 higher than normal.  *Surprise*
>
> The easiest solution was to take my monthly outgoing allotment of 250GB and
> divide it by 30, and see how much bandwidth I had per day, then convert it
> to kbits, and convert that down to seconds.  At this point I know I have X
> amount of kbits to allow per second all month long...
>
> ...but that is sub-optimal.  I don't have a constant influx of traffic all
> month long.  It's spikey--I get a few hundred unique visits per day,
> sometimes more, if only looking at legitimate traffic.  During this time,
> it would be fine to let those visitors see pages at full speed.  But it
> would NOT be fine to let visitor(s) continuously pull the maximum amount of
> bandwidth possible for a sustained period of time.
>
> So I'm kind of half-way to a solution.  I'm shaping my traffic (yay!)
>  Everything is under control and I know I won't get any more surprise
> bills--call them the "overdraft fees" of the web hosting industry.  But
> there is more to be desired: I'd like to allow legitimate visitors (if
> there is any way to classify that kind of traffic at all), to view pages at
> maximum throughput while still keeping an eye on my usage so that there
> aren't overages.  Bursting seems like a way to do this, but I'm not clear
> on just how to make it work.
>
> Any feedback is appreciated =)
>
> Thanks!
> _______________________________________________
> Firehol-support mailing list
> Firehol-support at lists.firehol.org
> http://lists.firehol.org/mailman/listinfo/firehol-support