[Bloat] Flow offload's impact on bufferbloat

[Rosen Penev]

OpenWrt has backported Netfilter's flow offload functionality from
kernel 4.17 to 4.14. I've been noticing higher speeds as well as
higher latency with it enabled. Anyone have any insight? My test
results are here:

On: http://www.dslreports.com/speedtest/37007587
Off: http://www.dslreports.com/speedtest/37007494

Note the latencies.

[Jonathan Morton]

Post by Rosen Penev
OpenWrt has backported Netfilter's flow offload functionality from
kernel 4.17 to 4.14. I've been noticing higher speeds as well as
higher latency with it enabled. Anyone have any insight? My test
On: http://www.dslreports.com/speedtest/37007587
Off: http://www.dslreports.com/speedtest/37007494
Note the latencies.

The latencies you're seeing there have nothing to do with the offload engine. You are bufferbloated in both cases, and apparently without effective mitigation in either direction.

- Jonathan Morton

[Dave Taht]

what device?

what sort of bql stats do you see?

In both of these cases you should just enable sqm set to 100/5 and it
shouldn't matter.

Post by Rosen Penev
OpenWrt has backported Netfilter's flow offload functionality from
kernel 4.17 to 4.14. I've been noticing higher speeds as well as
higher latency with it enabled. Anyone have any insight? My test
On: http://www.dslreports.com/speedtest/37007587
Off: http://www.dslreports.com/speedtest/37007494
Note the latencies.
_______________________________________________
Bloat mailing list
https://lists.bufferbloat.net/listinfo/bloat

[Rosen Penev]

Post by Dave Taht
what device?
what sort of bql stats do you see?
In both of these cases you should just enable sqm set to 100/5 and it
shouldn't matter.

Note that this is software offload, not hardware.

Device is a Netgear R7800. The ethernet on it is totally broken TBH.
But I've also seen this on a Turris Omnia.

My question is not really how to fix it. I already know that. I just
got the feeling that bypassing parts of the linux network stack would
result in less buffering.

Post by Dave Taht

Post by Rosen Penev
OpenWrt has backported Netfilter's flow offload functionality from
kernel 4.17 to 4.14. I've been noticing higher speeds as well as
higher latency with it enabled. Anyone have any insight? My test
On: http://www.dslreports.com/speedtest/37007587
Off: http://www.dslreports.com/speedtest/37007494
Note the latencies.
_______________________________________________
Bloat mailing list
https://lists.bufferbloat.net/listinfo/bloat

--
Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: [phone]

[Jonathan Morton]

I just got the feeling that bypassing parts of the linux network stack would
result in less buffering.

That buffering is not happening in the linux network stack. It's happening in the hardware, both in your modem (upload) and in the ISP's equipment (download) because those are the parts which meet the bottleneck link.

When making measurements, be sure of what you're measuring.

- Jonathan Morton

[Mikael Abrahamsson]

Post by Rosen Penev
My question is not really how to fix it. I already know that. I just
got the feeling that bypassing parts of the linux network stack would
result in less buffering.

On the OpenWrt configuration page for the "software flow offload":

"Experimental feature. Not fully compatible with QoS/SQM."

I don't know exactly what it does, it reduces amount of CPU cycles needed
to forward packets in an already established flow it seems, but I'd
imagine that it might very well bypass some of the scheduling code which
could explain what you're seeing. So you might get faster forwarding but
less AQM.

So if your device isn't fast enough to keep up with your total Internet
access speed, then this might be a good thing. If your device is faster
than what's needed, then you'd better spend the cycles on getting good AQM
instead of freeing up more CPU that isn't used for anything anyway.