Discussion:
Curious bufferbloat on DOCSIS connection with many downloads
(too old to reply)
Arie
2018-01-16 19:36:37 UTC
Permalink
Recently I accidentally discovered something quite odd about my home DOCSIS
connection (shaped with cake) while running a flent rrul test.
During the flent rrul test, another machine in my home had started a Steam
game update, this opened somewhere between 8-12 connections and saturated
my downstream.

The interesting part was the resulting rrul graph, showing perfect latency
control during the test, but 100-150ms of bufferbloat at the very start and
end of the rrul graph (when rrul is idle, just pinging).

Instead of a Steam download I started an iperf3 client on a remote machine
that opened 16 connections to a LAN machine. Same behavior as the Steam
download.

At first I thought this was a bug with cake, but no matter how I tuned cake
(limiting my 400/40 connection to 50/20 for example), the behavior was
similar.
I tried fq_codel+htb, and got similar results to cake, high latency before
the rrul test kicked off, great latency while rrul was doing its up and
downloads.

So I swapped out my self-compile LEDE router with an Edgerouter ER-X with
both stock and LEDE firmware, both with fq_codel and cake (on both stock
and LEDE firmware). Same results.
Even without any SQM active, the rrul test still improved the latency of my
connection when it's also downloading from 8+ connections at the same time.

I managed to take the flent rrul test out of the equation by starting a
hping3 "flood" to the hop next of my cable modem.

hping3 -2 -d 0 -s 10080 -k -p 80 -i u100 first-hop-or-ISP-gateway-goes-here
This also fixed the latency while many downloads were running by sending
about 300KB/s of empty UDP packets to some unresponsive port on my ISPs
local gateway.

I'm puzzled by this result, somehow sending 300KB/s of empty packets
upstream drastically improves the latency of my connection when it's
receiving many downloads.

Equipment used:
Cisco EPC3212 cable modem (8x4 EuroDOCSIS 3.0)
Linksys WRT1900ACS router with LEDE
Ubiquiti Edgerouter ER-X with both EdgeOS (stock) and LEDE
Dave Taht
2018-01-16 19:52:45 UTC
Permalink
Power save?
Post by Arie
Recently I accidentally discovered something quite odd about my home DOCSIS
connection (shaped with cake) while running a flent rrul test.
During the flent rrul test, another machine in my home had started a Steam
game update, this opened somewhere between 8-12 connections and saturated my
downstream.
The interesting part was the resulting rrul graph, showing perfect latency
control during the test, but 100-150ms of bufferbloat at the very start and
end of the rrul graph (when rrul is idle, just pinging).
Instead of a Steam download I started an iperf3 client on a remote machine
that opened 16 connections to a LAN machine. Same behavior as the Steam
download.
At first I thought this was a bug with cake, but no matter how I tuned cake
(limiting my 400/40 connection to 50/20 for example), the behavior was
similar.
I tried fq_codel+htb, and got similar results to cake, high latency before
the rrul test kicked off, great latency while rrul was doing its up and
downloads.
So I swapped out my self-compile LEDE router with an Edgerouter ER-X with
both stock and LEDE firmware, both with fq_codel and cake (on both stock and
LEDE firmware). Same results.
Even without any SQM active, the rrul test still improved the latency of my
connection when it's also downloading from 8+ connections at the same time.
I managed to take the flent rrul test out of the equation by starting a
hping3 "flood" to the hop next of my cable modem.
Post by Arie
hping3 -2 -d 0 -s 10080 -k -p 80 -i u100
first-hop-or-ISP-gateway-goes-here
This also fixed the latency while many downloads were running by sending
about 300KB/s of empty UDP packets to some unresponsive port on my ISPs
local gateway.
I'm puzzled by this result, somehow sending 300KB/s of empty packets
upstream drastically improves the latency of my connection when it's
receiving many downloads.
Cisco EPC3212 cable modem (8x4 EuroDOCSIS 3.0)
Linksys WRT1900ACS router with LEDE
Ubiquiti Edgerouter ER-X with both EdgeOS (stock) and LEDE
_______________________________________________
Bloat mailing list
https://lists.bufferbloat.net/listinfo/bloat
--
Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
Arie
2018-01-16 20:48:31 UTC
Permalink
Not powersave of any component in my control. Powersave of the CMTS? I'd
expect high latency for idle pings in that case too.
Post by Dave Taht
Power save?
Post by Arie
Recently I accidentally discovered something quite odd about my home
DOCSIS
Post by Arie
connection (shaped with cake) while running a flent rrul test.
During the flent rrul test, another machine in my home had started a
Steam
Post by Arie
game update, this opened somewhere between 8-12 connections and
saturated my
Post by Arie
downstream.
The interesting part was the resulting rrul graph, showing perfect
latency
Post by Arie
control during the test, but 100-150ms of bufferbloat at the very start
and
Post by Arie
end of the rrul graph (when rrul is idle, just pinging).
Instead of a Steam download I started an iperf3 client on a remote
machine
Post by Arie
that opened 16 connections to a LAN machine. Same behavior as the Steam
download.
At first I thought this was a bug with cake, but no matter how I tuned
cake
Post by Arie
(limiting my 400/40 connection to 50/20 for example), the behavior was
similar.
I tried fq_codel+htb, and got similar results to cake, high latency
before
Post by Arie
the rrul test kicked off, great latency while rrul was doing its up and
downloads.
So I swapped out my self-compile LEDE router with an Edgerouter ER-X with
both stock and LEDE firmware, both with fq_codel and cake (on both stock
and
Post by Arie
LEDE firmware). Same results.
Even without any SQM active, the rrul test still improved the latency of
my
Post by Arie
connection when it's also downloading from 8+ connections at the same
time.
Post by Arie
I managed to take the flent rrul test out of the equation by starting a
hping3 "flood" to the hop next of my cable modem.
Post by Arie
hping3 -2 -d 0 -s 10080 -k -p 80 -i u100
first-hop-or-ISP-gateway-goes-here
This also fixed the latency while many downloads were running by sending
about 300KB/s of empty UDP packets to some unresponsive port on my ISPs
local gateway.
I'm puzzled by this result, somehow sending 300KB/s of empty packets
upstream drastically improves the latency of my connection when it's
receiving many downloads.
Cisco EPC3212 cable modem (8x4 EuroDOCSIS 3.0)
Linksys WRT1900ACS router with LEDE
Ubiquiti Edgerouter ER-X with both EdgeOS (stock) and LEDE
_______________________________________________
Bloat mailing list
https://lists.bufferbloat.net/listinfo/bloat
--
Dave TÀht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
Michael Richardson
2018-01-16 21:34:09 UTC
Permalink
Post by Arie
I'm puzzled by this result, somehow sending 300KB/s of empty packets
upstream drastically improves the latency of my connection when it's
receiving many downloads.
Maybe not exactly power save, but perhaps asking for the upstream bandwidth
gets the CMTS to notice you more often and schedule a dump of it's queue of
traffic for you.

--
] Never tell me the odds! | ipv6 mesh networks [
] Michael Richardson, Sandelman Software Works | network architect [
] ***@sandelman.ca http://www.sandelman.ca/ | ruby on rails [
Benjamin Cronce
2018-01-25 00:15:02 UTC
Permalink
Along what Michael Richardson said, with GPON I have seen where idle
latency was ~1.5ms, but under load latency was about 0.2ms. I wouldn't be
surprised if TDMA scheduling playing a factor. I don't know what kind of
latency SLAs DOCSIS can or typically have. GPON has scheduling latency all
the way down to 0.5ms.
Post by Michael Richardson
Post by Arie
I'm puzzled by this result, somehow sending 300KB/s of empty packets
upstream drastically improves the latency of my connection when it's
receiving many downloads.
Maybe not exactly power save, but perhaps asking for the upstream bandwidth
gets the CMTS to notice you more often and schedule a dump of it's queue of
traffic for you.
--
] Never tell me the odds! | ipv6 mesh networks [
] Michael Richardson, Sandelman Software Works | network architect [
_______________________________________________
Bloat mailing list
https://lists.bufferbloat.net/listinfo/bloat
Loading...