Discussion:
Van Jacobson's slides on timing wheels at netdevconf
(too old to reply)
Dave Taht
2018-07-20 14:09:31 UTC
Permalink
https://www.files.netdevconf.org/d/46def75c2ef345809bbe/files/?p=/Evolving%20from%20AFAP%20%E2%80%93%20Teaching%20NICs%20about%20time.pdf

Talking about replacing queues with timing wheels.

I will refrain from commenting other than noting A) I like it. We
essentially have limits in the OS on packet scheduling that make it
harder and harder to have sub 2ms, much less sub 10usec, queues with
the existing qdiscs and pacing systems B) that NIC support seems
needed. I can think of a lot of things I'd like to have in a NIC
(which certainly include default timestamping on rx and multiple kinds
of X-tuple hash) - but hadn't thought about replacing queues entirely!

I haven't read up on carousel yet either.

Look forward to seeing the video when it comes out.
--
Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
Luca Muscariello
2018-07-20 14:47:00 UTC
Permalink
and C) you can implement any packet scheduler using a timing wheel using
virtual times.
Post by Dave Taht
https://www.files.netdevconf.org/d/46def75c2ef345809bbe/files/?p=/Evolving%20from%20AFAP%20%E2%80%93%20Teaching%20NICs%20about%20time.pdf
Talking about replacing queues with timing wheels.
I will refrain from commenting other than noting A) I like it. We
essentially have limits in the OS on packet scheduling that make it
harder and harder to have sub 2ms, much less sub 10usec, queues with
the existing qdiscs and pacing systems B) that NIC support seems
needed. I can think of a lot of things I'd like to have in a NIC
(which certainly include default timestamping on rx and multiple kinds
of X-tuple hash) - but hadn't thought about replacing queues entirely!
I haven't read up on carousel yet either.
Look forward to seeing the video when it comes out.
--
Dave TÀht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
_______________________________________________
Make-wifi-fast mailing list
https://lists.bufferbloat.net/listinfo/make-wifi-fast
Jonathan Morton
2018-07-21 14:38:22 UTC
Permalink
and C) you can implement any packet scheduler using a timing wheel using virtual times.
To do this perfectly, you have to assume that no packets will arrive within your scheduling horizon which should be scheduled within that horizon.

An example of such a situation would be sparse flows in DRR++, which is a key part of fq_codel and Cake. So to implement DRR++ using timing wheels, you have to choose your scheduling horizon carefully so as to minimise the delay to sparse packets.

Given that task scheduling latency in Linux is on the order of 1ms, a horizon of this magnitude could be acceptable. But dedicated hardware could do better still.

- Jonathan Morton
Mikael Abrahamsson
2018-07-23 12:52:24 UTC
Permalink
Post by Jonathan Morton
An example of such a situation would be sparse flows in DRR++, which is
a key part of fq_codel and Cake. So to implement DRR++ using timing
wheels, you have to choose your scheduling horizon carefully so as to
minimise the delay to sparse packets.
At the spring IETF, there was talk from IEEE person about using ethernet
pause frames to get senders to stop talking for a while. My understanding
was that this was on microsecond scale or even nanosecond time scales.

One of the mentions in the presentation was on slide 10 about
"fat-buffered router". In the data center, these are kind of going away,
because on-die memory is small and rates are high. A 64x100GE forwarding
asic might have 16MB of buffer, which is very little buffer for the kind
of bit rates we're talking here.

1h44m in (proposed IEEE
802.1Qcz work) is the one I am thinking of.

Wonder how this would interact with the timing wheel proposed by Van
Jacobson?
--
Mikael Abrahamsson email: ***@swm.pp.se
Dave Taht
2018-07-23 15:36:46 UTC
Permalink
Post by Mikael Abrahamsson
Post by Jonathan Morton
An example of such a situation would be sparse flows in DRR++, which is
a key part of fq_codel and Cake. So to implement DRR++ using timing
wheels, you have to choose your scheduling horizon carefully so as to
minimise the delay to sparse packets.
At the spring IETF, there was talk from IEEE person about using ethernet
pause frames to get senders to stop talking for a while. My understanding
was that this was on microsecond scale or even nanosecond time scales.
One of the mentions in the presentation was on slide 10 about
"fat-buffered router". In the data center, these are kind of going away,
because on-die memory is small and rates are high. A 64x100GE forwarding
asic might have 16MB of buffer, which is very little buffer for the kind
of bit rates we're talking here.
http://youtu.be/sJMvAqEQCBE 1h44m in (proposed IEEE
802.1Qcz work) is the one I am thinking of.
OK, I watched a bit of that. Lot of handwaving.

Prior to that I fast forwarded past BB's l4s thing and looked a little
at PSS, which essentially looked like it was doing FQ between dscp
marks....

I think the ietf should just rename itself to being DCTF (the data
center task force) and let some other org arise to take care of the
internet.
Post by Mikael Abrahamsson
Wonder how this would interact with the timing wheel proposed by Van Jacobson.
FIFO queues remain cheap.

I keep hitting reload hoping the video to the talk was up.
Post by Mikael Abrahamsson
Jacobson?
--
_______________________________________________
Bloat mailing list
https://lists.bufferbloat.net/listinfo/bloat
--
Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
d***@deepplum.com
2018-07-23 17:26:50 UTC
Permalink
Post by Dave Taht
I think the ietf should just rename itself to being DCTF (the data
center task force) and let some other org arise to take care of the
internet.
 
:-)

Internetworking is not really what DC's are about. But it's an indication of where investors want to spend their money.

The real "internetworking" challenges are happening, as always, where ownership/control domains make central planning, design and control difficult or impossible.

That is, at the "edges" not the "core". DC's are just another kind of "core".

Wireless (especially ad hoc connectivity wireless) and "IoT" (if done with the sensibility of internetworking at its core) are where internetworking matters.

That's why congestion control and performance are such open-ended problems in internetworking - you need a higher level of abstraction above the "hard standards" like Ethernet, etc. because you can't solve those problems "locally" in the hard-standardized-devices. (that's an "end-to-end argument" - you can't fully solve many internetworking problems in a subnetwork or router).

It's why I don't go to IETF. The attendees have their heads screwed on with crossed threads.
Matthias Tafelmeier
2018-07-21 13:19:16 UTC
Permalink
Post by Dave Taht
B) that NIC support seems
needed. I can think of a lot of things I'd like to have in a NIC
(which certainly include default timestamping on rx and multiple kinds
of X-tuple hash) - but hadn't thought about replacing queues entirely!
Still fond of the notion of RX|TX sk buffers wandering into the NIC as
well. Taking the plunge of asking: couldn't "all queuing|wheeling"
necessary be consolidated to the sk buffs in NIC then? TCP buffer
windowing should be fairly agnostic to that, shouldn't it?
--
Besten Gruß

Matthias Tafelmeier
Loading...