[Bloat] vyatta in AT&T 5G gear

[Dave Taht]

Vyos (the open source fork of vyatta) was one of the first to add
fq_codel support... I wonder....

http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/

[Mikael Abrahamsson]

Post by Dave Taht
Vyos (the open source fork of vyatta) was one of the first to add
fq_codel support... I wonder....
http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/

Isn't Vyos just running the Linux kernel for forwarding? So they received
fq_codel for free when the Linux kernel got support for it? They just had
to make it configurable?

[Stefan Alfredsson]

Post by Mikael Abrahamsson

Post by Dave Taht
Vyos (the open source fork of vyatta) was one of the first to add
fq_codel support... I wonder....
http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/

Isn't Vyos just running the Linux kernel for forwarding? So they
received fq_codel for free when the Linux kernel got support for it?
They just had to make it configurable?

Yes, according to this blog post,
http://www.five-ten-sg.com/mapper/blog/Bufferbloat%20solved%20with%20Vyos

"Now that Vyos "helium" is available with a Linux 3.13 kernel, the
fq_codel queueing discipline can be used to solve many bufferbloat
issues. The nightly "lithium" builds contain my patches that allow
fq_codel to be used via the native Vyos configuration system."

Anyway it's nice to see the Vyatta heritage living on in it's various
forms (the AT&T "production hardened" Vyatta, to the Ubiquity EdgeMax
and some UniFi devices, to the VyOS open version and now the future
plans with dNOS -> DANOS.

/Stefan

[Stephen Hemminger]

On Tue, 16 Oct 2018 11:59:18 +0200

Post by Stefan Alfredsson

Post by Mikael Abrahamsson

Post by Dave Taht
Vyos (the open source fork of vyatta) was one of the first to add
fq_codel support... I wonder....
http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/

Isn't Vyos just running the Linux kernel for forwarding? So they
received fq_codel for free when the Linux kernel got support for it?
They just had to make it configurable?

Yes, according to this blog post,
http://www.five-ten-sg.com/mapper/blog/Bufferbloat%20solved%20with%20Vyos
"Now that Vyos "helium" is available with a Linux 3.13 kernel, the
fq_codel queueing discipline can be used to solve many bufferbloat
issues. The nightly "lithium" builds contain my patches that allow
fq_codel to be used via the native Vyos configuration system."
Anyway it's nice to see the Vyatta heritage living on in it's various
forms (the AT&T "production hardened" Vyatta, to the Ubiquity EdgeMax
and some UniFi devices, to the VyOS open version and now the future
plans with dNOS -> DANOS.
/Stefan

There are two basic components to network OS, the control plane and the
data plane. VyOs has the old V1 which is filesystem based control plane
and kernel dataplane. DaNoS has yang/netconf database based control plane
(in Go) and DPDK (or switch offload?) based dataplane. Ubiquity redid
the control plane as well, and uses their own hardware for dataplane.

So more of "my grandfather's ax"...

[Dave Taht]

On Tue, Oct 16, 2018 at 8:06 AM Stephen Hemminger

Post by Stephen Hemminger
On Tue, 16 Oct 2018 11:59:18 +0200

Post by Stefan Alfredsson

Post by Mikael Abrahamsson

Post by Dave Taht
Vyos (the open source fork of vyatta) was one of the first to add
fq_codel support... I wonder....
http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/

Isn't Vyos just running the Linux kernel for forwarding? So they
received fq_codel for free when the Linux kernel got support for it?
They just had to make it configurable?

Yes, according to this blog post,
http://www.five-ten-sg.com/mapper/blog/Bufferbloat%20solved%20with%20Vyos
"Now that Vyos "helium" is available with a Linux 3.13 kernel, the
fq_codel queueing discipline can be used to solve many bufferbloat
issues. The nightly "lithium" builds contain my patches that allow
fq_codel to be used via the native Vyos configuration system."
Anyway it's nice to see the Vyatta heritage living on in it's various
forms (the AT&T "production hardened" Vyatta, to the Ubiquity EdgeMax
and some UniFi devices, to the VyOS open version and now the future
plans with dNOS -> DANOS.
/Stefan

There are two basic components to network OS, the control plane and the
data plane. VyOs has the old V1 which is filesystem based control plane
and kernel dataplane. DaNoS has yang/netconf database based control plane
(in Go) and DPDK (or switch offload?) based dataplane. Ubiquity redid
the control plane as well, and uses their own hardware for dataplane.
So more of "my grandfather's ax"...

so in other words we should have done a dpdk version long ago. And even then,
this white box brags of the "deep buffering" in the switch chip.

... and I have an initial report of 2 seconds of buffering on one of
the first 5G devices.

Sigh.

And what was so wrong with the "everything as a file" model??

double sigh.

Post by Stephen Hemminger
_______________________________________________
Bloat mailing list
https://lists.bufferbloat.net/listinfo/bloat

[Jan Ceuleers]

Post by Dave Taht
And what was so wrong with the "everything as a file" model??

On a single box - not a lot.

On a 5G network, which might consist of 10^3 - 10^5 boxes you need
aggregation and abstraction layers.

[Dave Taht]

Post by Jan Ceuleers

Post by Dave Taht
And what was so wrong with the "everything as a file" model??

On a single box - not a lot.
On a 5G network, which might consist of 10^3 - 10^5 boxes you need
aggregation and abstraction layers.

I was mostly just venting. Still... I'd hoped that the next generation
fios gear would get bufferbloat right... and with all the 5G boasting
about the 1ms mac, that they'd get queuing delay below 20ms also.

I was also hoping the DOCSIS 3.1 line cards would get it right...

It has been a disappointing day.

[Stephen Hemminger]

On Tue, 16 Oct 2018 17:46:44 -0700

Post by Dave Taht

Post by Jan Ceuleers

Post by Dave Taht
And what was so wrong with the "everything as a file" model??

On a single box - not a lot.
On a 5G network, which might consist of 10^3 - 10^5 boxes you need
aggregation and abstraction layers.

I was mostly just venting. Still... I'd hoped that the next generation
fios gear would get bufferbloat right... and with all the 5G boasting
about the 1ms mac, that they'd get queuing delay below 20ms also.
I was also hoping the DOCSIS 3.1 line cards would get it right...
It has been a disappointing day.

At scale, you need a usable programming model; on box CLI or files don't work well.
Netconf/yang was built for that.

[Stephen Hemminger]

On Tue, 16 Oct 2018 08:14:36 -0700

Post by Dave Taht
On Tue, Oct 16, 2018 at 8:06 AM Stephen Hemminger

Post by Stephen Hemminger
On Tue, 16 Oct 2018 11:59:18 +0200

Post by Stefan Alfredsson

Post by Mikael Abrahamsson

Post by Dave Taht
Vyos (the open source fork of vyatta) was one of the first to add
fq_codel support... I wonder....
http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/

Isn't Vyos just running the Linux kernel for forwarding? So they
received fq_codel for free when the Linux kernel got support for it?
They just had to make it configurable?

Yes, according to this blog post,
http://www.five-ten-sg.com/mapper/blog/Bufferbloat%20solved%20with%20Vyos
"Now that Vyos "helium" is available with a Linux 3.13 kernel, the
fq_codel queueing discipline can be used to solve many bufferbloat
issues. The nightly "lithium" builds contain my patches that allow
fq_codel to be used via the native Vyos configuration system."
Anyway it's nice to see the Vyatta heritage living on in it's various
forms (the AT&T "production hardened" Vyatta, to the Ubiquity EdgeMax
and some UniFi devices, to the VyOS open version and now the future
plans with dNOS -> DANOS.
/Stefan

There are two basic components to network OS, the control plane and the
data plane. VyOs has the old V1 which is filesystem based control plane
and kernel dataplane. DaNoS has yang/netconf database based control plane
(in Go) and DPDK (or switch offload?) based dataplane. Ubiquity redid
the control plane as well, and uses their own hardware for dataplane.
So more of "my grandfather's ax"...

so in other words we should have done a dpdk version long ago. And even then,
this white box brags of the "deep buffering" in the switch chip.
... and I have an initial report of 2 seconds of buffering on one of
the first 5G devices.
Sigh.
And what was so wrong with the "everything as a file" model??

Two things were an issue. At scale, the filesystem was a bottleneck and it
would have been harder to implement netconf/yang model as required by
the big boy market.

Filesystem works as toy. (see plan 9)

[Jonas Mårtensson]

Hi Stephen,

DaNoS has yang/netconf database based control plane

Post by Stephen Hemminger
(in Go) and DPDK (or switch offload?) based dataplane.

Do you have more information about DANOS and using Go for the control
plane? The AT&T design uses a Broadcom Qumran-AX switching chip for the
dataplane so no DPDK.

/Jonas

[Stephen Hemminger]

On Sun, 21 Oct 2018 11:36:43 +0200

Post by Jonas MÃ¥rtensson
Hi Stephen,
DaNoS has yang/netconf database based control plane

Post by Stephen Hemminger
(in Go) and DPDK (or switch offload?) based dataplane.

Do you have more information about DANOS and using Go for the control
plane? The AT&T design uses a Broadcom Qumran-AX switching chip for the
dataplane so no DPDK.
/Jonas

Linux Foundation announced Danos, but no code is yet available.

[Dave Taht]

On Mon, Oct 22, 2018 at 9:42 AM Stephen Hemminger

Post by Stephen Hemminger
On Sun, 21 Oct 2018 11:36:43 +0200

Post by Jonas MÃ¥rtensson
Hi Stephen,
DaNoS has yang/netconf database based control plane

Post by Stephen Hemminger
(in Go) and DPDK (or switch offload?) based dataplane.

Do you have more information about DANOS and using Go for the control
plane? The AT&T design uses a Broadcom Qumran-AX switching chip for the
dataplane so no DPDK.
/Jonas

Linux Foundation announced Danos, but no code is yet available.

Not even which license seems to be available.

Still... I guess this is going to be "a thing", and we should probably
get involved.

I'd like to see P4 evolve a bit more "our way".

Post by Stephen Hemminger
_______________________________________________
Bloat mailing list
https://lists.bufferbloat.net/listinfo/bloat

[Toke Høiland-Jørgensen]

Post by Dave Taht
On Mon, Oct 22, 2018 at 9:42 AM Stephen Hemminger

Post by Stephen Hemminger
On Sun, 21 Oct 2018 11:36:43 +0200

Post by Jonas MÃ¥rtensson
Hi Stephen,
DaNoS has yang/netconf database based control plane

Post by Stephen Hemminger
(in Go) and DPDK (or switch offload?) based dataplane.

Do you have more information about DANOS and using Go for the control
plane? The AT&T design uses a Broadcom Qumran-AX switching chip for the
dataplane so no DPDK.
/Jonas

Linux Foundation announced Danos, but no code is yet available.

Not even which license seems to be available.
Still... I guess this is going to be "a thing", and we should probably
get involved.
I'd like to see P4 evolve a bit more "our way".

There's a fairly new traffic management working group in the P4 org.
They're trying to define an extension to the P4 language to describe a
programmable interface to packet scheduling and traffic shaping.

You have to be a member of the P4 consortium to take part (most of the
lists on http://lists.p4.org/mailman/listinfo are members only). It's
free for individuals, though; but you do have to sign a CLA. See
https://p4.org/join/

-Toke

[Jonas Mårtensson]

Post by Stephen Hemminger
Linux Foundation announced Danos, but no code is yet available.

Yeah I know, but you seemed to know it is written in Go, which I can't find
any information about, so I thought maybe you had more information.

/Jonas

[Stephen Hemminger]

On Tue, 23 Oct 2018 09:38:10 +0200

Post by Jonas MÃ¥rtensson

Post by Stephen Hemminger
Linux Foundation announced Danos, but no code is yet available.

Yeah I know, but you seemed to know it is written in Go, which I can't find
any information about, so I thought maybe you had more information.
/Jonas

I used to work at Brocade. Didn't work directly on the control handler but did
write lots of yang models for it. Yes it was in Go at the time.