ECN blocking router found

[Dave Taht]

In my travels this month I have been testing ECN enablement at homes and
hotels everywhere I go.

Until today, I was able to have the following settings for ECN on my
laptop everywhere I've been.

net.ipv4.tcp_ecn=1
#net.ipv6.tcp_ecn=0
net.ipv4.tcp_sack=1
net.ipv4.tcp_dsack=1

However, I got to visit Brian Clapper [1] (friend/co-author of gnugol)
tonight, and discovered that his fairly recently purchased router, a:

Etherfast Cable/DSL router Model BEFSR41
Firmware version 2.0.0.4

flat out refused to pass ECN enabled connection attempts (returning an
ICMP unreachable message)

He'd not noticed the problem because ubuntu 10.4 (at least, he also runs
bsd) has tcp_ecn=2, which so far as I know "tries" a ECN enabled connect
then falls back to not using it.

I'm bummed that such a recent router doesn't pass ECN, and will look
into the problem further in the morning.

So I think we must use tcp_ecn = 1 to TEST to make sure ECN is being
passed, and tcp_ecn=2 as the default recommendation.

Perhaps we can synthesize TCP streams to more directly test ECN
capability in the future somehow as part of our testing tools. Are there
any tools that synthesize TCP/ip we could use as a starting point?

[1] http://brizzled.clapper.org/

[Brian Clapper]

Post by Dave Taht
In my travels this month I have been testing ECN enablement at homes and
hotels everywhere I go.
Until today, I was able to have the following settings for ECN on my laptop
everywhere I've been.
net.ipv4.tcp_ecn=1
#net.ipv6.tcp_ecn=0
net.ipv4.tcp_sack=1
net.ipv4.tcp_dsack=1
However, I got to visit Brian Clapper [1] (friend/co-author of gnugol)
Etherfast Cable/DSL router Model BEFSR41
Firmware version 2.0.0.4

NOTE: Per the support web site for this device, firmware revision 2.00.4 is
the latest.

Post by Dave Taht
flat out refused to pass ECN enabled connection attempts (returning an ICMP
unreachable message)
He'd not noticed the problem because ubuntu 10.4 (at least, he also runs bsd)
has tcp_ecn=2, which so far as I know "tries" a ECN enabled connect then falls
back to not using it.
I'm bummed that such a recent router doesn't pass ECN, and will look into the
problem further in the morning.
So I think we must use tcp_ecn = 1 to TEST to make sure ECN is being passed,
and tcp_ecn=2 as the default recommendation.
Perhaps we can synthesize TCP streams to more directly test ECN capability in
the future somehow as part of our testing tools. Are there any tools that
synthesize TCP/ip we could use as a starting point?
[1] http://brizzled.clapper.org/

[Rui Paulo]

In my travels this month I have been testing ECN enablement at homes and hotels everywhere I go.
Until today, I was able to have the following settings for ECN on my laptop everywhere I've been.
net.ipv4.tcp_ecn=1
#net.ipv6.tcp_ecn=0
net.ipv4.tcp_sack=1
net.ipv4.tcp_dsack=1
Etherfast Cable/DSL router Model BEFSR41
Firmware version 2.0.0.4
flat out refused to pass ECN enabled connection attempts (returning an ICMP unreachable message)
He'd not noticed the problem because ubuntu 10.4 (at least, he also runs bsd) has tcp_ecn=2, which so far as I know "tries" a ECN enabled connect then falls back to not using it.
I'm bummed that such a recent router doesn't pass ECN, and will look into the problem further in the morning.
So I think we must use tcp_ecn = 1 to TEST to make sure ECN is being passed, and tcp_ecn=2 as the default recommendation.
Perhaps we can synthesize TCP streams to more directly test ECN capability in the future somehow as part of our testing tools. Are there any tools that synthesize TCP/ip we could use as a starting point?

Are you looking for something like this?
http://www.icir.org/tbit/index.html#ECN

Regards,
--
Rui Paulo

[Richard Scheffenegger]

Hi Rui,

I was also to recommend TBIT (or some variant thereof; perhaps raw_sockets
and firewall settings can also be used for a native windows tool...)

Also, thanks again for addressing the ECN issue in BSD!

Best regards,
Richard

----- Original Message -----

Post by Rui Paulo

Post by Dave Taht
Perhaps we can synthesize TCP streams to more directly test ECN
capability in the future somehow as part of our testing tools. Are there
any tools that synthesize TCP/ip we could use as a starting point?

Are you looking for something like this?
http://www.icir.org/tbit/index.html#ECN

[Henrique de Moraes Holschuh]

Post by Dave Taht
He'd not noticed the problem because ubuntu 10.4 (at least, he also
runs bsd) has tcp_ecn=2, which so far as I know "tries" a ECN enabled
connect then falls back to not using it.

tcp_ecn=2 is "server mode ECN". It will enable ECN if, and ONLY if, the
other side requests it during the initial TCP handshake.

AFAICT, it effectively disables ECN on all outgoing connections and
enables it on incoming connections should the other host request ECN
(which it won't when it is also running in tcp_ecn=2 mode).

It is also the kernel default, and AFAIK, the default on every 2010+
Linux distro stable releases.

If you want to use ECN, you have to set tcp_ecn=1 in your Linux box when
you're the one originating TCP connections.

[Eric Dumazet]

FYI

IPV6 ECN support is buggy on current linux kernels

Fix is on the way, problem spotted by Steinar H. Gunderson

https://bugzilla.kernel.org/show_bug.cgi?id=34322

[Dave Taht]

I am curious as to what the correct behavior here should be for encapsulated
(6in4, 6to4, teredo) packets, and if this functionality was also borked. I
was under the impression that for encapsulated packets the tos field was
copied from the encapsulated packet to the ipv4 header.

I will make a note of this in the uberwrt/cerowrt/bismark efforts going on
and backport the upcoming patch to openwrt.

Post by Eric Dumazet
FYI
IPV6 ECN support is buggy on current linux kernels
Fix is on the way, problem spotted by Steinar H. Gunderson
https://bugzilla.kernel.org/show_bug.cgi?id=34322
_______________________________________________
Bloat mailing list
https://lists.bufferbloat.net/listinfo/bloat

[Jonathan Morton]

I am curious as to what the correct behavior here should be for encapsulated (6in4, 6to4, teredo) packets, and if this functionality was also borked. I was under the impression that for encapsulated packets the tos field was copied from the encapsulated packet to the ipv4 header.

Intuitively, these protocols are at the same level as IP in the stack, so they should preserve ECN information as much as possible. Copying the TOS field should be sufficient...

- Jonathan

[Dave Taht]

Post by Dave Taht
I am curious as to what the correct behavior here should be for

encapsulated (6in4, 6to4, teredo) packets, and if this functionality was
also borked. I was under the impression that for encapsulated packets the
tos field was copied from the encapsulated packet to the ipv4 header.
Intuitively, these protocols are at the same level as IP in the stack, so
they should preserve ECN information as much as possible. Copying the TOS
field should be sufficient...

My concern here is that a AQM-aware (ECN) qdisc such as SFB on the external
interface will not recognize a flow for what it is, when encapsulated...

[Dave Taht]

Final patch found, merged into openwrt (tested on 2.6.38, 2.6.37), and
incorporated into the bismark and iscwrt builds. Nice catch.

I hope the next round of sfb testing goes much better.

Post by Dave Taht

Post by Dave Taht
I am curious as to what the correct behavior here should be for

encapsulated (6in4, 6to4, teredo) packets, and if this functionality was
also borked. I was under the impression that for encapsulated packets the
tos field was copied from the encapsulated packet to the ipv4 header.
Intuitively, these protocols are at the same level as IP in the stack, so
they should preserve ECN information as much as possible. Copying the TOS
field should be sufficient...

My concern here is that a AQM-aware (ECN) qdisc such as SFB on the external
interface will not recognize a flow for what it is, when encapsulated...

[Lars Eggert]

Post by Jonathan Morton

I am curious as to what the correct behavior here should be for encapsulated (6in4, 6to4, teredo) packets, and if this functionality was also borked. I was under the impression that for encapsulated packets the tos field was copied from the encapsulated packet to the ipv4 header.

Intuitively, these protocols are at the same level as IP in the stack, so they should preserve ECN information as much as possible. Copying the TOS field should be sufficient...

Please see http://tools.ietf.org/html/rfc6040. No need to guess.

Lars

[Juliusz Chroboczek]

Post by Dave Taht
I am curious as to what the correct behavior here should be for encapsulated
(6in4, 6to4, teredo) packets,

RFC 3168 sections 9.1 and 9.2 (which I've read), but I fear it may have
been updated by RFC 6040 (which I haven't).

-- Juliusz

[Matthew Ford]

Post by Eric Dumazet
FYI
IPV6 ECN support is buggy on current linux kernels
Fix is on the way, problem spotted by Steinar H. Gunderson
https://bugzilla.kernel.org/show_bug.cgi?id=34322

FWIW, Mac OSX has similar behaviour and I filed the bug with Apple in July of last year.

Summary: When ECN is enabled (by setting net.inet.tcp.ecn_initiate_out = 1 and net.inet.tcp.ecn_negotiate_in = 1), the ECN capable-transport (ECT) bit is not set in the IPv6 Traffic Class field when communicating with ECN-enabled hosts.

Steps to Reproduce:

1. Set net.inet.tcp.ecn_initiate_out=1 and net.inet.tcp.ecn_negotiate_in = 1
2. Connect over IPv6 to another similarly ECN-enabled host and transfer some data over TCP (e.g. an HTTP transaction, or FTP transfer).
3. Monitor the connection with your favourite protocol analyser.
4. Observe that the ECN-Echo bits are correctly set in the initial TCP handshake.
5. Observe that the ECT bit is not being set in the IPv6 Traffic Class field in subsequent packets.

Expected Results:

RFC3168 defines ECN for IP. In IPv4 and IPv6, ECN codepoints are specified as bits 6 and 7 of the TOS Byte and Traffic Class Octet respectively. The ECN Capable Transport (ECT) bit should be set after TCP has successfully concluded ECN initialisation.

Actual Results:

ECT is set for IPv4 communications. ECT is not being set for IPv6 communications.

Mat