I should also point out that the kinds of routing latency numbers in
those blog entries was on very high end intel hardware. It would be
good to re-run those sort of tests on the armada and others for
1,10,100, 1000 routes. Clever complicated algorithms have a tendency
to bloat icache and cost more than they are worth, fairly often, on
hardware that typically has 32k i/d caches, and a small L2.
BQL's XMIT_MORE is one example - while on the surface it looked like a
win, it cost too much on the ar71xx to use. Similarly I worry about
the new rx batching code (
https://lwn.net/SubscriberLink/763056/f9a20ec24b8d29dd/ ) which looks
*GREAT* - on *intel* - although I *think* it will be a win everywhere
this time. I tend to think a smaller napi value would help, and
sometimes I think about revisiting napi itself.
(and I'm perfectly willing to wait til openwrt does the rest of the
port for mips to 4.19 before fiddling with it... or longer. I could
use a dayjob)
Still, it's been the rx side of linux that has been increasingly
worrisome of late, and anything that can be done there for any chip
seems like a goodness.
on the mvneta front... I've worked on that driver... oh... if I could
get a shot at ripping out all the bloat in it and see what happened...
On the marvell front... yes, they tend to produce hardware that runs
too hot. I too rather like the chipset, and it's become my default hw
for most things in the midrange.
Lastly... there are still billions of slower ISP links left in the
world to fix, with hardware that now costs well under
40 bucks. The edgerouter X is 50 bucks (sans wifi) and good to
~180mbps for inbound shaping presently. Can we get those edge
Post by Dave Taht
One of the things not readily evident in trying to scale up, is the
cost of even the most basic routing table lookup. A lot of good work
Lookup time for even the smallest number of routes is absolutely
miserable for IPv6 -
I think one of the biggest driving factors of the whole TSO/GRO thing
is due to trying to get smaller packets through this phase of the
kernel, and not that they are so much more efficient at the card
itself. Given the kerfuffle over here (
https://github.com/systemd/systemd/issues/9725 ) I'd actually like to
come up with a way to move the linux application socket buffers to the
post-lookup side of the routing table. We spend a lot of extra time
bloating up superpackets just so they are cheaper to route.
TCAMs are expensive as hell, but the addition of even a small one,
readily accessible to userspace or from the kernel, might help in the
general case. I've actually oft wished to be able to offload these
sort of lookups into higher level algorithms and languages like
python, as a general purpose facility. Hey, if we can have giant GPUs,
why can't our cpus have tcams?
programmable TCAM support got enabled in a recent (mellonox?) product.
Can't find the link at the moment TCAMs of course, is where big fat
dedicated routers and switches shine, over linux - and even arp table
lookups are expensive in linux, though I'm not sure if anyone has
CEO, TekLibre, LLC