BiTel

Возможно ли достижение 10G пропускной способности LINUX/BSD роутера?

> > This was using 2 dual-port 10-GigE NICs in the first two PCIe 2.0 slots.
> > We are using an Intel i7 965 quad-core 3.2 GHz Nehalem processor
> > (overclocked to 3.4 GHz) and 2000 MHz DDR3 memory. Adding an additional
> > dual-port 10-GigE NIC on the Nvidia N200 chip does only marginally
> > better, as it appears we are basically CPU limited at this point for
> > this test (the sum of the TX and RX CPU utilization for each pair of
> > 10-GigE interfaces is about 93%).
>
> Hey guys, those are really nice numbers. Since TCP splicing appeared in the
> kernel (once we got it fixed), I achieved 10 Gbps of HTTP proxying using
> haproxy with very low CPU usage (about 20% of a Core2Duo 2.66 GHz).

Nice, but I think we have a bug with the measured CPU usage. Eric
Dumazet did a fix, but also pointed out that in a later mail, at I seem
like it not fixed completely yet...

> Before buying the machines, I had been wandering around with the NICs
> donated by Myricom in order to try to find a machine capable of supporting
> this. My conclusion was that a lot of machines had difficulties getting
> above 3.5, 4.7 and 6.5 Gbps of output traffic (those 3 numbers were always
> the same, depending on the chipsets). There clearly was a bandwidth
> limitation imposed by the chipset.
>
> So I waited for the X38 and AM780FX chipsets to become available and
> bought 3 machines (1 C2D, 1 AMD X2, 1 AMD X4). Those ones have no problem
> with 10 Gbps of forwarded traffic (20 Gbps of total bus bandwidth), even
> with 1500 bytes frames, but I don't know how high they can go, maybe
> they will saturate slightly above.

My experience is also that the AMDs can easily do 10Gbit/s forwarding,
but doing bidirectional they suffer...

> Unfortunately, I only have 5 NICs in 3 machines and no switch (and CX4
> is hard to find these days), so I'm probably stuck at 10 Gbps max.

We are a fiber company, so I'm using our spare 10G optics, but I'm
limited by our supply of SFP+ currently.

I'll be getting two 6 port 10GbE NIC using PCIe2 x16 82599, in august,
so it will be interesting how high we can go!

> Interestingly, I had the impression that forwarding data with TCP
> splicing costs less CPU than IP forwarding, because the NICs can do
> LRO.
>
> Also, I know a french service provider who uses haproxy on Core i7
> machines and who has already reached 5 Gbps of sustained traffic
> with recent intel dual-port NICs (though I'm not sure exactly which
> ones). This is with very little CPU usage too, less than 2-3% user
> and 15% system+softirq. On previous machines (quad core xeons), it
> was impossible to go beyond 3 Gbps, it looked like the chipset was
> the limitating factor too (though I don't precisely remember which
> one it was).
>
> I really blamed the NICs because this guys machine was about 4 times
> more powerful than mine, but apparently it was just a chipset issue.
>
> I also happen to have a customer who recently received a few Sun NXGE,
> mounted in Sun x2100-m2 using an nvidia chipset which I tested OK at
> 10 Gbps with my myri10GE NICs. I'll try to see if I can run some tests
> there, as Davem once said those NICs are really good too.

The Sun NIU NIC has to use several hardware queues to achieve 10GbE.
Currently using these as generators, and thats one of my limiting
factors.