[Pc_Support] Re: Nforce kernel info

[Bryan J. Smith]

On Wed, 2005-03-16 at 08:20 -0800, coreyfro at coreyfro.com wrote:
> My first link should be http://www.coreyfro.com/~coreyfro/dreamboard.gif

Interesting, but it's doubtful any mainboard manufacturers can use two
primary chips like that because of legacy signaling issues.  I.e., the
nForce 2200 has the legacy support (similar to nForce4), and the
optional 2050 has additional logic.

> But I actually don't think I'll be needing SLI, I mainly want a
> workstation + VMWare ESX Testing Box.  If I get a second videocard, it
> will be to drive more monitors.

Then you will _definitely_ want the PCI-X channels.  All I/O (other than
video) on a single 0.125GBps shared PCI bus just sucks.  PCI-X will give
you a lot more headroom.

Yes, there are now PCIe x1 NICs which help, but the lack of PCIe storage
controllers -- except maybe the handful of $500+ PCIe x8 Intel IOP332-
based RAID controllers -- really hurts.

> I have heard bad things about 3ware performance,

Really?  I'd be interested to hear otherwise.  3Ware RAID-0, 1 and 0+1
pretty much _slaughter_ anything.  It's the 64-bit ASIC with 0 wait
state SRAM that does the job.

The only thing that used to affect the ASIC+SRAM 3Ware performance was
RAID-5, because the small amount of 1-4MB of SRAM (again, that's 0 wait
state _static_ RAM) would overflow.  But that has been solved with the
3Ware Escalade 9000 series which adds 128MB of SDRAM (optionally up to
1GB).

Now _nothing_ beats it on a server.  On the desktop, the NetCell SR5000
series with its "RAID-XL" fixed 4+1 configuration is ideal.

> and I am a big fan of Linux MD.

Linux MD pushes 2-5x as much I/O over the interconnect.  If your
ATA/SATA is on the shared 0.125GBps PCI bus, then you've just toasted
any I/O you have outside of video.

> I plan to mirror two highspeed SATA drives and call it done.

$125 3Ware Escalade 8006-2 = best performance, minimal headache.

The only time you'll get better performance is when you span RAID-0 over
2 cards on 2 different PCI-X channels.  But then that's where you can
use (2) 3Ware cards on two different PCI-X channels with LVM/LVM2
RAID-0.

> I am not interested in fast streaming or big storage (I already have more
> storage than I need
> http://ilneval.coreyfro.com/~coreyfro/server/full-tn.jpg) but the multi
> head seak built in to Linux MD for mirrors would be perfect for my VM
> requirements.  NCQ would help there too, but if it itsnot in, i'll deal.

3Ware gives you command queuing without the drive requiring it.  There
is absolutely _no_ ATA solution that can thread I/O _better_ than 3Ware
Escalade cards -- because it has an ASIC and SRAM.  It's like a "switch"
for ATA (the highest performance Ethernet switches use ASIC+SRAM as
well) -- hence why 3Ware calls it a "storage switch."

> If you are interested, i have an article I have been trying to write since
> before the dawn of the opteron.  I gave up on it long ago because everyone
> thought I was crazy.  I was wondering if you could sanity check it for me
> just so I can know, for sure, if I really am.
> http://ilneval.coreyfro.com/~coreyfro/cranial-tangles/hammer/
> Summary?  Blade opterons on a "Flat network neighborhood" using HT as a
> highspeed network interface to build a huge aggrogate bandwidth super
> computer.  I spent a lot of time on the images, so that will help you
> visualize what I am talking about.

You have heard of HTX, correct?
It's the new HyperTransport slot.
It's primary use is for Infiniband -- renamed InfiniPath for HTX --
interconnect between systems.

Intel can't break 800MBps with Infiniband even on PCI-X 2.0 slots on
proprietary Itanium2 systems.
AMD HTX Infinipath comes close to 2GBps!



-- 
Bryan J. Smith                                  b.j.smith at ieee.org 
---------------------------------------------------------------- 
Community software is all about choice, choice of technology.
Unfortunately, too many Linux advocates port over the so-called
"choice" from the commercial software world, brand name marketing.
The result is false assumptions, failure to focus on the real
technical similarities, but loyalty to blind vendor alignments.