[SCore-users-jp] [SCore-users] Network trunking problems

[Nick Birkett]

We need some help on this.

I am getting the same problems as I got when I tried this before.

Each single network performs extremely well.

But using PMB on 2x gigabit, the Sendrecv is very poor and uneven       
compared to single gigabit. 

Hardware set up: Smicro dual Xeon front end,
                 8x Smicro dual  Xeon 3.06GHz cpus comp nodes,
                    4096 Mbytes
                    2x inboard e1000
                    1x offboard e1000 workstation card in PCI-X slot

                 3 x Nortel baystack Gbit switches (not linked)
                 3 independent networks eth0,eth1,eth2

System network is eth1 - connects Front end and compute nodes.
Message networks  eth0,eth2

eth1,eth2 or the onboard gbit
eth0 is offboard gbit.

Software:  RedHat 9, Score 5.8.1 (compiled from source received end of
Sept 2004). Kernel 2.4.21-2SCORE smp , Intel e1000 driver, v 5.2.39 .

I have also tried it using Score 5.6.1 compiled for RedHat 9
and with Score 5.4.1 on RedHat 7.3,

I can run jobs on all 3 networks and dual network:

scout -F hosts -e scrun -nodes=2x1,network=ethernet0 ./PMB-MPI1 Pingpong
Sendrecv

works fine - see ethernet0.out.

scout -F hosts -e scrun -nodes=2x1,network=ethernet1 ./PMB-MPI1 Pingpong
Sendrecv

works fine 

scout -F hosts -e scrun -nodes=2x1,network=ethernet2 ./PMB-MPI1 Pingpong
Sendrecv

works fine - see ethernet2.out

scout -F hosts -e scrun -nodes=2x1,network=ethernetx2 ./PMB-MPI1
Pingpong Sendrecv

problems !!

Attached files:

Score configuration files:
score-etc-5.8.1.tgz

Output for PMB for ethernet0,ethernet2 and trunked ethernetx2:
ethernet0.out
ethernet2.out
ethernetx2.out

Info files from cat /proc/pm :

eth0.info
eth2.info

I have tried the following parameters: maxnsend 16,32
                                       backoff 1024,2048 4096

I have also tried with Jumbo frames set on eth0, eth2 network
(MTU=9000). The attached results are with Jumbo frames off.

I have also tested with interrupt reaping off.

This was much worse than Intreap 1 .


Any ideas ?

Best wishes,

Nick


-- 

 
----


PM Ethernet 0330
Version		:	"$Id: pm_ethernet_dev.c,v 1.5 2004/04/22 04:30:17 kameyama Exp $"
Device  	:	eth0
Irq     	:	0
MTU     	:	1468 Bytes
Maxnodes	:	512
MaxContext	:	16
Intreap 	:	on
Checksum 	:	on
Dev error	:	3
Header err	:	0
Header cksum err:	0
Data cksum err	:	0
Underrun err	:	0
----


PM Ethernet 0330
Version		:	"$Id: pm_ethernet_dev.c,v 1.5 2004/04/22 04:30:17 kameyama Exp $"
Device  	:	eth2
Irq     	:	0
MTU     	:	1468 Bytes
Maxnodes	:	512
MaxContext	:	16
Intreap 	:	on
Checksum 	:	on
Dev error	:	1
Header err	:	0
Header cksum err:	0
Data cksum err	:	0
Underrun err	:	0
-- 
Nick Birkett     
Streamline Computing Ltd


-------------- next part --------------
テキスト形式以外の添付ファイルを保管しました...
ファイル名: score-etc-5.8.1.tgz
型:         application/x-gzip
サイズ:     3799 バイト
説明:       無し
URL:        <http://new1.pccluster.org/pipermail/score-users-jp/attachments/20041012/efd44d77/attachment.tgz>

-------------- next part --------------
テキスト形式以外の添付ファイルを保管しました...
ファイル名: ethernet0.out
型:         application/octet-stream
サイズ:     5310 バイト
説明:       無し
URL:        <http://new1.pccluster.org/pipermail/score-users-jp/attachments/20041012/efd44d77/attachment.obj>

-------------- next part --------------
テキスト形式以外の添付ファイルを保管しました...
ファイル名: ethernet2.out
型:         application/octet-stream
サイズ:     5310 バイト
説明:       無し
URL:        <http://new1.pccluster.org/pipermail/score-users-jp/attachments/20041012/efd44d77/attachment-0001.obj>

-------------- next part --------------
テキスト形式以外の添付ファイルを保管しました...
ファイル名: ethernetx2.out
型:         application/octet-stream
サイズ:     5310 バイト
説明:       無し
URL:        <http://new1.pccluster.org/pipermail/score-users-jp/attachments/20041012/efd44d77/attachment-0002.obj>