[SCore-users-jp] [SCore-users] ia32/ia64 cluster

[Nick Birkett]

We need some help on configuring mixed clusters.

Here is our test system:

8 dual Xeon ia32 cluster nodes.
1 quad Itanium2   cluster node.

Score 5.4

Network = gigabit

The Xeon cluster has been running Score MPI jobs for the last 2 months.

The Itanium has kernel 2.4.19-1SCORE patched and compiled for Itanium as in
user documents.

The Itanium has been added, and is running SCore multiprocessor kernel (we have run several OpenMP codes and it works fine).


The front end server is called server and is a Xeon 32 bit.

scout works fine:

server nrcb:$ cat hosts
itanium01
server nrcb:$ scout -F hosts
done.
SCOUT: session started.
server nrcb:$ scout uname -a
[itanium01]:
Linux tiger4.streamline 2.4.19-1SCORE_ia64 #2 SMP Mon May 26 12:36:20 PDT 2003 ia64 unknown
server nrcb:$ 

Also mpi compilers work fine on Itanium:

[nrcb ＠ tiger4 mpi]$ mpif77 -compiler intel7 -O3 -w  -o jacobi_mpi_64 jacobi_mpi_param.f
   program JACOBI
   external subroutine OUTPUT
   external function UEXACT
   external function FEXT
   external subroutine INITIALISE
   external subroutine ERROR
   external subroutine ITERATE

2057 Lines Compiled
[nrcb ＠ tiger4 mpi]$  ./jacobi_mpi
<0:0> SCORE: One local node ready.
  Running with nprocs=           1
  Array size nxg,nyg =         4096        4096
  Iteration count    =          128
 cpus=           1 : Iteration =            1   2.24994472335565D+015
 cpus=           1 : Iteration =            2   2.24924498762084D+015
 cpus=           1 : Iteration =            3   2.24870040652968D+015
 cpus=           1 : Iteration =            4   2.24823845276853D+015
 cpus=           1 : Iteration =            5   2.24783004116974D+015
 cpus=           1 : Iteration =            6   2.24745998334097D+015
 cpus=           1 : Iteration =            7   2.24711915332825D+015

So code runs both on Itanium and Xeon (2 binaries).

I have copied 32 bit and 64 bit binaries to: 

/opt/score/bin/bin.ia64-redhat-linux2_4/jacobi_mpi.exe
/opt/score/bin/bin.i386-redhat7-linux2_4/jacobi_mpi.exe

on  itanium 64 bit and server 32 bit

and set link for .wrapper :

[nrcb ＠ tiger4 mpi]$ ls -al jacobi_mpi
lrwxrwxrwx    1 nrcb     streamc        23 May 29 11:46 jacobi_mpi -> /opt/score/bin/.wrapper


(server and Itanium share a common user filesystem via nfs).

I have an entry for scorehosts.db for Itanium:

server mpi:$ grep IA /opt/score/etc/scorehosts.db
itanium01.streamline    HOST_8 network=gigabit,shmem0,shmem1,shmem2,shmem3 group=_scoreall_,IA64,SHMEM smp=4 MSGBSERV
server mpi:$ 



scoreboard and msgbserv services are restarted.

Running a Xeon application on a Xeon node works fine:


server mpi:$  scout -F hosts -e scrun -nodes=2 ./jacobi_mpi
done.
FEP: Unable to connect with SCore-D (comp07)
FEP:WARNING checkpoint option is ignored in single-user mode.
SCore-D 5.4.0 connected.
<0:0> SCORE: 2 nodes (1x2) ready.
  Running with nprocs=           2
  Array size nxg,nyg =         4096        4096
  Iteration count    =          128
  Running with nprocs=           2


Trying to run Itanium application from Xeon server  gives this:

server mpi:$  scout -F hosts -e scrun -nodes=2 ./jacobi_mpi
done.
FEP: Unable to connect with SCore-D (comp07)
FEP:WARNING checkpoint option is ignored in single-user mode.
<0> SCore-D:WARNING Unable to open a network configuration file (2):
network='gigabit', attribute='-config:file'
<0> SCore-D:ERROR No PM device opened.


Itanium has these modules loaded:

pm_shmem               42976   0 (unused)
pm_ethernet_dev       148296   0 (unused)
pm_memory              19216   0 [pm_shmem pm_ethernet_dev]


Do  scoreboard and pm config files need to be running on Itanium as well as Xeon
main server ?


Any help appreciated.

Many thanks,

Nick
_______________________________________________
SCore-users mailing list
SCore-users ＠ pccluster.org
http://www.pccluster.org/mailman/listinfo/score-users