About PM/Infiniband for Topspin InfinBand fabric

The PM/Infiniband TS is implemented using TopSpin OS. The user library of the PM/InfiniBand is released, only. Its library requeires the TopSpin InfiniBand driver, TopSpin OS 2.0.0 build 436, and the PM/InfiniBand TS driver module. You must get its load moduls from InfiniBand vendor to use the PM/InfiniBand TS.

SCore installation on the server host and compute hosts

First of all, please install the SCore system for Ethernet using the EIT tool or by hand.

PM/InfiniBand TS installation on all compute hosts

1. Install the TopSpin OS
   Install the TopSpin OS rpm packages as follow:

           # rpm -i topspin-ib-rh80-2.0.0-436.i686.rpm
           # rpm -i topspin-ib-mod-rhXX-2.4.XX-2.0.0-436.i686.rpm
   

   The rpm package, topspin-ib-mod-XXXX-YYYY-2.0.0-436.i686.rpm, depends on the kernel version and the distribution on compute hosts.
   
   
2. Install the PM/InfiniBand TS driver
   Copy the PM/InfiniBand TS driver as follow:

           # cp pm_ibats.o /lib/modules/`uname -r`/ib/pm_ibats.o
   

3. Setup the IP address for IPoIB
   Create a configration files, /etc/sysconfig/network-scripts/ifcfg-ib0, as follow:

           DEVICE=ib0
   	ONBOOT=yes
           BOOTPROTO=static
   	IPADDR=192.168.0.1
   	NETMASK=255.255.255.0
   	BROADCAST=192.168.0.255
   	NETWORK=192.168.0.0
   

   The network parameter depends the each compute hosts.
   
   
4. reboot the compute host.

SCore Set up on the server host

1. Setup the PM/InfiniBand TS
   Create the PM/InfiniBand TS confiration file as follow:

   	# ( count=0; for hosts in `scorehosts -g pcc`; do \
   	      echo -n "$count	$hosts	"; \
   	      rsh $hosts /sbin/ifconfig ib0 | grep inet | sed 's/.*inet addr:\([0-9.]*\).*/\1/' ; \
   	      count=`expr $count + 1`; \
   	    done \
             ) > /opt/score/etc/pm-infiniband.conf
   

2. Configure the scoreaboard database
   Create the /opt/score/etc/scoreboard.db file as follows: The following entries are an example for PM/Infiniband TS where the server host name is server.pccluster.org and the compute host names are comp0.pccluster.org -- comp3.pccluster.org.

   	/* PM/InfiniBand TS */
   	infiniband	type=ib-ts -config:file=/opt/score/etc/pm-infiniband.conf
   
   	/* PM/Shmem */		
   	shmem0		type=shmem -node=0
   	shmem1		type=shmem -node=1
   
   	#define PCC     MSGBSERV(server.pccluster.org:8766) \
   	                cpugen=pentium-4 speed=3000 smp=2 \
   	                network=infiniband,shmem0,shmem1 group=pcc
   
   	comp0.pccluster.org	PCC
   	comp1.pccluster.org	PCC
   	comp2.pccluster.org	PCC
   	comp3.pccluster.org	PCC
   

3. reboot the server host.

PC Cluster Consotium