[SCore-users] Bug found!?
Amik St-Cyr CFD Lab
amik at cfdlab.mcgill.ca
Mon Sep 23 22:40:35 JST 2002
Hi all,
First of all our tech team found the
bug in our setup. Thank you for the help of mr.
Toyohisa. We had mixed up two cables...
We then have tried to benckmark the machine
with the LINPACK suite. The behaviour of the linpack
was very strange depending on the parameters given.
The for a specified setup I was able to crank up the
size of the matrix but then SCORE gave me the
following message:
| amik at stokes 18:22:56 Linux_ATHLON_CBLAS> scrun
-nodes=128x2,mpi_max_eager_myrinet=2000000 ./xhpl
SCore-D 5.0.1 connected.
<0:0> SCORE: 256 nodes (128x2) ready.
============================================================================
HPLinpack 1.0 -- High-Performance Linpack benchmark -- September 27,
2000
Written by A. Petitet and R. Clint Whaley, Innovative Computing Labs.,
UTK
============================================================================
An explanation of the input/output parameters follows:
T/V : Wall time / encoded variant.
N : The order of the coefficient matrix A.
NB : The partitioning blocking factor.
P : The number of process rows.
Q : The number of process columns.
Time : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.
The following parameter values will be used:
N : 200000
NB : 100
P : 16
Q : 16
PFACT : Crout
NBMIN : 1
NDIV : 16
RFACT : Right
BCAST : 2ringM
DEPTH : 1
SWAP : Mix (threshold = 16)
L1 : transposed form
U : transposed form
EQUIL : yes
ALIGN : 8 double precision words
----------------------------------------------------------------------------
- The matrix A is randomly generated for each test.
- The following scaled residual checks will be computed:
1) ||Ax-b||_oo / ( eps * ||A||_1 * N )
2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )
3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )
- The relative machine precision (eps) is taken to be
2.220446e-16
- Computational tests pass if scaled residuals are less than
16.0
<13:0> SCORE:WARNING MPICH/SCore: pmReceive(pmc=0x85bc7a0) failed,
errno=5
<13:0> SCORE:PANIC MPICH/SCore: critical error on message transfer
<13:0> Trying to attach GDB (DISPLAY=localhost:13.0): PANIC
SCORE: Program aborted.
| amik at stokes 23:14:53 Linux_ATHLON_CBLAS>
Can I prevent this from happening?
Here are the details:
| amik at stokes 09:40:11 Linux_ATHLON_CBLAS> cat HPL.dat
HPLinpack benchmark input file
Innovative Computing Laboratory, University of Tennessee
HPL256.out output file name (if any)
6 device out (6=stdout,7=stderr,file)
1 # of problems sizes (N)
200000 Ns
1 # of NBs
100 NBs
1 # of process grids (P x Q)
16 Ps
16 Qs
16.0 threshold
1 # of panel fact
1 PFACTs (0=left, 1=Crout, 2=Right)
1 # of recursive stopping criterium
1 NBMINs (>= 1)
1 # of panels in recursion
16 NDIVs
1 # of recursive panel fact.
2 RFACTs (0=left, 1=Crout, 2=Right)
1 # of broadcast
3 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
1 # of lookahead depth
1 DEPTHs (>=0)
2 SWAP (0=bin-exch,1=long,2=mix)
16 swapping threshold
0 L1 in (0=transposed,1=no-transposed) form
0 U in (0=transposed,1=no-transposed) form
1 Equilibration (0=no,1=yes)
8 memory alignment in double (> 0)
Make file using Portland group compiler and linking with
the mpi in SCORE.
| amik at stokes 09:40:47 hpl> cat Make.Linux_ATHLON_CBLAS
#
# -- High Performance Computing Linpack Benchmark (HPL)
# HPL - 1.0 - September 27, 2000
# Antoine P. Petitet
# University of Tennessee, Knoxville
# Innovative Computing Laboratories
# (C) Copyright 2000 All Rights Reserved
#
# -- Copyright notice and Licensing terms:
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions, and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# 3. All advertising materials mentioning features or use of this
# software must display the following acknowledgement:
# This product includes software developed at the University of
# Tennessee, Knoxville, Innovative Computing Laboratories.
#
# 4. The name of the University, the name of the Laboratory, or the
# names of its contributors may not be used to endorse or promote
# products derived from this software without specific written
# permission.
#
# -- Disclaimer:
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE UNIVERSITY
# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# ######################################################################
#
# ----------------------------------------------------------------------
# - shell --------------------------------------------------------------
# ----------------------------------------------------------------------
#
SHELL = /bin/sh
#
CD = cd
CP = cp
LN_S = ln -s
MKDIR = mkdir
RM = /bin/rm -f
TOUCH = touch
#
# ----------------------------------------------------------------------
# - Platform identifier ------------------------------------------------
# ----------------------------------------------------------------------
#
ARCH = Linux_ATHLON_CBLAS
#
# ----------------------------------------------------------------------
# - HPL Directory Structure / HPL library ------------------------------
# ----------------------------------------------------------------------
#
TOPdir = $(HOME)/hpl
INCdir = $(TOPdir)/include
BINdir = $(TOPdir)/bin/$(ARCH)
LIBdir = $(TOPdir)/lib/$(ARCH)
#
HPLlib = $(LIBdir)/libhpl.a
#
# ----------------------------------------------------------------------
# - Compilers / linkers - Optimization flags ---------------------------
# ----------------------------------------------------------------------
#
CC = pgcc
NOOPT =
CCFLAGS = -O2 -Munroll=c:4 -Mvect=prefetch -Mvect=cachesize:32768
LINKER = pgcc
LINKFLAGS = $(CCFLAGS)
#
ARCHIVER = ar
ARFLAGS = r
RANLIB = echo
#
# ----------------------------------------------------------------------
# - MPI directories - library ------------------------------------------
# ----------------------------------------------------------------------
# MPinc tells the C compiler where to find the Message Passing library
# header files, MPlib is defined to be the name of the library to be
# used. The variable MPdir is only used for defining MPinc and MPlib.
#
MPdir = /opt/score/mpi/mpich-1.2.0/i386-redhat7-linux2_4
MPinc = -I$(MPdir)/include
MPlib = -Bstatic -L$(MPdir)/lib
-L/opt/score/lib/i386-redhat7-linux2_4 -lmpich -lscoreusr -lpm -lult
-lscorecommon -lpthread -lscboard -lscwrap -L/usr/pgi/linux86/lib
#
# ----------------------------------------------------------------------
# - F77 / C interface --------------------------------------------------
# ----------------------------------------------------------------------
# You can skip this section if and only if you are not planning to use
# a BLAS library featuring a Fortran 77 interface. Otherwise, it is
# necessary to fill out the F2CDEFS variable with the appropriate
# options. **One and only one** option should be chosen in **each** of
# the 3 following categories:
#
# 1) name space (How C calls a Fortran 77 routine)
#
# -DAdd_ : all lower case and a suffixed underscore (Suns,
# Intel, ...),
# -DNoChange : all lower case (IBM RS6000),
# -DUpCase : all upper case (Cray),
# -Df77IsF2C : the FORTRAN compiler in use is f2c.
#
# 2) C and Fortran 77 integer mapping
#
# -DF77_INTEGER=int : Fortran 77 INTEGER is a C int,
# -DF77_INTEGER=long : Fortran 77 INTEGER is a C long,
# -DF77_INTEGER=short : Fortran 77 INTEGER is a C short.
#
# 3) Fortran 77 string handling
#
# -DStringSunStyle : The string address is passed at the string loca-
# tion on the stack, and the string length is then
# passed as an F77_INTEGER after all explicit
# stack arguments,
# -DStringStructPtr : The address of a structure is passed by a
# Fortran 77 string, and the structure is of the
# form: struct {char *cp; F77_INTEGER len;},
# -DStringStructVal : A structure is passed by value for each Fortran
# 77 string, and the structure is of the form:
# struct {char *cp; F77_INTEGER len;},
# -DCrayStyle : Special option for Cray machines, which uses
# Cray fcd (fortran character descriptor) for
# interoperation.
#
F2CDEFS =
#
# ----------------------------------------------------------------------
# - Linear Algebra library (BLAS or VSIPL) -----------------------------
# ----------------------------------------------------------------------
# LAinc tells the C compiler where to find the Linear Algebra library
# header files, LAlib is defined to be the name of the library to be
# used. The variable LAdir is only used for defining LAinc and LAlib.
#
LAdir = /home/amik/Linux_ATHLON
LAinc = -I$(LAdir)/include
LAlib = $(LAdir)/lib/libcblas.a $(LAdir)/lib/libatlas.a
#
# ----------------------------------------------------------------------
# - HPL includes / libraries / specifics -------------------------------
# ----------------------------------------------------------------------
#
HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc)
HPL_LIBS = $(HPLlib) $(LAlib) $(MPlib)
#
# - Compile time options -----------------------------------------------
#
# -DHPL_COPY_L force the copy of the panel L before bcast;
# -DHPL_CALL_CBLAS call the cblas interface;
# -DHPL_CALL_VSIPL call the vsip library;
# -DHPL_DETAILED_TIMING enable detailed timers;
#
# By default HPL will:
# *) not copy L before broadcast,
# *) call the Fortran 77 BLAS interface
# *) not display detailed timing information.
#
HPL_OPTS = -DHPL_CALL_CBLAS
#
# ----------------------------------------------------------------------
#
HPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
#
# ----------------------------------------------------------------------
--
_____________________________________________________
Dr. A. St-Cyr
Research Associate, CFD Lab
Department of Mechanical Engineering
McGill University
688 Sherbrooke Street West, 7th floor
Montreal, Qc, Canada H3A 2S6
Tel: +1 (514) 398-1710, Admin. Fax : 2203
amik at cfdlab.mcgill.ca
_____________________________________________________
More information about the SCore-users
mailing list