[SCore-users] Bug found!?
Amik St-Cyr CFD Lab
amik at cfdlab.mcgill.ca
Thu Sep 26 23:17:35 JST 2002
On Thu, 2002-09-26 at 01:51, Shinji Sumimoto wrote:
> Hi.
>
> From: Amik St-Cyr CFD Lab <amik at cfdlab.mcgill.ca>
> Subject: Re: [SCore-users] Bug found!?
> Date: 25 Sep 2002 09:22:17 -0400
> Message-ID: <1032960138.3499.62.camel at stan.cfdlab.mcgill.ca>
>
> amik> On Tue, 2002-09-24 at 20:37, Shinji Sumimoto wrote:
> amik> > Hi.
> amik> >
> amik> > This situation is very quorious.
> amik> >
> amik> > The message says some messages are lost without CRC error. If this
> amik> > situation is true, something wrong with SCore MPI or PM/Myrinet.
> amik> >
> amik>
> amik> For 80% of 384Gigs the maximal matrix size is about N=203000.
> amik> I have tried N=200000 with a memory fill per node of 2.4 Gigs/3 Gigs.
> amik>
> amik> > Your Ns is 120000. How much does your cluster have?
> amik> > No swapping out memory?
> amik> >
> amik> > Could you use /opt/score/share/lanai/lanaiM2k-safe.mcp instead of
> amik> > /opt/score/share/lanai/lanaiM2k.mcp(default).
> amik> >
> amik>
> amik> Yes for N=120000 It takes forever with lanaiM2K-safe. Once on a
> amik> good day the system managed to do N=100000 with a 347GF score.
> amik> What we dont have is stability of the overall system.
> amik>
> amik> I have tried N=120000 last night and the system had not finished
> amik> this morning.
> amik>
> amik> I will retry N=100000 with lanaiM2k-safe (it took 33min when we were
> amik> using lanaiM2k).
>
> Same situation...
>
> Have you restarted scoreboard?
> # /etc/rc.d/init.d/scoreboard reload
>
> I would like to separate whether this problem depends on MPICH or PM/Myrinet.
> So, could you test HPL on following patterns?
>
> 1) set BCASTs to 0 and 1
> 1 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
>
---------------------
rg
---------------------
| amik at stokes 09:01:14 Linux_ATHLON_CBLAS> scrun ./xhpl
SCore-D 5.0.1 connected.
<0:0> SCORE: 256 nodes (128x2) ready.
============================================================================
HPLinpack 1.0 -- High-Performance Linpack benchmark -- September 27,
2000
Written by A. Petitet and R. Clint Whaley, Innovative Computing Labs.,
UTK
============================================================================
An explanation of the input/output parameters follows:
T/V : Wall time / encoded variant.
N : The order of the coefficient matrix A.
NB : The partitioning blocking factor.
P : The number of process rows.
Q : The number of process columns.
Time : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.
The following parameter values will be used:
N : 10000
NB : 100
P : 16
Q : 16
PFACT : Crout
NBMIN : 1
NDIV : 16
RFACT : Right
BCAST : 1ring
DEPTH : 1
SWAP : Mix (threshold = 16)
L1 : transposed form
U : transposed form
EQUIL : yes
ALIGN : 8 double precision words
----------------------------------------------------------------------------
- The matrix A is randomly generated for each test.
- The following scaled residual checks will be computed:
1) ||Ax-b||_oo / ( eps * ||A||_1 * N )
2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )
3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )
- The relative machine precision (eps) is taken to be
2.220446e-16
- Computational tests pass if scaled residuals are less than
16.0
============================================================================
T/V N NB P Q Time
Gflops
----------------------------------------------------------------------------
W10R16C1 10000 100 16 16 8.10
8.234e+01
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0285008 ......
PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0067441 ......
PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0015075 ......
PASSED
============================================================================
Finished 1 tests with the following results:
1 tests completed and passed residual checks,
0 tests completed and failed residual checks,
0 tests skipped because of illegal input values.
----------------------------------------------------------------------------
End of Tests.
============================================================================
----------------------
rM
----------------------
| amik at stokes 09:02:12 Linux_ATHLON_CBLAS> scrun ./xhpl
SCore-D 5.0.1 connected.
<0:0> SCORE: 256 nodes (128x2) ready.
============================================================================
HPLinpack 1.0 -- High-Performance Linpack benchmark -- September 27,
2000
Written by A. Petitet and R. Clint Whaley, Innovative Computing Labs.,
UTK
============================================================================
An explanation of the input/output parameters follows:
T/V : Wall time / encoded variant.
N : The order of the coefficient matrix A.
NB : The partitioning blocking factor.
P : The number of process rows.
Q : The number of process columns.
Time : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.
The following parameter values will be used:
N : 10000
NB : 100
P : 16
Q : 16
PFACT : Crout
NBMIN : 1
NDIV : 16
RFACT : Right
BCAST : 1ringM
DEPTH : 1
SWAP : Mix (threshold = 16)
L1 : transposed form
U : transposed form
EQUIL : yes
ALIGN : 8 double precision words
----------------------------------------------------------------------------
- The matrix A is randomly generated for each test.
- The following scaled residual checks will be computed:
1) ||Ax-b||_oo / ( eps * ||A||_1 * N )
2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )
3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )
- The relative machine precision (eps) is taken to be
2.220446e-16
- Computational tests pass if scaled residuals are less than
16.0
============================================================================
T/V N NB P Q Time
Gflops
----------------------------------------------------------------------------
W11R16C1 10000 100 16 16 9.24
7.220e+01
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0256112 ......
PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0060604 ......
PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0013546 ......
PASSED
============================================================================
Finished 1 tests with the following results:
1 tests completed and passed residual checks,
0 tests completed and failed residual checks,
0 tests skipped because of illegal input values.
----------------------------------------------------------------------------
End of Tests.
---------------------
2rM
---------------------
| amik at stokes 09:04:14 Linux_ATHLON_CBLAS> scrun ./xhpl
SCore-D 5.0.1 connected.
<0:0> SCORE: 256 nodes (128x2) ready.
============================================================================
HPLinpack 1.0 -- High-Performance Linpack benchmark -- September 27,
2000
Written by A. Petitet and R. Clint Whaley, Innovative Computing Labs.,
UTK
============================================================================
An explanation of the input/output parameters follows:
T/V : Wall time / encoded variant.
N : The order of the coefficient matrix A.
NB : The partitioning blocking factor.
P : The number of process rows.
Q : The number of process columns.
Time : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.
The following parameter values will be used:
N : 10000
NB : 100
P : 16
Q : 16
PFACT : Crout
NBMIN : 1
NDIV : 16
RFACT : Right
BCAST : 2ringM
DEPTH : 1
SWAP : Mix (threshold = 16)
L1 : transposed form
U : transposed form
EQUIL : yes
ALIGN : 8 double precision words
----------------------------------------------------------------------------
- The matrix A is randomly generated for each test.
- The following scaled residual checks will be computed:
1) ||Ax-b||_oo / ( eps * ||A||_1 * N )
2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )
3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )
- The relative machine precision (eps) is taken to be
2.220446e-16
- Computational tests pass if scaled residuals are less than
16.0
============================================================================
T/V N NB P Q Time
Gflops
----------------------------------------------------------------------------
W13R16C1 10000 100 16 16 9.60
6.947e+01
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0287260 ......
PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0067974 ......
PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0015194 ......
PASSED
============================================================================
Finished 1 tests with the following results:
1 tests completed and passed residual checks,
0 tests completed and failed residual checks,
0 tests skipped because of illegal input values.
----------------------------------------------------------------------------
End of Tests.
============================================================================
> 2) with mpi_zerocopy without mpi_eager option
>
> scrun -nodes=128x2,mpi_zerocopy=on ./xhpl
>
----------------------
rM
----------------------
| amik at stokes 09:05:08 Linux_ATHLON_CBLAS> scrun
-nodes=128x2,mpi_zerocopy=on ./xhpl
SCore-D 5.0.1 connected.
<0:0> SCORE: 256 nodes (128x2) ready.
============================================================================
HPLinpack 1.0 -- High-Performance Linpack benchmark -- September 27,
2000
Written by A. Petitet and R. Clint Whaley, Innovative Computing Labs.,
UTK
============================================================================
An explanation of the input/output parameters follows:
T/V : Wall time / encoded variant.
N : The order of the coefficient matrix A.
NB : The partitioning blocking factor.
P : The number of process rows.
Q : The number of process columns.
Time : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.
The following parameter values will be used:
N : 10000
NB : 100
P : 16
Q : 16
PFACT : Crout
NBMIN : 1
NDIV : 16
RFACT : Right
BCAST : 1ring
DEPTH : 1
SWAP : Mix (threshold = 16)
L1 : transposed form
U : transposed form
EQUIL : yes
ALIGN : 8 double precision words
----------------------------------------------------------------------------
- The matrix A is randomly generated for each test.
- The following scaled residual checks will be computed:
1) ||Ax-b||_oo / ( eps * ||A||_1 * N )
2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )
3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )
- The relative machine precision (eps) is taken to be
2.220446e-16
- Computational tests pass if scaled residuals are less than
16.0
[117] Receive Error: myriIsReadDone: Bad address(14)(1000e): 000011d0
00000041 0000c468 00001718 00007080 00001180 000052a8 08978000
[117] Receive Error: myriIsReadDone: Bad address(14)(1000e): 000011d0
00000041 0000c468 00007080 00007080 000011d0 00005968 0000c000
comp_is_read_done(0x85bc7a0): pmIsReadDone(0x85bec48): Bad address(14)
<117:1> SCORE:WARNING MPICH/SCore: pmIsReadDone(pmc=0x85bc7a0) failed,
errno=14
<117:1> SCORE:PANIC MPICH/SCore: critical error on message transfer
<117:1> Trying to attach GDB (DISPLAY=localhost:10.0): PANIC
comp_is_read_done(0x85bc7a0): pmIsReadDone(0x85bec48): Bad address(14)
<117:0> SCORE:WARNING MPICH/SCore: pmIsReadDone(pmc=0x85bc7a0) failed,
errno=14
<117:0> SCORE:PANIC MPICH/SCore: critical error on message transfer
<117:0> Trying to attach GDB (DISPLAY=localhost:10.0): PANIC
SCORE: Program aborted.
----------------------
rM
----------------------
| amik at stokes 09:06:31 Linux_ATHLON_CBLAS> scrun
-nodes=128x2,mpi_zerocopy=on ./xhpl
SCore-D 5.0.1 connected.
<0:0> SCORE: 256 nodes (128x2) ready.
============================================================================
HPLinpack 1.0 -- High-Performance Linpack benchmark -- September 27,
2000
Written by A. Petitet and R. Clint Whaley, Innovative Computing Labs.,
UTK
============================================================================
An explanation of the input/output parameters follows:
T/V : Wall time / encoded variant.
N : The order of the coefficient matrix A.
NB : The partitioning blocking factor.
P : The number of process rows.
Q : The number of process columns.
Time : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.
The following parameter values will be used:
N : 10000
NB : 100
P : 16
Q : 16
PFACT : Crout
NBMIN : 1
NDIV : 16
RFACT : Right
BCAST : 1ringM
DEPTH : 1
SWAP : Mix (threshold = 16)
L1 : transposed form
U : transposed form
EQUIL : yes
ALIGN : 8 double precision words
----------------------------------------------------------------------------
- The matrix A is randomly generated for each test.
- The following scaled residual checks will be computed:
1) ||Ax-b||_oo / ( eps * ||A||_1 * N )
2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )
3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )
- The relative machine precision (eps) is taken to be
2.220446e-16
- Computational tests pass if scaled residuals are less than
16.0
============================================================================
T/V N NB P Q Time
Gflops
----------------------------------------------------------------------------
W11R16C1 10000 100 16 16 10.18
6.550e+01
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0300470 ......
PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0071100 ......
PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0015892 ......
PASSED
============================================================================
Finished 1 tests with the following results:
1 tests completed and passed residual checks,
0 tests completed and failed residual checks,
0 tests skipped because of illegal input values.
----------------------------------------------------------------------------
End of Tests.
============================================================================
>
> When these tests do not help with analysis of the situation,
> I will send special Myrinet control programs.
>
> By the way, you are using Athlon processor, so
>
> What chipset does the MB have? Early 760MP-X chipset has very quorious
> situation on zero-copy communication.
>
Yes Indeed, we have those.
> If you have a time, I recommend bustest (PCI DMA bandwidth performance
> measurement) and communication test of PM message and zero-copy
> communication .
>
We have time, we want this project to work. We will do, I will ask our
technician to do the testing.
PM/Myrinet Test Procedure:
1)
myrinet
| amik at stokes 09:24:26 sbin> ./rpminit cn1 myrinet
| amik at stokes 09:27:13 sbin> ./rpmtest cn1 myrinet -dest 0 -ping
8 5.8716e-06
myrinet2k
| amik at stokes 10:14:19 sbin> ./rpminit cn1 myrinet2k
| amik at stokes 10:14:24 sbin> ./rpmtest cn1 myrinet2k -dest 0 -ping
8 7.49937e-06
(see 3 (slow node 61))
| amik at stokes 10:15:46 sbin> ./rpminit cn61 myrinet2k
| amik at stokes 10:15:53 sbin> ./rpmtest cn61 myrinet2k -dest 60 -ping
8 1.03231e-05
2)
Myrinet:
Shell A:
| amik at stokes 09:29:47 sbin> ./rpmtest cn2 myrinet -reply
Shell B:
| amik at stokes 09:30:57 sbin> ./rpmtest cn1 myrinet -dest 1 -ping
8 1.00363e-05
Myrinet2k:
Shell A:
./rpmtest cn2 myrinet2k -reply
Shell B:
| amik at stokes 10:13:14 sbin> ./rpmtest cn1 myrinet2k -dest 1 -ping
8 1.20059e-05
(see 3 (slow node 61))
Shell A:
| amik at stokes 10:18:09 sbin> ./rpmtest cn61 myrinet2k -reply
Shell B:
| amik at stokes 10:18:27 sbin> ./rpmtest cn1 myrinet2k -dest 60 -ping
8 1.4752e-05
3)
Shell A:
| amik at stokes 09:56:39 sbin> ./rpmtest cn3 myrinet2k -vreply
Shell B:
| amik at stokes 10:18:33 sbin> cat ~/bin/do_all
#!/bin/bash
liste=`(cat /etc/hosts | grep cn | awk '{print $3}')`
for i in $liste;do
echo $i
/opt/score/sbin/rpmtest $i myrinet2k -dest $1 -vwrite
done
| amik at stokes 09:53:47 sbin> do_all 2
cn1
8 1.21528e+06
cn2
8 1.21555e+06
cn3
myriMapLANai(0, 0xbffff794, 0): pm_open("/dev/pmmyri/0", O_RDWR, 0):
Device or resource busy(16)
myriOpenDevice("/var/scored/scoreboard/stokes.0000Z000C-WE",
"/var/scored/scoreboard/stokes.0000Z0006MGS", 0xbffff7d8):
myriMapLANai(0, 0xbffff794): Device or resource busy(16)
myri_open_device(0, 0xbffff9c0, 0x80adc80):
myriOpenDevice("/var/scored/scoreboard/stokes.0000Z000C-WE",
"/var/scored/scoreboard/stokes.0000Z0006MGS", 0xbffff7d8): Device or
resource busy(16)
pmOpenDevice: Device or resource busy(16)
cn4
8 1.22084e+06
cn5
8 1.22031e+06
cn6
8 1.21936e+06
cn7
8 1.21954e+06
cn8
8 1.21908e+06
cn9
8 1.17938e+06
cn10
8 1.18443e+06
cn11
8 1.18103e+06
cn12
8 1.17963e+06
cn13
8 1.18098e+06
cn14
8 1.20557e+06
cn15
8 1.17934e+06
cn16
8 1.17928e+06
cn17
8 1.17943e+06
cn18
8 1.17936e+06
cn19
8 1.17979e+06
cn20
8 1.18086e+06
cn21
8 1.17987e+06
cn22
8 1.17724e+06
cn23
8 1.18379e+06
cn24
8 1.18124e+06
cn25
8 1.17841e+06
cn26
8 1.19677e+06
cn27
8 1.17885e+06
cn28
8 1.18305e+06
cn29
8 1.18064e+06
cn30
8 1.17977e+06
cn31
8 1.17963e+06
cn32
8 1.17994e+06
cn33
8 1.17912e+06
cn34
8 1.19678e+06
cn35
8 1.20869e+06
cn36
8 1.18085e+06
cn37
8 1.18299e+06
cn38
8 1.17977e+06
cn39
8 1.18088e+06
cn40
8 1.18038e+06
cn41
8 1.17923e+06
cn42
8 1.17943e+06
cn43
8 1.18067e+06
cn44
8 1.18168e+06
cn45
8 1.18139e+06
cn46
8 1.18099e+06
cn47
8 1.18203e+06
cn48
8 1.18082e+06
cn49
8 1.17946e+06
cn50
8 1.1803e+06
cn51
8 1.1804e+06
cn52
8 1.18191e+06
cn53
8 1.18401e+06
cn54
8 1.18331e+06
cn55
8 1.18326e+06
cn56
8 1.18173e+06
cn57
8 1.18018e+06
cn58
8 1.18091e+06
cn59
8 1.18022e+06
cn60
8 1.1818e+06
cn61
8 755405
cn62
8 1.18278e+06
cn63
8 1.20865e+06
cn64
8 1.18166e+06
cn65
8 1.18006e+06
cn66
8 1.17926e+06
cn67
8 1.17955e+06
cn68
8 1.20513e+06
cn69
8 1.17968e+06
cn70
8 1.17986e+06
cn71
8 1.18094e+06
cn72
8 1.18195e+06
cn73
8 1.19813e+06
cn74
8 1.18053e+06
cn75
8 1.17967e+06
cn76
8 1.17978e+06
cn77
8 1.17905e+06
cn78
8 1.18084e+06
cn79
8 1.18611e+06
cn80
8 755487
cn81
8 1.17952e+06
cn82
8 1.17863e+06
cn83
8 1.17988e+06
cn84
8 1.20254e+06
cn85
8 1.20848e+06
cn86
8 1.18039e+06
cn87
8 1.17919e+06
cn88
8 1.17975e+06
cn89
8 755277
cn90
8 1.1792e+06
cn91
8 1.18081e+06
cn92
8 1.17974e+06
cn93
8 1.17997e+06
cn94
8 1.17914e+06
cn95
8 1.18022e+06
cn96
8 1.17897e+06
cn97
8 1.18451e+06
cn98
8 1.18447e+06
cn99
8 1.20977e+06
cn100
8 1.18888e+06
cn101
8 1.18841e+06
cn102
8 1.18935e+06
cn103
8 1.18774e+06
cn104
8 1.18739e+06
cn105
8 1.20806e+06
cn106
8 1.18413e+06
cn107
8 1.18674e+06
cn108
8 1.1868e+06
cn109
8 1.21215e+06
cn110
8 755543
cn111
8 1.18941e+06
cn112
8 1.18727e+06
cn113
8 1.18388e+06
cn114
8 1.18409e+06
cn115
8 1.18515e+06
cn116
8 1.18756e+06
cn117
8 1.18788e+06
cn118
8 759960
cn119
8 1.2122e+06
cn120
8 1.18718e+06
cn121
8 1.18379e+06
cn122
8 1.18409e+06
cn123
8 1.18601e+06
cn124
8 1.18686e+06
cn125
8 1.18783e+06
cn126
8 1.18639e+06
cn127
8 1.18707e+06
cn128
8 1.18881e+06
Nodes 61,80,89,110,118 are slow.
I have repeated the test with ./rpmtest cn2 myrinet2k -vreply and
the same nodes are slow : 61,80,89,110,118 (!)
4)
| amik at stokes 09:55:02 deploy> ./scstest -network myrinet2k
SCSTEST: BURST on myrinet2k(chan=0,ctx=0,len=16)
50 K packets.
100 K packets.
150 K packets.
200 K packets.
250 K packets.
300 K packets.
...
7150 K packets.
etc
Seems to work fine.
> Shinji.
>
> amik> I will give you the results as soon as they show up in 30-40 minutes.
> amik>
> amik>
> amik> Thank you for all the great help,
> amik>
> amik> Amik
> amik>
> amik>
> amik> > Shinji.
> amik> >
> amik> > From: Amik St-Cyr CFD Lab <amik at cfdlab.mcgill.ca>
> amik> > Subject: Re: [SCore-users] Bug found!?
> amik> > Date: 24 Sep 2002 14:19:16 -0400
> amik> > Message-ID: <1032891560.1223.144.camel at stan.cfdlab.mcgill.ca>
> amik> >
> amik> > amik> HPL.dat:
> amik> > amik>
> amik> > amik> HPLinpack benchmark input file
> amik> > amik> Innovative Computing Laboratory, University of Tennessee
> amik> > amik> HPL.out output file name (if any)
> amik> > amik> 6 device out (6=stdout,7=stderr,file)
> amik> > amik> 1 # of problems sizes (N)
> amik> > amik> 120000 Ns
> amik> > amik> 1 # of NBs
> amik> > amik> 100 NBs
> amik> > amik> 1 # of process grids (P x Q)
> amik> > amik> 16 Ps
> amik> > amik> 16 Qs
> amik> > amik> 16.0 threshold
> amik> > amik> 1 # of panel fact
> amik> > amik> 1 PFACTs (0=left, 1=Crout, 2=Right)
> amik> > amik> 1 # of recursive stopping criterium
> amik> > amik> 1 NBMINs (>= 1)
> amik> > amik> 1 # of panels in recursion
> amik> > amik> 16 NDIVs
> amik> > amik> 1 # of recursive panel fact.
> amik> > amik> 2 RFACTs (0=left, 1=Crout, 2=Right)
> amik> > amik> 1 # of broadcast
> amik> > amik> 3 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
> amik> > amik> 1 # of lookahead depth
> amik> > amik> 1 DEPTHs (>=0)
> amik> > amik> 2 SWAP (0=bin-exch,1=long,2=mix)
> amik> > amik> 16 swapping threshold
> amik> > amik> 0 L1 in (0=transposed,1=no-transposed) form
> amik> > amik> 0 U in (0=transposed,1=no-transposed) form
> amik> > amik> 1 Equilibration (0=no,1=yes)
> amik> > amik> 8 memory alignment in double (> 0)
> amik> > amik>
> amik> > amik>
> amik> > amik> <30:1> SCORE:WARNING MPICH/SCore: receive-request-queue:
> amik> > amik> <30:1> SCORE:WARNING MPICH/SCore [buffer=0x2367f868, type=1025,
> amik> > amik> from=-1, size=40000, offset=0]
> amik> > amik> <30:1> SCORE:WARNING MPICH/SCore: receive-message-queue:
> amik> > amik> <30:1> SCORE:WARNING MPICH/SCore (empty)
> amik> > amik> <30:1> SCORE:WARNING MPICH/SCore: received-fragment:
> amik> > amik> <30:1> SCORE:WARNING MPICH/SCore [buffer=0x400a4570, type=1025,
> amik> > amik> from=59, size=48000, fragment_size=8240, offset=0]
> amik> > amik> <30:1> SCORE:WARNING MPICH/SCore: expected-message:
> amik> > amik> <30:1> SCORE:WARNING MPICH/SCore [buffer=0x2367f868, type=1025,
> amik> > amik> from=-1, size=40000, offset=0]
> amik> > amik> <30:1> SCORE:WARNING MPICH/SCore: received message is larger than
> amik> > amik> expected
> amik> > amik> <30:1> SCORE:PANIC MPICH/SCore: critical error on message transfer
> amik> > amik> <30:1> Trying to attach GDB (DISPLAY=localhost:10.0): PANIC
> amik> > amik> SCORE: Program aborted.
> amik> > amik> | amik at stokes 14:12:25 Linux_ATHLON_CBLAS> rsh-all -g cn cat
> amik> > amik> /proc/pm/myrinet/0/info |grep CRC
> amik> > amik> cn1.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn5.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn2.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn52.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn17.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn3.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn51.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn50.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn6.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn7.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn4.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn49.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn48.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn9.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn47.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn10.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn11.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn46.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn8.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn43.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn13.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn12.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn14.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn58.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn45.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn15.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn42.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn41.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn38.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn40.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn39.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn16.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn19.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn37.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn36.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn59.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn56.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn18.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn54.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn44.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn22.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn31.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn20.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn55.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn53.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn35.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn23.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn34.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn33.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn30.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn32.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn57.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn21.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn24.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn29.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn65.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn26.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn27.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn63.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn28.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn64.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn62.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn25.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn60.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn61.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn68.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn66.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn71.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn67.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn72.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn75.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn69.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn79.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn73.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn74.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn78.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn83.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn76.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn77.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn84.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn82.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn81.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn85.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn94.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn91.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn70.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn80.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn89.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn99.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn92.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn95.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn87.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn86.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn93.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn88.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn100.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn96.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn124.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn120.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn98.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn105.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn97.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn107.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn104.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn103.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn102.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn106.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn113.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn90.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn108.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn114.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn112.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn111.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn101.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn122.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn115.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn119.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn109.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn123.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn110.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn117.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn116.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn118.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn121.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn128.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn127.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn125.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik> cn126.clumeq.mcgill.ca: CRC errors : 0
> amik> > amik>
> amik> > amik>
> amik> > amik> On Tue, 2002-09-24 at 05:36, Shinji Sumimoto wrote:
> amik> > amik> > Hi.
> amik> > amik> >
> amik> > amik> > These messages are output when some messages do not send properly
> amik> > amik> > node# 52, and node# 52 re-send the messages.
> amik> > amik> >
> amik> > amik> > Could you check CRC error on Myrinet NICs?
> amik> > amik> >
> amik> > amik> > When the messages are displayed:
> amik> > amik> >
> amik> > amik> > $ rsh-all -g pcc cat /proc/pm/myrinet/0/info |grep CRC
> amik> > amik> >
> amik> > amik> > Shinji.
> amik> > amik> >
> amik> > amik> > From: Amik St-Cyr CFD Lab <amik at cfdlab.mcgill.ca>
> amik> > amik> > Subject: Re: [SCore-users] Bug found!?
> amik> > amik> > Date: 23 Sep 2002 17:26:03 -0400
> amik> > amik> > Message-ID: <1032816364.9729.1279.camel at stan.cfdlab.mcgill.ca>
> amik> > amik> >
> amik> > amik> > amik> Hi,
> amik> > amik> > amik>
> amik> > amik> > amik> I have tried the flag PM_DEBUG=1 and it seems that they
> amik> > amik> > amik> were fewer errors for the linpack benchmark for N=20000,50000,100000.
> amik> > amik> > amik> Now I am going for the almost biggest matrix we can factorize
> amik> > amik> > amik> N=200000 and I got:
> amik> > amik> > amik>
> amik> > amik> > amik>
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00802040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00802040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00002040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Send Warning: myriSendDMA: (ff10007c,ff10): 00000036 00000008
> amik> > amik> > amik> 0000000d 00001740 00001880 00001740 00402040 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik> [52] Receive Warning: myriReceiveDMA: (1a007c): 00402040 00000036
> amik> > amik> > amik> 0000a140 00000000 00000000 00000000 00000000 00000000
> amik> > amik> > amik>
> amik> > amik> > amik> But it still continues to compute.
> amik> > amik> > amik>
> amik> > amik> > amik>
> amik> > amik> > amik> Amik
> amik> > amik> > amik>
> amik> > amik> > amik>
> amik> > amik> > amik> On Mon, 2002-09-23 at 10:24, Shinji Sumimoto wrote:
> amik> > amik> > amik> > Hi.
> amik> > amik> > amik> >
> amik> > amik> > amik> > From: Amik St-Cyr CFD Lab <amik at cfdlab.mcgill.ca>
> amik> > amik> > amik> > Subject: Re: [SCore-users] Bug found!?
> amik> > amik> > amik> > Date: 23 Sep 2002 10:18:37 -0400
> amik> > amik> > amik> > Message-ID: <1032790717.9729.1167.camel at stan.cfdlab.mcgill.ca>
> amik> > amik> > amik> >
> amik> > amik> > amik> > amik> I only need to link with -device ch_score2 ??
> amik> > amik> > amik> >
> amik> > amik> > amik> > You are right, if ch_score2 library is installed on your cluster. But
> amik> > amik> > amik> > ch_score2 library is not installed at default, you have to compile and
> amik> > amik> > amik> > install from source code of SCore.
> amik> > amik> > amik> >
> amik> > amik> > amik> > If you want to use eager protocol on MPICH/SCore, use following option.
> amik> > amik> > amik> >
> amik> > amik> > amik> > $ scrun -nodes=128x2,mpi_eager=2000000 ./xhpl
> amik> > amik> > amik> >
> amik> > amik> > amik> > Shinji.
> amik> > amik> > amik> >
> amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> Amik
> amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> On Mon, 2002-09-23 at 10:09, Shinji Sumimoto wrote:
> amik> > amik> > amik> > amik> > Hi.
> amik> > amik> > amik> > amik> >
> amik> > amik> > amik> > amik> > Could you run the HPL program with setting PM_DEBUG=1 shell
> amik> > amik> > amik> > amik> > environment variable?
> amik> > amik> > amik> > amik> >
> amik> > amik> > amik> > amik> > $ export PM_DEBUG=1
> amik> > amik> > amik> > amik> > $ scrun -nodes=128x2,mpi_max_eager_myrinet=2000000 ./xhpl
> amik> > amik> > amik> > amik> >
> amik> > amik> > amik> > amik> > The error:
> amik> > amik> > amik> > amik> > amik> <13:0> SCORE:WARNING MPICH/SCore: pmReceive(pmc=0x85bc7a0) failed,
> amik> > amik> > amik> > amik> > amik> errno=5
> amik> > amik> > amik> > amik> > shows an error has occurred on Myrinet communication.
> amik> > amik> > amik> > amik> >
> amik> > amik> > amik> > amik> > By setting PM_DEBUG=1, more informative messages are displayed.
> amik> > amik> > amik> > amik> >
> amik> > amik> > amik> > amik> > Maybe you are using Myrinet2000. Is you Myrinet2000 with fiber link or
> amik> > amik> > amik> > amik> > serial link ?
> amik> > amik> > amik> > amik> >
> amik> > amik> > amik> > amik> > PS: The mpi_max_eager_myrinet option is only for ch_score2 mpi device.
> amik> > amik> > amik> > amik> > Are you using MPICH/SCore2?
> amik> > amik> > amik> > amik> >
> amik> > amik> > amik> > amik> > Shinji.
> amik> > amik> > amik> > amik> >
> amik> > amik> > amik> > amik> > From: Amik St-Cyr CFD Lab <amik at cfdlab.mcgill.ca>
> amik> > amik> > amik> > amik> > Subject: [SCore-users] Bug found!?
> amik> > amik> > amik> > amik> > Date: 23 Sep 2002 09:40:35 -0400
> amik> > amik> > amik> > amik> > Message-ID: <1032788436.9725.1156.camel at stan.cfdlab.mcgill.ca>
> amik> > amik> > amik> > amik> >
> amik> > amik> > amik> > amik> > amik> Hi all,
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> First of all our tech team found the
> amik> > amik> > amik> > amik> > amik> bug in our setup. Thank you for the help of mr.
> amik> > amik> > amik> > amik> > amik> Toyohisa. We had mixed up two cables...
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> We then have tried to benckmark the machine
> amik> > amik> > amik> > amik> > amik> with the LINPACK suite. The behaviour of the linpack
> amik> > amik> > amik> > amik> > amik> was very strange depending on the parameters given.
> amik> > amik> > amik> > amik> > amik> The for a specified setup I was able to crank up the
> amik> > amik> > amik> > amik> > amik> size of the matrix but then SCORE gave me the
> amik> > amik> > amik> > amik> > amik> following message:
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> | amik at stokes 18:22:56 Linux_ATHLON_CBLAS> scrun
> amik> > amik> > amik> > amik> > amik> -nodes=128x2,mpi_max_eager_myrinet=2000000 ./xhpl
> amik> > amik> > amik> > amik> > amik> SCore-D 5.0.1 connected.
> amik> > amik> > amik> > amik> > amik> <0:0> SCORE: 256 nodes (128x2) ready.
> amik> > amik> > amik> > amik> > amik> ============================================================================
> amik> > amik> > amik> > amik> > amik> HPLinpack 1.0 -- High-Performance Linpack benchmark -- September 27,
> amik> > amik> > amik> > amik> > amik> 2000
> amik> > amik> > amik> > amik> > amik> Written by A. Petitet and R. Clint Whaley, Innovative Computing Labs.,
> amik> > amik> > amik> > amik> > amik> UTK
> amik> > amik> > amik> > amik> > amik> ============================================================================
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> An explanation of the input/output parameters follows:
> amik> > amik> > amik> > amik> > amik> T/V : Wall time / encoded variant.
> amik> > amik> > amik> > amik> > amik> N : The order of the coefficient matrix A.
> amik> > amik> > amik> > amik> > amik> NB : The partitioning blocking factor.
> amik> > amik> > amik> > amik> > amik> P : The number of process rows.
> amik> > amik> > amik> > amik> > amik> Q : The number of process columns.
> amik> > amik> > amik> > amik> > amik> Time : Time in seconds to solve the linear system.
> amik> > amik> > amik> > amik> > amik> Gflops : Rate of execution for solving the linear system.
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> The following parameter values will be used:
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> N : 200000
> amik> > amik> > amik> > amik> > amik> NB : 100
> amik> > amik> > amik> > amik> > amik> P : 16
> amik> > amik> > amik> > amik> > amik> Q : 16
> amik> > amik> > amik> > amik> > amik> PFACT : Crout
> amik> > amik> > amik> > amik> > amik> NBMIN : 1
> amik> > amik> > amik> > amik> > amik> NDIV : 16
> amik> > amik> > amik> > amik> > amik> RFACT : Right
> amik> > amik> > amik> > amik> > amik> BCAST : 2ringM
> amik> > amik> > amik> > amik> > amik> DEPTH : 1
> amik> > amik> > amik> > amik> > amik> SWAP : Mix (threshold = 16)
> amik> > amik> > amik> > amik> > amik> L1 : transposed form
> amik> > amik> > amik> > amik> > amik> U : transposed form
> amik> > amik> > amik> > amik> > amik> EQUIL : yes
> amik> > amik> > amik> > amik> > amik> ALIGN : 8 double precision words
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> ----------------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> - The matrix A is randomly generated for each test.
> amik> > amik> > amik> > amik> > amik> - The following scaled residual checks will be computed:
> amik> > amik> > amik> > amik> > amik> 1) ||Ax-b||_oo / ( eps * ||A||_1 * N )
> amik> > amik> > amik> > amik> > amik> 2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )
> amik> > amik> > amik> > amik> > amik> 3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )
> amik> > amik> > amik> > amik> > amik> - The relative machine precision (eps) is taken to be
> amik> > amik> > amik> > amik> > amik> 2.220446e-16
> amik> > amik> > amik> > amik> > amik> - Computational tests pass if scaled residuals are less than
> amik> > amik> > amik> > amik> > amik> 16.0
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> <13:0> SCORE:WARNING MPICH/SCore: pmReceive(pmc=0x85bc7a0) failed,
> amik> > amik> > amik> > amik> > amik> errno=5
> amik> > amik> > amik> > amik> > amik> <13:0> SCORE:PANIC MPICH/SCore: critical error on message transfer
> amik> > amik> > amik> > amik> > amik> <13:0> Trying to attach GDB (DISPLAY=localhost:13.0): PANIC
> amik> > amik> > amik> > amik> > amik> SCORE: Program aborted.
> amik> > amik> > amik> > amik> > amik> | amik at stokes 23:14:53 Linux_ATHLON_CBLAS>
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> Can I prevent this from happening?
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> Here are the details:
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> | amik at stokes 09:40:11 Linux_ATHLON_CBLAS> cat HPL.dat
> amik> > amik> > amik> > amik> > amik> HPLinpack benchmark input file
> amik> > amik> > amik> > amik> > amik> Innovative Computing Laboratory, University of Tennessee
> amik> > amik> > amik> > amik> > amik> HPL256.out output file name (if any)
> amik> > amik> > amik> > amik> > amik> 6 device out (6=stdout,7=stderr,file)
> amik> > amik> > amik> > amik> > amik> 1 # of problems sizes (N)
> amik> > amik> > amik> > amik> > amik> 200000 Ns
> amik> > amik> > amik> > amik> > amik> 1 # of NBs
> amik> > amik> > amik> > amik> > amik> 100 NBs
> amik> > amik> > amik> > amik> > amik> 1 # of process grids (P x Q)
> amik> > amik> > amik> > amik> > amik> 16 Ps
> amik> > amik> > amik> > amik> > amik> 16 Qs
> amik> > amik> > amik> > amik> > amik> 16.0 threshold
> amik> > amik> > amik> > amik> > amik> 1 # of panel fact
> amik> > amik> > amik> > amik> > amik> 1 PFACTs (0=left, 1=Crout, 2=Right)
> amik> > amik> > amik> > amik> > amik> 1 # of recursive stopping criterium
> amik> > amik> > amik> > amik> > amik> 1 NBMINs (>= 1)
> amik> > amik> > amik> > amik> > amik> 1 # of panels in recursion
> amik> > amik> > amik> > amik> > amik> 16 NDIVs
> amik> > amik> > amik> > amik> > amik> 1 # of recursive panel fact.
> amik> > amik> > amik> > amik> > amik> 2 RFACTs (0=left, 1=Crout, 2=Right)
> amik> > amik> > amik> > amik> > amik> 1 # of broadcast
> amik> > amik> > amik> > amik> > amik> 3 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
> amik> > amik> > amik> > amik> > amik> 1 # of lookahead depth
> amik> > amik> > amik> > amik> > amik> 1 DEPTHs (>=0)
> amik> > amik> > amik> > amik> > amik> 2 SWAP (0=bin-exch,1=long,2=mix)
> amik> > amik> > amik> > amik> > amik> 16 swapping threshold
> amik> > amik> > amik> > amik> > amik> 0 L1 in (0=transposed,1=no-transposed) form
> amik> > amik> > amik> > amik> > amik> 0 U in (0=transposed,1=no-transposed) form
> amik> > amik> > amik> > amik> > amik> 1 Equilibration (0=no,1=yes)
> amik> > amik> > amik> > amik> > amik> 8 memory alignment in double (> 0)
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> Make file using Portland group compiler and linking with
> amik> > amik> > amik> > amik> > amik> the mpi in SCORE.
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> | amik at stokes 09:40:47 hpl> cat Make.Linux_ATHLON_CBLAS
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # -- High Performance Computing Linpack Benchmark (HPL)
> amik> > amik> > amik> > amik> > amik> # HPL - 1.0 - September 27, 2000
> amik> > amik> > amik> > amik> > amik> # Antoine P. Petitet
> amik> > amik> > amik> > amik> > amik> # University of Tennessee, Knoxville
> amik> > amik> > amik> > amik> > amik> # Innovative Computing Laboratories
> amik> > amik> > amik> > amik> > amik> # (C) Copyright 2000 All Rights Reserved
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # -- Copyright notice and Licensing terms:
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # Redistribution and use in source and binary forms, with or without
> amik> > amik> > amik> > amik> > amik> # modification, are permitted provided that the following conditions
> amik> > amik> > amik> > amik> > amik> # are met:
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # 1. Redistributions of source code must retain the above copyright
> amik> > amik> > amik> > amik> > amik> # notice, this list of conditions and the following disclaimer.
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # 2. Redistributions in binary form must reproduce the above copyright
> amik> > amik> > amik> > amik> > amik> # notice, this list of conditions, and the following disclaimer in the
> amik> > amik> > amik> > amik> > amik> # documentation and/or other materials provided with the distribution.
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # 3. All advertising materials mentioning features or use of this
> amik> > amik> > amik> > amik> > amik> # software must display the following acknowledgement:
> amik> > amik> > amik> > amik> > amik> # This product includes software developed at the University of
> amik> > amik> > amik> > amik> > amik> # Tennessee, Knoxville, Innovative Computing Laboratories.
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # 4. The name of the University, the name of the Laboratory, or the
> amik> > amik> > amik> > amik> > amik> # names of its contributors may not be used to endorse or promote
> amik> > amik> > amik> > amik> > amik> # products derived from this software without specific written
> amik> > amik> > amik> > amik> > amik> # permission.
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # -- Disclaimer:
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> amik> > amik> > amik> > amik> > amik> # ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> amik> > amik> > amik> > amik> > amik> # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> amik> > amik> > amik> > amik> > amik> # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE UNIVERSITY
> amik> > amik> > amik> > amik> > amik> # OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> amik> > amik> > amik> > amik> > amik> # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> amik> > amik> > amik> > amik> > amik> # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> amik> > amik> > amik> > amik> > amik> # DATA OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> amik> > amik> > amik> > amik> > amik> # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> amik> > amik> > amik> > amik> > amik> # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> amik> > amik> > amik> > amik> > amik> # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> amik> > amik> > amik> > amik> > amik> # ######################################################################
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # - shell --------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> SHELL = /bin/sh
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> CD = cd
> amik> > amik> > amik> > amik> > amik> CP = cp
> amik> > amik> > amik> > amik> > amik> LN_S = ln -s
> amik> > amik> > amik> > amik> > amik> MKDIR = mkdir
> amik> > amik> > amik> > amik> > amik> RM = /bin/rm -f
> amik> > amik> > amik> > amik> > amik> TOUCH = touch
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # - Platform identifier ------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> ARCH = Linux_ATHLON_CBLAS
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # - HPL Directory Structure / HPL library ------------------------------
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> TOPdir = $(HOME)/hpl
> amik> > amik> > amik> > amik> > amik> INCdir = $(TOPdir)/include
> amik> > amik> > amik> > amik> > amik> BINdir = $(TOPdir)/bin/$(ARCH)
> amik> > amik> > amik> > amik> > amik> LIBdir = $(TOPdir)/lib/$(ARCH)
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> HPLlib = $(LIBdir)/libhpl.a
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # - Compilers / linkers - Optimization flags ---------------------------
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> CC = pgcc
> amik> > amik> > amik> > amik> > amik> NOOPT =
> amik> > amik> > amik> > amik> > amik> CCFLAGS = -O2 -Munroll=c:4 -Mvect=prefetch -Mvect=cachesize:32768
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> LINKER = pgcc
> amik> > amik> > amik> > amik> > amik> LINKFLAGS = $(CCFLAGS)
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> ARCHIVER = ar
> amik> > amik> > amik> > amik> > amik> ARFLAGS = r
> amik> > amik> > amik> > amik> > amik> RANLIB = echo
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # - MPI directories - library ------------------------------------------
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # MPinc tells the C compiler where to find the Message Passing library
> amik> > amik> > amik> > amik> > amik> # header files, MPlib is defined to be the name of the library to be
> amik> > amik> > amik> > amik> > amik> # used. The variable MPdir is only used for defining MPinc and MPlib.
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> MPdir = /opt/score/mpi/mpich-1.2.0/i386-redhat7-linux2_4
> amik> > amik> > amik> > amik> > amik> MPinc = -I$(MPdir)/include
> amik> > amik> > amik> > amik> > amik> MPlib = -Bstatic -L$(MPdir)/lib
> amik> > amik> > amik> > amik> > amik> -L/opt/score/lib/i386-redhat7-linux2_4 -lmpich -lscoreusr -lpm -lult
> amik> > amik> > amik> > amik> > amik> -lscorecommon -lpthread -lscboard -lscwrap -L/usr/pgi/linux86/lib
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # - F77 / C interface --------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # You can skip this section if and only if you are not planning to use
> amik> > amik> > amik> > amik> > amik> # a BLAS library featuring a Fortran 77 interface. Otherwise, it is
> amik> > amik> > amik> > amik> > amik> # necessary to fill out the F2CDEFS variable with the appropriate
> amik> > amik> > amik> > amik> > amik> # options. **One and only one** option should be chosen in **each** of
> amik> > amik> > amik> > amik> > amik> # the 3 following categories:
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # 1) name space (How C calls a Fortran 77 routine)
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # -DAdd_ : all lower case and a suffixed underscore (Suns,
> amik> > amik> > amik> > amik> > amik> # Intel, ...),
> amik> > amik> > amik> > amik> > amik> # -DNoChange : all lower case (IBM RS6000),
> amik> > amik> > amik> > amik> > amik> # -DUpCase : all upper case (Cray),
> amik> > amik> > amik> > amik> > amik> # -Df77IsF2C : the FORTRAN compiler in use is f2c.
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # 2) C and Fortran 77 integer mapping
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # -DF77_INTEGER=int : Fortran 77 INTEGER is a C int,
> amik> > amik> > amik> > amik> > amik> # -DF77_INTEGER=long : Fortran 77 INTEGER is a C long,
> amik> > amik> > amik> > amik> > amik> # -DF77_INTEGER=short : Fortran 77 INTEGER is a C short.
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # 3) Fortran 77 string handling
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # -DStringSunStyle : The string address is passed at the string loca-
> amik> > amik> > amik> > amik> > amik> # tion on the stack, and the string length is then
> amik> > amik> > amik> > amik> > amik> # passed as an F77_INTEGER after all explicit
> amik> > amik> > amik> > amik> > amik> # stack arguments,
> amik> > amik> > amik> > amik> > amik> # -DStringStructPtr : The address of a structure is passed by a
> amik> > amik> > amik> > amik> > amik> # Fortran 77 string, and the structure is of the
> amik> > amik> > amik> > amik> > amik> # form: struct {char *cp; F77_INTEGER len;},
> amik> > amik> > amik> > amik> > amik> # -DStringStructVal : A structure is passed by value for each Fortran
> amik> > amik> > amik> > amik> > amik> # 77 string, and the structure is of the form:
> amik> > amik> > amik> > amik> > amik> # struct {char *cp; F77_INTEGER len;},
> amik> > amik> > amik> > amik> > amik> # -DCrayStyle : Special option for Cray machines, which uses
> amik> > amik> > amik> > amik> > amik> # Cray fcd (fortran character descriptor) for
> amik> > amik> > amik> > amik> > amik> # interoperation.
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> F2CDEFS =
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # - Linear Algebra library (BLAS or VSIPL) -----------------------------
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # LAinc tells the C compiler where to find the Linear Algebra library
> amik> > amik> > amik> > amik> > amik> # header files, LAlib is defined to be the name of the library to be
> amik> > amik> > amik> > amik> > amik> # used. The variable LAdir is only used for defining LAinc and LAlib.
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> LAdir = /home/amik/Linux_ATHLON
> amik> > amik> > amik> > amik> > amik> LAinc = -I$(LAdir)/include
> amik> > amik> > amik> > amik> > amik> LAlib = $(LAdir)/lib/libcblas.a $(LAdir)/lib/libatlas.a
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> # - HPL includes / libraries / specifics -------------------------------
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc)
> amik> > amik> > amik> > amik> > amik> HPL_LIBS = $(HPLlib) $(LAlib) $(MPlib)
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # - Compile time options -----------------------------------------------
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # -DHPL_COPY_L force the copy of the panel L before bcast;
> amik> > amik> > amik> > amik> > amik> # -DHPL_CALL_CBLAS call the cblas interface;
> amik> > amik> > amik> > amik> > amik> # -DHPL_CALL_VSIPL call the vsip library;
> amik> > amik> > amik> > amik> > amik> # -DHPL_DETAILED_TIMING enable detailed timers;
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # By default HPL will:
> amik> > amik> > amik> > amik> > amik> # *) not copy L before broadcast,
> amik> > amik> > amik> > amik> > amik> # *) call the Fortran 77 BLAS interface
> amik> > amik> > amik> > amik> > amik> # *) not display detailed timing information.
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> HPL_OPTS = -DHPL_CALL_CBLAS
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> HPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
> amik> > amik> > amik> > amik> > amik> #
> amik> > amik> > amik> > amik> > amik> # ----------------------------------------------------------------------
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> --
> amik> > amik> > amik> > amik> > amik> _____________________________________________________
> amik> > amik> > amik> > amik> > amik> Dr. A. St-Cyr
> amik> > amik> > amik> > amik> > amik> Research Associate, CFD Lab
> amik> > amik> > amik> > amik> > amik> Department of Mechanical Engineering
> amik> > amik> > amik> > amik> > amik> McGill University
> amik> > amik> > amik> > amik> > amik> 688 Sherbrooke Street West, 7th floor
> amik> > amik> > amik> > amik> > amik> Montreal, Qc, Canada H3A 2S6
> amik> > amik> > amik> > amik> > amik> Tel: +1 (514) 398-1710, Admin. Fax : 2203
> amik> > amik> > amik> > amik> > amik> amik at cfdlab.mcgill.ca
> amik> > amik> > amik> > amik> > amik> _____________________________________________________
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > amik> _______________________________________________
> amik> > amik> > amik> > amik> > amik> SCore-users mailing list
> amik> > amik> > amik> > amik> > amik> SCore-users at pccluster.org
> amik> > amik> > amik> > amik> > amik> http://www.pccluster.org/mailman/listinfo/score-users
> amik> > amik> > amik> > amik> > amik>
> amik> > amik> > amik> > amik> > -----
> amik> > amik> > amik> > amik> > Shinji Sumimoto E-Mail: s-sumi at bd6.so-net.ne.jp
> amik> > amik> > amik> > amik> --
> amik> > amik> > amik> > amik> _____________________________________________________
> amik> > amik> > amik> > amik> Dr. A. St-Cyr
> amik> > amik> > amik> > amik> Research Associate, CFD Lab
> amik> > amik> > amik> > amik> Department of Mechanical Engineering
> amik> > amik> > amik> > amik> McGill University
> amik> > amik> > amik> > amik> 688 Sherbrooke Street West, 7th floor
> amik> > amik> > amik> > amik> Montreal, Qc, Canada H3A 2S6
> amik> > amik> > amik> > amik> Tel: +1 (514) 398-1710, Admin. Fax : 2203
> amik> > amik> > amik> > amik> amik at cfdlab.mcgill.ca
> amik> > amik> > amik> > amik> _____________________________________________________
> amik> > amik> > amik> > amik>
> amik> > amik> > amik> > -----
> amik> > amik> > amik> > Shinji Sumimoto E-Mail: s-sumi at bd6.so-net.ne.jp
> amik> > amik> > amik> --
> amik> > amik> > amik> _____________________________________________________
> amik> > amik> > amik> Dr. A. St-Cyr
> amik> > amik> > amik> Research Associate, CFD Lab
> amik> > amik> > amik> Department of Mechanical Engineering
> amik> > amik> > amik> McGill University
> amik> > amik> > amik> 688 Sherbrooke Street West, 7th floor
> amik> > amik> > amik> Montreal, Qc, Canada H3A 2S6
> amik> > amik> > amik> Tel: +1 (514) 398-1710, Admin. Fax : 2203
> amik> > amik> > amik> amik at cfdlab.mcgill.ca
> amik> > amik> > amik> _____________________________________________________
> amik> > amik> > amik>
> amik> > amik> > amik>
> amik> > amik> > ------
> amik> > amik> > Shinji Sumimoto, Fujitsu Labs
> amik> > amik> --
> amik> > amik> _____________________________________________________
> amik> > amik> Dr. A. St-Cyr
> amik> > amik> Research Associate, CFD Lab
> amik> > amik> Department of Mechanical Engineering
> amik> > amik> McGill University
> amik> > amik> 688 Sherbrooke Street West, 7th floor
> amik> > amik> Montreal, Qc, Canada H3A 2S6
> amik> > amik> Tel: +1 (514) 398-1710, Admin. Fax : 2203
> amik> > amik> amik at cfdlab.mcgill.ca
> amik> > amik> _____________________________________________________
> amik> > amik>
> amik> > amik>
> amik> > ------
> amik> > Shinji Sumimoto, Fujitsu Labs
> amik> --
> amik> _____________________________________________________
> amik> Dr. A. St-Cyr
> amik> Research Associate, CFD Lab
> amik> Department of Mechanical Engineering
> amik> McGill University
> amik> 688 Sherbrooke Street West, 7th floor
> amik> Montreal, Qc, Canada H3A 2S6
> amik> Tel: +1 (514) 398-1710, Admin. Fax : 2203
> amik> amik at cfdlab.mcgill.ca
> amik> _____________________________________________________
> amik>
> amik>
> ------
> Shinji Sumimoto, Fujitsu Labs
--
_____________________________________________________
Dr. A. St-Cyr
Research Associate, CFD Lab
Department of Mechanical Engineering
McGill University
688 Sherbrooke Street West, 7th floor
Montreal, Qc, Canada H3A 2S6
Tel: +1 (514) 398-1710, Admin. Fax : 2203
amik at cfdlab.mcgill.ca
_____________________________________________________
More information about the SCore-users
mailing list