[SCore-users-jp] Re: [SCore-users] MPI and PM at the same time

[Atsushi HORI]

Hi.

>> Define the number of network sets with the RESOURCE MACRO, like this.
>> 
>> SCORE_RSRC_NUM_NETS(N)
>
>I've already tried to set directly the score_num_pmnet variable which is
>mentioned in the score_initialize() man page and after MPI_Init(), the
>number of contexts is always 1. When using this macro, the compiler (with
>-Wall) warns that "unused variable `score_resource_num_netsets'" and the
>result is always 1 context.

Aha, the macro should be placed outside of function declaration. 

>But the real problem is that I can't use this method. The ARMCI library
>has to be initialized *after* MPI, so that it already has all processes up
>and running. That's why I asked how to obtain another context starting
>from the one used by MPI.

Thus you do not have to mind this. The macro is a declaration (of a 
global static variable), not a function call. And can be placed 
anywhere in your program.

>In order to have another context, I was trying to get the device used by
>the MPI context so that I can call pmOpenContext and get a second context
>on this device - that's where I discovered that ->device was NULL and of
>course I couldn't use it in the pmOpenContext call. I also tried to get
>->device for the "children" contexts attached to real devices which are in
>one case only shmem and in the other only myrinet (and I've also tried on
>a larger number on nodes to have both shmem and myrinet at the same time,
>but the output becomes long - available on request).

1. Any user programs are not wllowed to open or create a new PM 
contexts. It is SCore-D responsible. SCore-D allocate appropriate 
contexts for users according to cluster configuration, user program 
request and runtime options given by user.

2. All user programs can do is attach context(s) to a given file 
descriptor(s). Thus you have to ASK (or declare ?) to SCore-D how 
many PM contexts to use.

3. PM's context is not a point-to-point communication. Just one 
context is enough to communicate with the other nodes.

>Is there any other way of getting another context ? How about using 
>pmSaveContext/pmRestoreContext to get a copy of the first context (as we 
>want the same connectivity) ? 

No. They are for checkpoint and restart.

>What is pmAttachContext used for ? The documentation for pmCreateAttachFd 
>says that the fd obtained there could be used in pmAttachContext. But what 
>for ? If I have a context I attach a fd to it so that I can use select(2), 
>but then I use this fd and a context type to create another context ???

You have to pay attention to the PM documents. Some functions are 
privileged and can be used only by SCore-D. As mentioned above, 
SCore-D creates PM context and pass FDs created by pmCreateAttachFd() 
function. All user program can do is attaching PM contexts to given 
FDs. And this is done by score_initialize() function.

And your program should be written as the following;

---------------------------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>
#include <sc.h>
#include <score.h>
#include <errno.h>
#include <string.h>
#include <score_resource.h>

SCORE_RSRC_NUM_NETS(2); /* PLACE THIS MACRO OUTSIDE OF FUNCTION 
DECLARATION */

pmContext *mpic, *pc;
pmDevice *pd;

void fatal(char *s, int err) {
printf("#%d: %s %s\n", score_self_node, s, pmErrorString(err)); 
fflush(stdout);
exit(1);
}

int main(int argc, char **argv)
{
int err, i;
pmContextConfig cc;
pmContext *allc[PM_MAX_NODE]; /*** ITHINK ONLY ONE IS ENOUGH HERE ***/
int allnr[PM_MAX_NODE];

/*** DO NOT PLACE THIS MACRO IN A FUNCTION DECLARATION
SCORE_RSRC_NUM_NETS(2);
***/

MPI_Init(&argc, &argv);

if (score_num_pmnet < 1) {
	printf("No context !!!\n");
	return 1;
	}
else printf("#%d: Nr. of contexts: %d\n", score_self_node, 
score_num_pmnet);


mpic = score_pmnet[0]; /* This context is for MPICH/SCORE ***/
/*** AND NEVER USE FOR THE OETHER COMMUNICATIONS ***/

/* truncated */

----
Atsushi HORI
Swimmy Software, Inc.

_______________________________________________
SCore-users mailing list
SCore-users ＠ pccluster.org
http://www.pccluster.org/mailman/listinfo/score-users