How to use STG * Installation ./configure --prefix=/your/home/directory/local make make install * Usage mpiexec /your/home/directory/local/bin/stg -i -f stagein.cfg mpiexec your_mpi_program mpiexec /your/home/directory/local/bin/stg -o -f stageout.cfg !! options for mpiexec must be same as ones for your_mpi_program. !! * STG Configuration file Each line should contain file names for staging with following rules: 1st column: "in", or "out" 2nd column: filename on a file server 3rd column: filename on a local disk - 1st column in: means staging files in compute nodes. Valid only when STG is invoked with -i option. out: means staging files out of compute nodes. Valid only when STG is invoked with -o option. - 2nd column Filename on a file server is written here. %d is substituted with rank number. - 3rd column Filename on a local disk is written here. %d is substituted with rank number. - examples (1) in /home/a00000/data/data.%d /tmp/data.%d By this line, STG performs the following copy operations: /home/a00000/data/data.0 --> rank0:/tmp/data.0 /home/a00000/data/data.1 --> rank0:/tmp/data.1 /home/a00000/data/data.2 --> rank0:/tmp/data.2 ..... (2) in /home/a00000/program/a.out /tmp/a.out By this line, STG performs the following copy operations: /home/a00000/program/a.out --> /tmp/a.out (in all nodes) (3) out /home/a00000/data/result.%d /tmp/data.%d By this line, STG performs the following copy operations: rank0:/tmp/data.0 --> /home/a00000/data/result.0 rank1:/tmp/data.1 --> /home/a00000/data/result.1 rank2:/tmp/data.2 --> /home/a00000/data/result.2 ..... * How to test STG Test STG with the follwing procedure. In this test, it is assumed that home directories can be accessed both from a login-node and compute nodes, and that /tmp is local file system in each compte node. (1) Install STG as described above. (2) $ mkdir ~/stgtest (3) $ cd ~/stgtest (4) $ for i in 00 01 02 03;do dd if=/dev/zero of=data.$i bs=1024 count=$[64 * 1024];done (5) $ dd if=/dev/zero of=a.out bs=4096 count=1 (6) Make stg.cfg with the following contents: in data.%02d /tmp/data/data.%02d in a.out /tmp/a.out out outdata.%d /tmp/data/data.%02d (7) $ mpiexec -1 -n 4 /home/a00000/mpi/stg/bin/stg -i -f stg.cfg * If you can find /tmp/data/data.%02d and /tmp/a.out in compute node, staging-in is working fine. (8) $ mpiexec -1 -n 4 /home/a00000/mpi/stg/bin/stg -o -f stg.cfg * If you can find outdata.%d in the current directory, staging-out works fine.