SCore logo

SCore Glossary
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
AFS
A distributed file system originally developed at CMU. AFS is a product of IBM Transarc Lab See http://www.transarc.com/index.html.
auto-negotiation
Ethernet has several physical layers in its speed (10Mbps, 100Mbps and 1Gbps) and direction (uni- or bi-direction). Network hardware has to negotiate with each other how to link with at the very first connection. Most Ethernet hardware (NIC or HUB) has auto-negotiation facility, however, sometimes network hardware fails to negotiate, and a connection is made in mistakenly. When this happens, communication performance can be degraded far from its specifications. To avoid this situation, user has to set up the connection manually.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
binary type
A unique symbol to distinguish processor type and host OS.
Proessor Type
OS
Binary Type
i386SuSE Linuxi386-suse-linux2_6
i386Centos 4i386-centos4-linux2_6
i386Fedora Core 5i386-fedoracore5-linux2_6
x86_64Centos 4x86_64-centos4-linux2_6
x86_64Fedora Core 5x86_64-fedoracore5-linux2_6
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
C-Area
C-Area is a shared memory region between SCore-D process and user process in a user parallel process. When a user process is forked, all the information to initialize user process is passed via C-Area, such as node number, number of nodes, PMv2 context information, and so on. C-Area is also used when a user program raises a systemcall to SCore-D.
CentOS
A Linux distribution. See http://www.centos.org/.
(PM) channel
PM channel is a virtualized network. In PMv2, physical network is multiplexed by software. Each multiplexed network is called (PM) channel.
cluster
Networked computers. In SCore, a cluster consists of homogeneous computers.
cluster of clusters
Networked clusters. In SCore, a cluster of clusters can be heterogeneous.
compute host
A host that is a member of a cluster.
cluster host
See compute host.
(PM) Composite
PM/Composite device is not an actual network device. It only forwards communication requests to other real PM device(s) according to a routing table which PM/Composite has. Having unique PM/Composite, users can combinme any arbitray PM devices. PM/Composite is also designed for thread safe, although the other real PM devices are not. When communication requests are forwarded, PM/Composite locks not to accept the other request from the other thread. Therfore, SCore-D always allocates one PM/Composite device and the othe PM device(s) for a user parallel process running on two or more nodes.
(PM) configuration file
A file needed to configure a PM device as a network. The file format of the configuraiton file depends on a PM device.
(PM) context
A set of state representing PM channel status.
co-scheduling
A scheduling technique for multi processors proposed by Ousterhout 1982. In co-scheduling, context switching of some or all processes in a parallel process is synchronized to obtain higher efficiency. See also gang-scheduling.
COW
COW stands for Cluster Of Workstations.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
deadlock detection
Deadlock detection is one of the most unique features of SCore-D. SCore-D can detect a deadlock situation of a user parallel process. SCore-D can detect more than deadlock. The situation such that all process in a parallel process is just waiting for incoming messages can also be detected. Strictly speaking, this kind of situation is not a deadlock, since there is no resource contention. The term of 'deadlock' is used here, just because it inspires users the situation where parallel computation will never proceed. This situation is defined as that there is no transient messages and no busy processes in a parallel process. SCore-D checks this everytime multiple context switch (gang scheduling) takes place.
(PM) device
There are several PM devices having the same PMv2 API. Currently, there are six devices, PM/Etherner, PM/Myrinet, PM/Shmem, and PM/Composite developed.
(SCore-D) device
Actually SCore-D device is not a physical device, but a sequential Unix process written in C. Thus users can develop their own devices, and attach them to SCore-D. This SCore-D device is one of the unique features of SCore-D. Using SCore-D device, user parallel program can wait for the I/O through the device in a blocking way. One user of SCore system developed a parallel server running on a cluster accepting requests from WWW. The user also developed an SCore-D device to accept requests from a CGI program, and the server is waiting for the requests. When there is no request, the server parallel process is blocked. When a request arrives, server parallel process begins to run with other users parallel procceses in TSSS. Thus, the cluster can be used for a server and program develepment simultaneously.
DMA
DMA stands for Direct Memory Access. A memory copying hardware to move data between memory regions, or an I/O device and a memory region. In many cases, DMA is used for moving I/O data. In general, DMA has longer setup time than CPU memory copying, but DMA may have larger copying bandwidth.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
(PM) Ethernet
PM/Ethernet is a PM device for Ethernet NICs. It supports any 10, 100 and 1Gbps NICs. For enabling high performance and high portability, PM/Ethernet is desinged in a unique way. There is a small kernel patch to discriminate IP (Internet Protocol) packets and PM/Ethernet packets. IP packets are handled in IP protocol layer, and PM/Ethernet packets are handled in the PM/Ethernet driver. Thus, an Ethernet NIC can handle both IP and PM/Ehernet packets. PM/Ethernet achieves higher performance compared with the communication using UDP or TCP protocol, however, there is one drawback. Further, PM/Ethernet supports network trunking. PM/Ethernet uses the Ethernet MAC address directly, and there is no way to route the PM/Ethernet packets out of a network segment. In this case, use PM/Agent instead. PM/Ethernet is not a user-level communication.
EIT
EIT stands for Easy Installation Tool to install SCore Cluster System easily with a GUI. It is designed for those who are unfamiliar with Linux installation.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
FEP
FEP stands for Front End Process. scrun is a FEP.
Fedora
A linux distribution. See http://www.fedoraproject.org/.
FQDN
FQDN stands for Full Qualified Domain Name. For example, pccluster.org is a FQDN. See also official name.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
global clock
PMv2 has some API functions for supporting globaly synchronized clock. However, these functions are only effective for PM device having a clock. Currently only Myrinet NIC has such a clock. SCore-D cooperating with PMv2 periodically synchronize NIC clocks. So that user program can read and use the clock value. However, due to the distributed nature, each clock is not synchronized fully, but consistently. This means that the there can be some clock skew, but the skew is not observed. Further, the clock synchronization is done in a best-effort way, and SCore-D does not guarantee for the consistency.
gang scheduling
A time sharing scheduling technique in that all process switching in a parallel process is synchronized. In this way, a time sharing scheduling can be practical even for fine grain parallel programs. In SCore-D, user parallel processes are gang-scheduled. See also co-scheduling.
global time stamp
See global clock.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
host
A computer having unique hostname(s).
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
idle flag
It is difficult to distinguish idle or busy state of a process waiting for incoming messages with polling. This is the typical case with the parallel programs using user-level communication. The idle flag in C-Area is used for SCore-D to obtain the status of a user process. Parallel runtime libraries, such as MPICH-SCore or MPC++/ ULT set the flag appropriately. SCore-D uses this information to determine if the user parallel process is deadlocked, and output the user process activity via loadmonitor.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
LANai (processor)
A special purpose processor for controling Myrinet NIC developed by Myricom.
Linux
The most populer Unix like operating system developed by Linus Torvalds. Several companies (such as RedHat, Suse and Turbo Linux) are distributing Linux as a software package. See http://www.linux.org/.
Loadmonitor (3D)
SCore-D samples the idle flag periodically (200ms, in default configuration), and its status is output graphically via loadmonitor program running on FEP host. Currently SCore-D samples two kinds of state on ech node, one is the idle flag and the other is PMv2 messaging status.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
Mesa
A 3-D graphics library supporting OpenGL like API. See http://www.mesa3d.org/.
Messageboard
Messageboard is a server to lock cluster hosts.
MTTL
MTTL stands for Multi-Thread Template Library. It is a C++ template library for multi-thread programming on distributed memory clusters. MTTL is a part of MPC++.
MPC++
A multi-threaded C++. MPC++ stands for Multi-Thread, Meta-Programming or Massively Parallel C++. MPC++ has two unique features. One is the multi-thread feature which is implemented using C++ template, called MTTL. And the other is meta-level architecture. MPC++ meta-level-architecture is still under development, due to the complexity of the C++ syntax.
MPI
MPI stands for Message Passing Interface. See http://www-unix.mcs.anl.gov/mpi/.
MPICH
A reference implementation of MPI developed by Argonne National Laboratory. See http://www-unix.mcs.anl.gov/mpi/mpich/.
MPICH-SCore
An MPICH porting for SCore Cluster System Software.
Myricom
See http://www.myri.com/.
Myrinet
A gigabit-per-second class network developed by Myricom.
(PM) Myrinet
PM/Myrinet is a PM device using Myrinet NIC. Since PM/Myrinet was the first PM device among the others, PM/Myrinet is tuned very well, stable, and providing the largest optional operations. PM/Myrinet is a user-level communication.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
network preemption
In user-level communication, user programs are allowed to access hardware to obtain higher performance. In a multi-programming (multi-user) environment, however, accessing hardware directly should be prohibited, or, hardware status must be saved and restored when switching processes. SCore-D and PMv2 cooperate for the latter processing. This procedure is called network preemption in SCore.
network interleaving
SCore-D supports interleaving two or more networks to obtain higher bisection bandwidth.
network trunking
See Ethernet trunking.
NFS
The most widely used distributed file system.
NIC
NIC stands for Network Interface Card. In general, NIC is a interface board plugged in an I/O bus (e.g. PCI) slot.
node
An entity to communicate with. A node is distinguished with the others by a node number (a positive integer). In MPI, the node number is called "rank."
NOW
NOW stands for Network Of Workstations.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
official (host) name
An official name is a hostname obtained by the gethostbyname function in glibc. For more details, see the gethostbyname(3) man page.
Omni
An OpenMP compiler developed at RWCP. Omni supports not only SMP parallel programming environment, but also distributed memory parallel programming environment.
OpenGL
OpenGL is a widely used 3-D graphics library. See http://www.opengl.org/.
OpenMP
OpenMP is an SMP programming environment. See http://www.openmp.org/.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
page coloring
If a processor has a set-associative cache, memory pages having the same address bit pattern are cached in the same cache entry, and if the entry overflows then cache miss ratio increases. Page coloring is a technique for avoiding this situation. Memory pages are colored according to the bit pattern, and allocate different colored pages if possible. In SCore Cluster System Software, a kernel patch for page coloring is included.
parallel job
In SCore Cluster System Software, a parallel job is a set of parallel processes. However, in the parallel job scheduling research community, a parallel job means a set of processes doing a parallel computation.
parallel process
A parallel process is a set of Unix processes communicating each other for a parallel computation.
PBS
PBS stands for Protable Batch System. PBS is a bacth job scheduling system, originally developed by NASA. See http://pbs.mrj.com.
PCC
PCC stands for PC Cluster.
pin-down cache
PMv2's pin-down cache is a kind of cache holding locked mmemory pages to reduce the overhead of memory page locking. See also remote memory access.
PM
A low level communication library and a device driver dedicated for Myrinet. PM was used in SCore Cluster System Software Version 2.X. See also PMv2.
PMv2[pee-em-vee-two]
PMv2 is the name of the API for low-level, high-performance communication library supporintg various network devices. PMv2 is the successor of PM. PM stands for nothing. There is a joke for the origin of PM. "PM library was written only in the afternoon." See also PM device.
PM/Composite
See (PM) Composite.
PM/Ethernet
See (PM) Ethernet.
PM/Myrinet
See (PM) Myrinet.
PM/Shmem
See (PM) Shmem.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
RedHat Linux
A Linux distribution. See http://www.redhat.com/.
(PM) remote memory access
PMv2 supprts several ways for zero-copy communication. In PMv2, the communication using zero-copy communication technique is called remote memory access. Since zero-copy communication uses NIC DMA, memory pages must belocked before DMA, so that the physical address of the memory region sit on a same logical address. Because most of modern OS, such as Linux, swaps memory pages between physical memory and disk. The locking and unlocking memory pages must be done by an OS, and they are costly operations. To reduce the memory locking overhead, PMv2 has a kind of cache for locked memory pages, called pin-down cache.
RPM
RPM stands for Red hat Package Manager. See rpm(8) Unix man page or http://www.rpm.org/.
RWCP
RWCP stands for Real World Computing Partnership (Project). RWCP is a ten year project funded by Ministry of International Trade and Industry (MITI) of Japan, starting from 1992.
In despite of the name, none of the RWCP researchers uses FORTRAN nor COBOL, and wears tie in their office (See also Real World in The New Hacker's Dictionary). Actually the name in Japanese means "New Information Processing Technology", however, it is never used because its abbreviation is "NIPT."
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
SCASH [es-cache]
Yet another software distributed shared memory system in SCore Cluster System. SCASH is also a runtime library for Omni enabling shared memory programming on distributed memory clusters.
SCore [es-core]
SCore stands for System CORE. At first, SCore was the name of parallel operating system, but now it is the name of a cluster system software.
SCore-D [es-core-dee]
The name of SCore-D used be a user-level (daemon) implementation of a parallel operating system. Now it is the name of a cluster operating system.
scored (scout command)
scored scout command is to run SCore-D in a multi-user mode.
Scoreboard
A database program holding all the information to form a cluster or a cluster of clusters.
scorehosts.db
The default database file name for scoreboard.
Scout
Scout provides a remote shell execution environment in which Unix commands are executed in SIMD parallel way or SCore-D runs.
scout command
Scout command is a special command effective only within a scout environment. Currently, scout, scrun, and scored are the scout commands. When scout is invoked outside of a scout environment, it is a normal user command to create a scout environment. When scout is invoked within a scout environment as a scout command, it checks if the current scout environment is correctly operating or not.
scout environment
Scout environment is a virtualized (sub)cluster created by the scout command. In a scout environment, users can use local host and cluster hosts in a transparent way. Not to interference with the other users, scout locks cluster hosts via messageboard.
scrun (command)
scrun command is to run a user parallel program on a cluster. When scrun command is invoked inside of a scout environment without scored option, as a scout command, it executes user parallel program on SCore-D running in a single-user mode. When scrun command is invoked with scored option, regardless of the scout environment where scrun is invoked, scrun submit a parallel job to the SCore-D specified with the scored option running in a multi-user mode. In both cases, the invoked scrun process is called a FEP process.
server host
A host serving for a cluster. Usually, the messageboard server, and the scoreboard server are running on the server host.
shadow process
In general, when a user process terminates, the memory pages for the process are reclaimed. Suppose that a user program is using zero-copy communication, and a user process terminates while NIC is copying packets into the process's address space via DMA. The memory pages are reclaimed and reused, but NIC DMA is still working. Eventually reused memory contents might be clobbered by the DMA. In SCore, to avoid this situation, SCore runtime forks a process via Linux clone(2) systemcall, so that the memory pages are not reclaimed until entire parallel process terminates. This cloned process is called shadow process.
(PM) Shmem
PM/Shmem is a PM device. It does not use any NIC, but it uses a memory-mapped segement shared by the processes in a SMP host to communicate with the other process. There is a special Unix device associated with PM/Shmem for enabling less-copy communication to get higher communication performance.
SMP
SMP stands for Symmetric Multi-Processor. In scorehosts.db file of SCore, each host record must have a smp attribute. The value of the smp attribute must be the number of processor in a host.
SuSE Linux
A Linux distribution. See http://www.suse.com/.
(SCore-D) systemcall
As most operating systems have systemcalls, SCore-D cluster operating system has several systemcalls.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
Time Slice
An interval of process swithcing. In SCore-D, time slice is an interval of gang scheduling.
(Ethernet) Trunking
(Ethernet) trunking is a technology for combining two or more Ethernet to get higher bandwidth. PM/Etherner supprts (Ethernet) trunking.
TSSS
TSSS stands for Time Space Sharing Scheduling. SCore-D packs parallel jobs in a binary buddy way, and the jobs are scheduled in space sharing and time sharing (gang scheduling) simultaneously.
Turbo Linux
A Linux distribution. See http://www.turbolinux.com/.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
ULT
ULT stands for User-Level Thread or Ultra Light Thread. ULT is a multi-thread runtime library for MTTL. SCore-D is written in MTTL using ULT.
user-level communication
Nowadays, NICs are getting faster and faster, and communicatin latency can be less than 10 micro-seconds. And communication software becomes the bottleneck. User-level communication is developed as a technique to reduce the software overhead. In user-level communication, user program is allowed to access NIC hardware directly, not by some systemcalls. Incoming messages are polled, and interrupts are disabled. In this way, neither systemcalls nor interrupts are used in user-level communication, because they are costly operation in recent high speed RISC processors.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
WSC
WSC stands for WorkStation Cluster.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
zero-copy communication
Copying a memory region in an I/O used be a costly operation. To get higher communication performance, copying has been considered harmful. To avoid message copying, DMA of a NIC is used. This technique is called zero-copy communication. In PMv2, several zero-copy communication functions are supported as remote memory access functions. However, this technique is not always effective. Some processors nowadays have higher bandwidth than DMA. Further, setup time of DMA engine is not negligible. It should be noted that zero-copy communication may not contribute to the performance of applications.
(SCore) zombie process
A process of a user parallel process keeps alive even when a SCore-D terminates. This situation may happen when SCore-D crashes for some reasons. In Unix, zombie process almost nothing harmful. SCore zombie process, however, just like a zombie in a horror movie does, sabotages SCore-D, because it keeps holding some PM device.

PCCC logo PC Cluster Consortium