libmpi(7) LAM OVERVIEW libmpi(7)NAMElibmpi - LAM/MPI implementation
DESCRIPTION
LAM features a full implementation of Message Passing Interface (MPI) 1
standard with many features included from the MPI-2 standard.
Compliant applications are source code portable between LAM and any
other implementation of MPI. In addition to meeting the standard in a
high quality manner, LAM offers extensive monitoring capabilities to
support debugging. Monitoring happens on two levels. LAM has the
hooks to allow a snapshot of process and message status to be taken at
any time during an application run. The status includes all aspects of
synchronization plus datatype map / signature, communicator group mem‐
bership and message contents. On the second level, the MPI library is
instrumented to produce a cumulative record of communication, which can
be visualized either at runtime or post-mortem.
LAM/MPI features the ability to change its underlying message transport
mechanism, as well as pass tunable parameters to different components
in LAM/MPI at run-time -- without the need to recompile or relink user
MPI applications. This provides great flexibility for both developers
of MPI software as well as researchers investigating MPI performance.
This modual framework is called the System Services Interface (SSI),
and is responsible for much of the back-end functionality in LAM/MPI.
Another strength of this MPI implementation is the movement of non-
blocking communication requests, including those that result from
buffered sends. This is the real challenge of implementing MPI; every‐
thing else is mostly a straight forward wrapping of an underlying com‐
munication mechanism. LAM allows messages to be buffered on the source
end in any state of progression, including partially transmitted pack‐
ets. This capability leads to great portability and robustness.
User Information
Users are strongly encouraged to read the LAM/MPI User's Guide that is
included with the LAM/MPI distribution, and is provided on the main
LAM/MPI web site (http://www.lam-mpi.org/).
Up-to-Date Information
The LAM home page can be found on the World Wide Web at:
http://www.lam-mpi.org/. It should be consulted for the most current
information about LAM, as well as updates, patches, etc.
MPI Communication
The sophisticated message advancing engine at the heart of the MPI
library uses only a handful of routines to drive the underlying commu‐
nication system. Runtime flags decide which message passing engine
module is used, so recompilation of user programs is not necessary.
The different message passing engines are commonly referred to as
"Request Progression Interface" (RPI) modules. The LAM/MPI distribu‐
tion includes multiple RPI modules; see the lamssi_rpi(7) man page for
more details.
One notable module uses LAM's network message-passing subsystem,
including its buffer daemon. In this "daemon" mode, LAM's extensive
monitoring features are fully available. Although the "daemon" mode
typically incurrs higher latency than the "native" RPI modules, appli‐
cations that can utilize latency-hiding techniques may experience
greater performance due to the daemon-mode's ability to exhibit true
asynchronous message passing.
Guaranteed Envelope Resources
Applications may fail, legitimately, on some implementations but not
others due to an escape hatch in the MPI Standard called "resource lim‐
itations". Most resources are managed locally and it is easy for an
implementation to provide a helpful error code and/or message when a
resource is exhausted. Buffer space for message envelopes is often a
remote resource (as in LAM) which is difficult to manage. An overflow
may not be reported (as in some other implementations) to the process
that caused the overflow. Moreover, interpretation of the MPI guaran‐
tee on message progress may confuse the debugging of an application
that actually died on envelope overflow.
LAM has a property called "Guaranteed Envelope Resources" (GER) which
serves two purposes. It is a promise from the implementation to the
application that a minimum amount of envelope buffering will be avail‐
able to each process pair. Secondly, it ensures that the producer of
messages that overflows this resource will be throttled or cited with
an error as necessary.
A minimum GER is configured when LAM is built. The MPI library uses a
protocol to ensure GER when running in daemon mode. The default C2C
mode (TCP/IP) does not use a protocol, because process-pair protection
is provided by TCP/IP itself. Errors are only reported to the receiv‐
ing process in C2C mode. An option to mpirun(1) disables GER.
Input and Output
The MPI standard does not specify standard I/O functionality. LAM does
not interfere with the I/O capabilities of the underlying system but it
does make special provisions for remote terminal I/O using the
ANSI/POSIX routines. See mpirun(1) and tstdio(3).
LAM now includes the ROMIO distribution for MPI-2 file input and out‐
put. If ROMIO support is compiled into LAM, the functionality from
Chapter 9 of the MPI-2 standard is provided.
ROMIO has some important limitations under LAM; the User's Guide in the
LAM distribution should be consulted before writing MPI programs that
use MPI I/O.
Dynamic Processes
LAM includes an implementation of MPI-2 dynamic process creation.
LAM Extensions to MPI
Debugging Aids
LAM includes the MPI-2 functionality for naming opaque types. Support
for the Etnus TotalView parallel debugger is also provided; see the
User's Guide for more details.
Additionally, LAM provides the capability to launch non-MPI programs on
remote nodes. This includes shell scripts, debuggers, etc. As long as
an MPI program is eventually launched (as a child, grandchild, etc.),
LAM can handle executing as many intermediate programs as necessary.
This can greatly help debugging and logging of user programs.
Trace Generation
To avoid being swamped with trace data from a long running application,
LAM supplies collective operations to turn the tap on and off. See
MPIL_Trace_on(2) and MPIL_Trace_off(2).
Asynchronous Signals
LAM has an signal handling package which mirrors but does not interfere
with POSIX signal handling. An MPI extension routine delivers a signal
to a process. See MPIL_Signal(2).
SEE ALSO
Overview of Commands and Libraries
introu(1), introc(2), INTROF(2)
System Services Interface (SSI)
lamssi(7), lamssi_boot(7), lamssi_coll(7), lamssi_rpi(7)
Starting / Stopping LAM
recon(1), lamboot(1), lamhalt(1), lamnodes(1), lamwipe(1), tping(1),
lamgrow(1), lamshrink(1)
Compiling MPI Applications
mpicc(1), mpiCC(1), mpif77(1)
Running MPI Applications
mpirun(1), lamclean(1)
Running Non-MPI Applications
lamexec(1)
Monitoring MPI Applications
mpitask(1)
Unloading MPI Trace Data
lamtrace(1)
Reference Documents
"LAM/MPI Installation Guide"
included in the LAM/MPI distribution and available on
http://www.lam-mpi.org/
"LAM/MPI User's Guide"
included in the LAM/MPI distribution and available on
http://www.lam-mpi.org/
"LAM Frequently Asked Questions"
at http://www.lam-mpi.org/faq/
"MPI Primer / Developing with LAM", Ohio Supercomputer Center
"MPI: A Message-Passing Interface Standard", Message-Passing Interface
Forum, version 1.1
at http://www.mpi-forum.org/
"MPI-2: Extensions to the Message Passing Interface", Message Passing
Interface Forum, version 2.0
at http://www.mpi-forum.org/
MPI Quick Tutorials
"LAM/MPI ND User Guide / Introduction"
at http://www.lam-mpi.org/mpi/tutorials/lam/
"MPI: It's Easy to Get Started"
"MPI: Everyday Datatypes"
"MPI: Everyday Collective Communication"
Guaranteed Envelope Resources
"Robust MPI Message Delivery Through Guaranteed Resources", MPI Devel‐
oper's Conference, 1995
LAM 7.1.2 March, 2006 libmpi(7)
