NSEND(2) LAM NETWORK LIBRARY NSEND(2)NAME
nsend, ntry_send, nrecv, ntry_recv - Send and receive LAM network mes‐
sages.
C SYNOPSIS
#include <net.h>
int nsend (struct nmsg *header);
int ntry_send (struct nmsg *header);
int nrecv (struct nmsg *header);
int ntry_recv (struct nmsg *header);
FORTRAN SYNOPSIS
subroutine NSND (nnode, nevent, ntype, nlength, nflags, ndata, ndsize,
nmsg, ierror)
subroutine NRCV (nevent, ntype, nlength, nflags, ndata, ndsize, nmsg,
ierror)
integer nevent, ntype, nlength, nflags, ndata(*), ndsize, ierror
<type> nmsg(*)
DESCRIPTION
The network message-passing functions add routing and packetization to
the datalink functions, dsend(2) and drecv(2). nrecv() blocks if there
is no synchronizing message to receive. nsend() will block if there is
no synchronizing receiving process or forwarding process to take its
message (see "Blocking").
ntry_send() and ntry_recv() never cause the calling process to block.
The message is either immediately transferred, or an error is immedi‐
ately returned, indicating that the process would have blocked. See
nprobe(2) for similar functionality.
Network Message Descriptor
All of the functions accept a pointer to a network message descriptor
which is an extension of the local level message descriptor used by
ksend(2) and krecv(2). The network message descriptor is defined in
<net.h>.
struct nmsg {
int nh_dl_event;
int nh_dl_link;
int nh_node;
int nh_event;
int nh_type;
int nh_length;
int nh_flags;
int nh_data[NHDSIZE];
char *nh_msg;
};
nh_dl_event
unused
This field is unchanged by nrecv() but is set to the event of
the synchronizing process (either the intended receiver or a
forwarding process) after calling nsend(). See dsend(2).
nh_dl_link
unused
This field is unchanged by nrecv() but is set to the output link
number if the message was forwarded to a datalink output process
after calling nsend(). See dsend(2).
nh_node
This field is used by nsend() to identify the remote node run‐
ning the intended receiver. It is not used by nrecv(). A
receiving process thus cannot directly specify the source node
of a message. Instead, receiving processes are "matched" to
messages by one or both of nh_event and nh_type. Two special
node identifiers are defined in <net.h>. LOCAL refers to the
local node and causes nsend() to bypass its routing step. ORI‐
GIN refers to the node from which lamboot(1) was invoked.
This field is never altered.
nh_event
An event is an arbitrary positive integer used by the LAM kernel
to synchronize processes within a node. Synchronization occurs
when two events are equal. nsend() transfers the message to the
destination node and then to the highest priority process
blocked on the event in the message's nh_event field and a
matching type (see below). Thus, the sender calling nsend()
must set nh_event to the same value as the receiver calling
nrecv().
This field is never altered.
nh_type
This field further filters messages that match on event. A mes‐
sage will be transferred to a receiver only if the nh_type
fields of the sender and receiver processes have at least one
bit set in an identical position. In other words, the bitwise
logical AND of the type fields specified by the two parties must
not equal zero. A zero value matches any other value of
nh_type.
This field remains unchanged after calling nsend(), but is set
to the sender's nh_type after calling nrecv().
nh_length
This field holds the length (in bytes) of the message to be
sent. If the sender and the receiver specify different lengths,
the lesser amount will be transferred to the receiver. Messages
longer than the maximum network packet size, defined by MAXNMS‐
GLEN in <net.h>, will be implicitly broken down into a series of
smaller messages, which will be recombined by the receiver.
Because of this packetization, one call to nsend() can introduce
many messages into the network, a possible source of confusion
during debugging.
This field remains unchanged after calling nsend(), but is set
to the minimum of the sender's and receiver's lengths after
calling nrecv().
nh_flags
This field is normally set to 0. When the NOBUF flag (defined
in <net.h>) is set in nh_flags buffers will not be used. Flags
used to assure that the data representation is correct for the
receiving node are discussed under "Data Representation".
This field is never altered.
nh_data
This field is a convenient data pouch within the network message
descriptor. Its array size is NHDSIZE words, which is defined
in <net.h> and is set to 8. It can be used for sending short
messages (in which case nh_length is set to 0) or for appending
control information to the message body.
After calling nrecv() the nh_data field is overwritten with the
sender's values of the same field. The sender's nh_data will
not change.
nh_msg This field holds the address of the first byte of data to be
sent or received. The data must be stored contiguously in mem‐
ory.
This field is never altered.
Data Representation
On nodes of different architectures, data may have different represen‐
tations. For example, integers may be stored with the most significant
byte first in memory (big-endian) or with the most significant byte
last in memory (little-endian). Also, the representation of floating
point numbers may conform to the IEEE standard or may follow a vendor
specific format. All fields in the network message structure, except
the data referenced by nh_msg, are automatically converted if passed to
a node with different data representation. The nh_data field is
assumed to hold all integers.
The nh_flags field of the message structure can be set to the following
data representation flags. Each flag assumes a data type, and will
make the appropriate change in the data representation of the given
field. They will have no effect if data conversion is not needed.
DINT4DATA nh_data holds 8 32-bit integers (default).
DFLT4DATA nh_data holds 8 single 32-bit real numbers.
DFLT8DATA nh_data holds 4 64-bit real numbers.
DRAWDATA nh_data representation will not be changed.
DINT4MSG nh_msg points to 32-bit integers.
DFLT4MSG nh_msg points to 32-bit real numbers.
DFLT8MSG nh_msg points to 64-bit real numbers.
DRAWMSG nh_msg representation will not be changed (default).
If nh_data or nh_msg contains a mixture of data types, the user will
have to change the representation using the function suites ltot(3),
ttol(3), etc.
Example Usage
The following example passes a message between two nodes with similar
data representations, utilizing a minimum level of synchronization.
This is intended only as a summary of a simple case. Many variations
can be constructed using the detailed information given in the above
section.
/* Sender */
#include <net.h>
struct nmsg nhead;
char *msg = "Hello, world";
nhead.nh_node = 10
nhead.nh_event = 6
nhead.nh_type = 0
nhead.nh_flags = 0
nhead.nh_length = strlen(msg) + 1;
nhead.nh_msg = msg;
nsend(&nhead);
/* Receiver */
/* Assume this code is running on node 10. */
#include <net.h>
struct nmsg nhead;
char msg[16]
nhead.nh_event = 6
nhead.nh_type = 0
nhead.nh_flags = 0
nhead.nh_length = sizeof(msg);
nhead.nh_msg = msg;
nrecv(&nhead);
Blocking
A process calling nrecv() blocks until the message sent by the process
calling nsend() entirely arrives. A process calling nsend() blocks
only until its message is picked up by:
a) a local receiver calling nrecv()
b) a local buffer process
c) a local forwarding process such as a datalink
The only thing that is guaranteed by a successful return from nsend()
is that the message has entirely left the calling process.
The loose blocking behaviour of nsend() introduces a fundamental danger
of LAM message passing: a sender can transmit a message that may never
be received due to programming error or deadlock. This message will
never be dropped or timed out. Some LAM process will always be stuck
with it, waiting for a synchronizing nrecv() that may never happen. If
that unfortunate process is a buffer, it can be located by the user and
swept clean (see sweep(1)). However, if the process is a link propri‐
etor, the link is henceforth plugged and useless.
Besides the legitimate buffer process, datalink processes can each hold
one or more messages. NOBUF does not affect these implicit buffers.
ERRORS
Errors return LAMERROR and set errno appropriately. The lam_perror()
and lam_errorstr() functions can be used to retrieve the error string
associated with errno.
Some common errno values include:
EWOULDBLOCK ntry_send() or ntry_recv() failed because the message
could not be sent or received, respectively. A call to
nsend() or nrecv() would have blocked.
ENOTATTACHED The calling program is not attached to the LAM run time
environment.
BUGS
Multi-packet messages can inter-mingle packets if sent to the same
node, event and type. The solution for this type of communication
structure is to use tsend(2) and trecv(2).
SEE ALSOdsend(2), nprobe(2), tsend(2)LAM 7.1.2 March, 2006 NSEND(2)
