NEWSFEEDS(5)NEWSFEEDS(5)NAMEnewsfeeds - determine where Usenet articles get sent
DESCRIPTION
The file <pathetc in inn.conf>/newsfeeds specifies how
incoming articles should be distributed to other sites.
It is parsed by the InterNetNews server innd(8) when it
starts up, or when directed to by ctlinnd(8).
The file is interpreted as a set of lines according to the
following rules. If a line ends with a backslash, then
the backslash, the newline, and any whitespace at the
start of the next line is deleted. This is repeated until
the entire ``logical'' line is collected. If the logical
line is blank, or starts with a number sign (``#''), it is
ignored.
All other lines are interpreted as feed entries. An entry
should consist of four colon-separated fields; two of the
fields may have optional sub-fields, marked off by a
slash. Fields or sub-fields that take multiple parameters
should be separated by a comma. Extra whitespace can
cause problems. Except for the site names, case is sig-
nificant. The format of an entry is:
sitename[/exclude,exclude,...]\
:pattern,pattern...[/distrib,distrib...]\
:flag,flag...\
:param
Each field is described below.
The sitename is the name of the site to which a news arti-
cle can be sent. It is used for writing log entries and
for determining if an article should be forwarded to a
site. If sitename already appears in the article's Path
header, then the article will not be sent to the site.
The name is usually whatever the remote site uses to iden-
tify itself in the Path line, but can be almost any word
that makes sense; special local entries (such as archivers
or gateways) should probably end with an exclamation point
to make sure that they do not have the same name as any
real site. For example, ``gateway'' is an obvious name
for the local entry that forwards articles out to a mail-
ing list. If a site with the name ``gateway'' posts an
article, when the local site receives the article it will
see the name in the Path and not send the article to its
own ``gateway'' entry. See also the description of the
``Ap'' flag, below. If an entry has an exclusion sub-
field, then the article will not be sent to that site if
any of the names specified as excludes appear in the Path
header. The same sitename can be used more than once --
the appropriate action will be taken for each entry that
should receive the article, regardless of the name --
although this is recommended only for program feeds to
avoid confusion. Case is not significant in site names.
1
NEWSFEEDS(5)NEWSFEEDS(5)
The patterns specify which groups to send to the site and
are interpreted to build a ``subscription list'' for the
site. The default subscription is to get all groups. The
patterns in the field are wildmat(3)-style patterns, and
are matched in order against the list of newsgroups that
the local site receives. If the first character of a pat-
tern is an exclamation mark, then any groups matching the
pattern are removed from the subscription, otherwise any
matching groups are added. For example, to receive all
``comp'' groups, but only comp.sources.unix within the
sources newsgroups, the following set of patterns can be
used:
comp.*,!comp.sources.*,comp.sources.unix
There are three things to note about this example. The
first is that the trailing ``.*'' is required. The second
is that, again, the result of the last match is the most
important. The third is that ``comp.sources.*'' could be
written as ``comp.sources*'' but this would not have the
same effect if there were a ``comp.sources-only'' group.
There is also a way to subscribe to a newsgroup nega-
tively. That is to say, do not send this group even if
the article is cross-posted to a subscribed newsgroup. If
the first character of a pattern is an atsign ``@'', it
means that any article posted to a group matching the pat-
tern will not be sent even though the article may be
cross-posted to a group which is subscribed. The same
rules of precedence apply in that the last match is the
one which counts. For example, if you want to prevent all
articles posted to any "alt.binaries.warez" group from
being propagated even if it is cross-posted to another
"alt" group or any other group for that matter, then the
following set of patterns can be used:
alt.*,@alt.binaries.warez.*,misc.*
If you reverse the alt.* and alt.binaries.warez.* pat-
terns, it would nullify the atsign because the result of
the last match is the one that counts. Using the above
example, if an article is posted to one or more of the
alt.binaries.warez.* groups and is cross-posted to
misc.test, then the article is not sent.
See innd(8) for details on the propagation of control mes-
sages.
A subscription can be further modified by specifying
``distributions'' that the site should or should not
receive. The default is to send all articles to all sites
that subscribe to any of the groups where it has been
posted , but if an article has a Distribution header and
any distribs are specified, then they are checked accord-
ing to the following rules:
1. If the Distribution header matches any of the val-
ues in the sub-field, then the article is sent.
2
NEWSFEEDS(5)NEWSFEEDS(5)
2. If a distrib starts with an exclamation point, and
it matches the Distribution header, then the arti-
cle is not sent.
3. If Distribution header does not match any distrib
in the site's entry, and no negations were used,
then the article is not sent.
4. If Distribution header does not match any distrib
in the site's entry, and any distrib started with
an exclamation point, then the article is sent.
If an article has more than one distribution specified,
then each one is according to the above rules. If any of
the specified distributions indicate that the article
should be sent, it is; if none do, it is not sent -- the
rules are used as a ``logical or.'' It is almost defi-
nitely a mistake to have a single feed that specifies dis-
tributions that start with an exclamation point along with
some that don't.
Distributions are text words, not patterns; entries like
``*'' or ``all'' have no special meaning.
The flags parameter specifies miscellaneous parameters.
They may be specified in any order; flags that take values
should have the value immediately after the flag letter
with no whitespace. The valid flags are:
<size An article will only be sent to the site if it is
less than size bytes long. The default is no
limit.
>size An article will only be sent to the site if it is
greater than size bytes long. The default is no
limit.
Achecks
An article will only be sent to the site if it
meets the requirements specified in the checks,
which should be chosen from the following set:
c Exclude all kinds of control messages
C Only include all kinds of control messages
d Distribution header required
e Only include message whose newsgroups in
Newsgroups header all exist in active
o Overview data is created if storage
api is turned on
p Do not check Path header for the sitename before
propagating (the exclusions are still checked).
If both ``c'' and ``C'' are specified simultane-
ously, the last specified one is adopted.
3
NEWSFEEDS(5)NEWSFEEDS(5)
Bhigh/low
If a site is being fed by a file, channel, or
exploder (see below), the server will normally
start trying to write the information as soon as
possible. Providing a buffer may give better sys-
tem performance and help smooth out overall load if
a large batch of news comes in. The value of the
this flag should be two numbers separated by a
slash. The first specifies the point at which the
server can start draining the feed's I/O buffer,
and the second specifies when to stop writing and
begin buffering again; the units are bytes. The
default is to do no buffering, sending output as
soon as it is possible to do so.
Fname This flag specifies the name of the file that
should be used if it is necessary to begin spooling
for the site (see below). If name is not an abso-
lute pathname, it is taken to be relative to
<pathoutgoing in inn.conf>. Then, if the destina-
tion is a directory, the file togo in that direc-
tory will be used as filename.
Gcount If this flag is specified, an article will only be
sent to the site if it is posted to no more than
count newsgroups.
Hcount If this flag is specified, an article will only be
sent to the site if it has count or fewer sites in
its Path line. This flag should only be used as a
rough guide because of the loose interpretation of
the Path header; some sites put the poster's name
in the header, and some sites that might logically
be considered to be one hop become two because they
put the posting workstation's name in the header.
The default value for count is one.
Isize The flag specifies the size of the internal buffer
for a file feed. If there are more file feeds than
allowed by the system, they will be buffered inter-
nally in least-recently-used order. If the inter-
nal buffer grows bigger then size bytes, however,
the data will be written out to the appropriate
file. The default value is <SITE_BUFFER_SIZE in
config.data> bytes (typically (16 * 1024 ) .)
Nmodifiers
The newsgroups that a site receives are modified
according to the modifiers, which should be chosen
from the following set:
m Only moderated groups
u Only unmoderated groups
4
NEWSFEEDS(5)NEWSFEEDS(5)
Ppriority
The nice priority that this channel or program feed
should receive. This should be a positive number
between 0 and 20, and is the priority that the new
process will run with. This can be used to raise
the priority to normal if you are using the
nicekids inn.conf variable.
OOriginator
If this is used then articles sent to this feed
must contain a X-Trace header and the first field
in the header must match the parameter used with
this flag. Originator can be a wildmat(3)-style
pattern. One use of this is to restrict the feed
to locally generated posts.
Ssize If the amount of data queued for the site gets to
be larger than size bytes, then the server will
switch to spooling, appending to a file specified
by the ``F'' flag, or <pathoutgoing in
inn.conf>/sitename if the ``F'' flag is not speci-
fied. Spooling usually happens only for channel or
exploder feeds.
Ttype This flag specifies the type of feed for the site.
Type should be a letter chosen from the following
set:
c Channel
f File
l Log entry only
m Funnel (multiple entries feed into one)
p Program
x Exploder
Each feed is described below in the section on feed
types. The default is Tf.
Witems If a site is fed by file, channel, or exploder,
this flag controls what information is written. If
a site is fed by a program, only the asterisk
(``*'') has any effect. The items should be chosen
from the following set:
b Size of the article in bytes
f Article's full pathname
g The newsgroup the article is in;
if cross-posted, then the first of the groups this
site gets
h Article's Message-ID hash key
m Article's Message-ID
n Article's pathname relative to the spool directory or
token of the article if storage api is turned on
p The time the article was posted as seconds since epoch.
s The site that fed the article to the server; from the
Path: header or the IP address of the site that sent
the article depending on the ``logipaddr'' field in
5
NEWSFEEDS(5)NEWSFEEDS(5)inn.conf(8)
t Time article was received as seconds since epoch
* Names of the appropriate funnel entries;
or all sites that get the article
D Value of the Distribution header;
? if none present
H All headers
N Value of the Newsgroups header
P Article's Path header
O Overview data
R Information needed for replication
More than one letter can be used; the entries will
be separated by a space, and written in the order
in which they are specified. The default is Wn.
The ``H'' and ``O'' items are intended for use by
programs that create news overview databases. If
``H'' is present, then the all the article's head-
ers are written followed by a blank line. An Xref
header (even if one does not appear in the filed
article) and a Bytes header, specifying the arti-
cle's size (see ``b'' description below for clari-
fying size meaning), will also be part of the head-
ers. If used, this should be the only item in the
list; if preceded by other items, however, a new-
line will be written before the headers. The
``hR'' (for storage api) or ``O'' (for non-storage
api) generates input to the overchan(8) program.
It, too, should be the only item in the list.
The asterisk has special meaning. It expands to a
space-separated list of all sites that received the
current article. If the site is the target of a
funnel however (i.e., it is named by other sites
which have a ``Tm'' flag), then the asterisk
expands to the names of the funnel feeds that
received the article. If the site is fed by a pro-
gram, then an asterisk in the param field will be
expanded into the list of funnel feeds that
received the article. A site fed by a program can-
not get the site list unless it is the target of
other ``Tm'' feeds.
If storage api is turned on, the ``b'' and ``f''
items differs from non-storage api slightly. The
``b'' item shows the size of wire formatted size of
the article. The ``f'' item shows the token of the
article, and this is equivalent to the ``n'' item.
The interpretation of the param field depends on the type
of feed, and is explained in more detail below in the sec-
tion on feed types. It can be omitted.
The site named ME is special. There must be exactly one
6
NEWSFEEDS(5)NEWSFEEDS(5)
such entry, and it should be the first entry in the file.
If the ME entry has an exclusion sub-field, then the arti-
cles are rejected if any of the names specified as
excludes appear in the Path header. If the ME entry has a
subscription list, then that list is automatically
prepended to the subscription list of all other entries.
For example, ``*,!control,!junk,!foo.*'' can be used to
set up the initial subscription list for all feeds so that
local postings are not propagated unless ``foo.*'' explic-
itly appears in the site's subscription list. Note that
most subscriptions should have ``!junk,!control'' in their
pattern list; see the discussion of ``control messages''
in innd(8). (Unlike other news software, it does not
affect what groups are received; that is done by the
active(5) file.)
If the ME entry has a distribution subfield, then only
articles that match the distribution list are accepted;
all other articles are rejected. A commercial news
server, for example, might have ``/!local'' to reject
local postings from other, misconfigured, sites.
FEED TYPES
Innd provides four basic types of feeds: log, file, pro-
gram, and channel. An exploder is a special type of chan-
nel. In addition, several entries can feed into the same
feed; these are funnel feeds, that refer to an entry that
is one of the other types. Note that the term ``feed'' is
technically a misnomer, since the server does not transfer
articles, but reports that an article should be sent to
the site.
The simplest feed is one that is fed by a log entry.
Other than a mention in the news logfile,
<pathlogininn.conf>/news, no data is ever written out.
This is equivalent to a ``Tf'' entry writing to /dev/null
except that no file is opened.
A site fed by a file is the next simplest type of feed.
When the site should receive an article, one line is writ-
ten to the file named by the param field. If param is not
an absolute pathname, it is taken to be relative to
<pathoutgoing in inn.conf>. If empty, the filename
defaults to <pathoutgoing in inn.conf>/sitename. This
name should be unique.
When a site fed by a file is flushed (see ctlinnd), the
following steps are performed. The script doing the flush
should have first renamed the file. The server tries to
write out any buffered data, and then closes the file.
The renamed file is now available for use. The server
will then re-open the original file, which will now get
created.
7
NEWSFEEDS(5)NEWSFEEDS(5)
A site fed by a program has a process spawned for every
article that the site receives. The param field must be a
sprintf(3) format string that may have a single %s parame-
ter, which will be given a pathname for the article rela-
tive to the news spool directory or a token. If the name
is not a token, the full path name may be obtained by pre-
fixing the %s in the param field by the news spool direc-
tory prefix. Standard input will be set to the article,
unless the article name is a token. Standard output and
error will be set to the error log (
<pathlogininn.conf>/errlog. The process will run with the
user and group ID of the <pathrun in inn.conf> directory.
Innd will try to avoid spawning a shell if the command has
no shell meta-characters; this feature can be defeated by
appending a semi-colon to the end of the command. The
full pathname of the program to be run must be specified;
for security, PATH environment is not searched.
If the entry is the target of a funnel, and if the ``W*''
flag is used, then a single asterisk may be used in the
param field where it will be replaced by the names of the
sites that fed into the funnel. If the entry is not a
funnel, or if the ``W*'' flag is not used, then the aster-
isk has no special meaning.
Flushing a site fed by a program does no action.
When a site is fed by a channel or exploder, the param
field names the process to start. Again, the full path-
name of the process must be given. When the site is to
receive an article, the process receives a line on its
standard input telling it about the article. Standard
output and error, and the user and group ID of the all
sub-process are set as for a program feed, above. If the
process exits, it will be restarted. If the process can-
not be started, the server will spool input to a file
named <pathoutgoing in inn.conf>/sitename. It will then
try to start the process some time later.
When a site fed by a channel or exploder is flushed, the
server closes down its end of the pipe. Any pending data
that has not been written will be spooled; see the
description of the ``S'' flag, above. No signal is sent;
it is up to the program to notice EOF on its standard
input and exit. The server then starts a new process.
Exploders are a superset of channel feeds. In addition to
channel behavior, exploders can be sent command lines.
These lines start with an exclamation point, and their
interpretation is up to the exploder. The following mes-
sages are generated automatically by the server:
newgroup group
rmgroup group
flush
8
NEWSFEEDS(5)NEWSFEEDS(5)
flush site
These messages are sent when the ctlinnd command of the
same name is received by the server. In addition, the
``send'' command can be used to send an arbitrary command
line to the exploder child-process. The primary exploder
is buffchan(8).
Funnel feeds provide a way of merging several site entries
into a single output stream. For a site feeding into a
funnel, the param field names the actual entry that does
the feeding.
EXAMPLES
## Initial subscription list and our distributions.
ME:*,!junk,!foo.*/world,usa,na,ne,foo,ddn,gnu,inet\
::
## Feed all moderated source postings to an archiver
source-archive!:!*,*sources*,!*wanted*,!*.d\
:Tc,Wn:<PREFIX specified with --prefix at configure>/bin/archive -f -i \
/usr/spool/news.archive/INDEX
## Watch for big postings
watcher!:*\
:Tc,Wbnm\
:exec awk '$1 > 1000000 { print "BIG", $2, $3 }' >/dev/console
## A UUCP feed, where we try to keep the "batching" between 4 and 1K.
ihnp4:/world,usa,na,ddn,gnu\
:Tf,Wnb,B4096/1024:
## Usenet as mail; note ! in funnel name to avoid Path conflicts.
## Can't use ! in "fred" since it would like look a UUCP address.
fred:!*,comp.sources.unix,comp.sources.bugs\
:Tm:mailer!
barney@bar.com:!*,news.software.nntp,comp.sources.bugs\
:Tm:mailer!
mailer!:!*\
:W*,Tp:/usr/ucb/Mail -s "News article" *
## NNTP feeds fed off-line via nntpsend or equivalent.
feed1::Tf,Wnm:feed1.domain.name
peer.foo.com:foo.*:Tf,Wnm:peer.foo.com
## Real-time transmission.
mit.edu:/world,usa,na,ne,ddn,gnu,inet\
:Tc,Wnm:<PREFIX specified with --prefix at configure>/bin/nntplink -i stdin mit.edu
## Two sites feeding into a hypothetical NNTP fan-out program:
nic.near.net:\
:Tm:nntpfunnel1
uunet.uu.net/uunet:!ne.*/world,usa,na,foo,ddn,gnu,inet\
:Tm:nntpfunnel1
nntpfunnel1:!*\
:Tc,Wmn*:<PREFIX specified with --prefix at configure>/bin/nntpfanout
## A UUCP site that wants comp.* and moderated soc groups
uucpsite!comp:!*,comp.*/world,usa,na,gnu\
:Tm:uucpsite
uucpsite!soc:!*,soc.*/world,usa,na,gnu\
:Tm,Nm:uucpsite
uucpsite:!*\
9
NEWSFEEDS(5)NEWSFEEDS(5)
:Tf,Wnb:/usr/spool/batch/uucpsite
The last two sets of entries show how funnel feeds can be
used. For example, the nntpfanout program would receive
lines like the following on its standard input:
<123@litchi.foo.com> comp/sources/unix/888 nic.near.net uunet.uu.net
<124@litchi.foo.com> ne/general/1003 nic.near.net
Since the UUCP funnel is only destined for one site, the
asterisk is not needed and entries like the following will
be written into the file:
<qwe#37x@snark.uu.net> comp/society/folklore/3
<123@litchi.foo.com> comp/sources/unix/888
HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNet-
News. This is revision 1.1.2.2, dated 1999/09/22.
SEE ALSOactive(5), buffchan(8), ctlinnd(8), inn.conf(5), innd(8),
wildmat(3).
10
