CRASH(8)CRASH(8)NAMEcrash - UNIX system failures
DESCRIPTION
This section explains what happens when the system crashes and (very
briefly) how to analyze crash dumps.
When the system crashes voluntarily it prints a message of the form
panic: why i gave up the ghost
on the console, takes a dump on a mass storage peripheral, and then
invokes an automatic reboot procedure as described in reboot(8). (If
auto-reboot is disabled on the front panel of the machine the system
will simply halt at this point.) Unless some unexpected inconsistency
is encountered in the state of the file systems due to hardware or
software failure, the system will then resume multi-user operations.
The system has a large number of internal consistency checks; if one of
these fails, then it will panic with a very short message indicating
which one failed. In many instances, this will be the name of the rou‐
tine which detected the error, or a two-word description of the incon‐
sistency. A full understanding of most panic messages requires perusal
of the source code for the system.
The most common cause of system failures is hardware failure, which can
reflect itself in different ways. Here are the messages which are most
likely, with some hints as to causes. Left unstated in all cases is
the possibility that hardware or software error produced the message in
some unexpected way.
iinit This cryptic panic message results from a failure to mount the
root filesystem during the bootstrap process. Either the root
filesystem has been corrupted, or the system is attempting to
use the wrong device as root filesystem. Usually, an alternate
copy of the system binary or an alternate root filesystem can be
used to bring up the system to investigate.
Can't exec /sbin/init
This is not a panic message, as reboots are likely to be futile.
Late in the bootstrap procedure, the system was unable to locate
and execute the initialization process, init(8). The root
filesystem is incorrect or has been corrupted, or the mode or
type of /sbin/init forbids execution.
IO err in push
hard IO err in swap
The system encountered an error trying to write to the paging
device or an error in reading critical information from a disk
drive. The offending disk should be fixed if it is broken or
unreliable.
realloccg: bad optim
ialloc: dup alloc
alloccgblk: cyl groups corrupted
ialloccg: map corrupted
free: freeing free block
free: freeing free frag
ifree: freeing free inode
alloccg: map corrupted
These panic messages are among those that may be produced when
filesystem inconsistencies are detected. The problem generally
results from a failure to repair damaged filesystems after a
crash, hardware failures, or other condition that should not
normally occur. A filesystem check will normally correct the
problem.
timeout table overflow
This really shouldn't be a panic, but until the data structure
involved is made to be extensible, running out of entries causes
a crash. If this happens, make the timeout table bigger.
KSP not valid
SBI fault
CHM? in kernel
These indicate either a serious bug in the system or, more
often, a glitch or failing hardware. If SBI faults recur, check
out the hardware or call field service. If the other faults
recur, there is likely a bug somewhere in the system, although
these can be caused by a flakey processor. Run processor micro‐
diagnostics.
machine check %x:
description
machine dependent machine-check information
Machine checks are different on each type of CPU. Most of the
internal processor registers are saved at the time of the fault
and are printed on the console. For most processors, there is
one line that summarizes the type of machine check. Often, the
nature of the problem is apparent from this messaage and/or the
contents of key registers. The VAX Hardware Handbook should be
consulted, and, if necessary, your friendly field service people
should be informed of the problem.
trap type %d, code=%x, pc=%x
A unexpected trap has occurred within the system; the trap types
are:
0 reserved addressing fault
1 privileged instruction fault
2 reserved operand fault
3 bpt instruction fault
4 xfc instruction fault
5 system call trap
6 arithmetic trap
7 ast delivery trap
8 segmentation fault
9 protection fault
10 trace trap
11 compatibility mode fault
12 page fault
13 page table fault
The favorite trap types in system crashes are trap types 8 and
9, indicating a wild reference. The code is the referenced
address, and the pc at the time of the fault is printed. These
problems tend to be easy to track down if they are kernel bugs
since the processor stops cold, but random flakiness seems to
cause this sometimes. The debugger can be used to locate the
instruction and subroutine corresponding to the PC value. If
that is insufficient to suggest the nature of the problem, more
detailed examination of the system status at the time of the
trap usually can produce an explanation.
init died
The system initialization process has exited. This is bad news,
as no new users will then be able to log in. Rebooting is the
only fix, so the system just does it right away.
out of mbufs: map full
The network has exhausted its private page map for network buf‐
fers. This usually indicates that buffers are being lost, and
rather than allow the system to slowly degrade, it reboots imme‐
diately. The map may be made larger if necessary.
That completes the list of panic types you are likely to see.
When the system crashes it writes (or at least attempts to write) an
image of memory into the back end of the dump device, usually the same
as the primary swap area. After the system is rebooted, the program
savecore(8) runs and preserves a copy of this core image and the cur‐
rent system in a specified directory for later perusal. See
savecore(8) for details.
To analyze a dump you should begin by running adb(1) with the -k flag
on the system load image and core dump. If the core image is the
result of a panic, the panic message is printed. Normally the command
``$c'' will provide a stack trace from the point of the crash and this
will provide a clue as to what went wrong. For more detail see ``Using
ADB to Debug the UNIX Kernel''.
SEE ALSOadb(1), reboot(8)
VAX 11/780 System Maintenance Guide and VAX Hardware Handbook for more
information about machine checks.
Using ADB to Debug the UNIX Kernel
4th Berkeley Distribution June 5, 1993 CRASH(8)