AIX:Repairing Corrupt File Systems or File System Log Devices

This topic covers the use of the fsck and logform commands to repair inconsistencies or corruption in file systems and associated log devices. 

• About the fsck and logform commands
• Recovering superblock errors
• Correcting primary file systems and log

About the fsck and logform commands

The fsck command is used to detect and repair inconsistencies in the journaled file system (JFS) and the enhanced journaled file system (JFS2) structure. It is not intended to correct problems with corrupt data, but only the file system structure itself. The basic syntax is:

     fsck [options] /dev/<LVname> 

OR

     fsck [options] /<fsname>

The file system MUST be unmounted for any operation to occur. The fsck command requires the file system to be in a consistent state while performing its checks. Attempts by users to write to the file system might cause fsck to report corruption that does not exist. For this reason, any errors reported by fsck while the file system is mounted may not be relevant.

Various options for fsck exist:

-f    performs a fast check; files systems with check = true entries in /etc/filesystems are checked
-p    fixes minor problems without interaction from user
-y    gives permission to correct every problem found
-n    indicates not to correct any problems

See the man page for fsck for more information.

While the -y flag is certainly time-saving, be careful with this option. If fsck cannot read a block due to a missing disk, for example, it will ask to clear the block. If the disk is missing due to an adapter failure, for instance, you may be removing recoverable data by responding yes or by giving the fsck command explicit permission to fix everything it finds.

Examples of fsck use:

     fsck /dev/lv00

     fsck -y /data

     fsck -p /dev/lv00

The logform command formats a logical volume for use as a log device, which stores transactional information about file system metadata changes and can be used to roll back incomplete operations if the machine crashes. The logform command is destructive; it wipes out all data in the logical volume. Accidentally running this on a file system completely destroys all file system data. The logform command should only be run on CLOSED logical volumes. If a log device is open due to its use by a mounted file system, the file system should be unmounted prior to running logform against the log device.

Run the following to ensure that the log device is closed:

     lsvg -l <VGname>

Here are some examples of messages you might receive that would indicate a corrupt log device:

     failure replaying log

     media is not formatted or format is not correct

Examples of logform use:

     logform /dev/loglv00

     logform: destroy /dev/loglv00 (y)?y

---

Recovering superblock errors

If you receive one of the following errors from the fsck or mount commands, the problem may be a corrupted superblock.

fsck: Not an AIX4 file system

fsck: Not an AIXV4 file system

fsck: Not a recognized file system type

0506-342 The superblock is dirty.  Run a full fsck to fix.

mount: invalid argument

The backup superblock can be copied over the primary superblock via one of these commands:

     dd count=1 bs=4k skip=31 seek=1 if=/dev/lv00 of=/dev/lv00 (JFS)

     dd count=1 bs=4k skip=15 seek=8 if=/dev/lv00 of=/dev/lv00 (JFS2) (Version 

5 only)

     fsck -p /dev/lv00 (works for both JFS and JFS2)

Once the copying over is completed, check the integrity of the file system by issuing:

     fsck /dev/lv00

In many cases, copying the backup superblock to the primary superblock will recover the file system. If this does not work, you will have to recreate the file system and restore the data from a backup.

---

Correcting primary file systems and log

It is not possible under normal circumstances to unmount /, /usr, /tmp, and /var, and thus close /dev/hd8 (the primary rootvg log device) so they can be checked or fixed. This can only be done in maintenance mode.

1.    Boot the machine into maintenance mode, access the rootvg volume group, and start a shell prior to mounting the file systems. If you need assistance with this, contact your AIX support center.

2.    Run the following commands to fsck the primary file systems:

3.          fsck /dev/hd4

4.          fsck /dev/hd2

5.          fsck /dev/hd3

6.          fsck /dev/hd9var

7.          fsck /dev/hd1

Other fsck options as outlined previously can also be used, where appropriate.

8.    Format the default jfslog for the rootvg JFS file systems with:

9.          /usr/sbin/logform /dev/hd8

Answer y when asked if you want to destroy the log.

10.  Type exit to exit from the shell. The primary file systems will automatically mount.

11.  Shutdown and reboot with the key in normal:

12.       sync;sync;sync;shutdown -Fr

You can also run fsck on any user-created file systems in rootvg, if needed. This can typically be done in normal mode.