Fix fstab When Filesystems Fail to Mount at Boot

If a filesystem referenced in /etc/fstab is not mounting during the boot process, this can halt the boot process and invoke emergency maintenance mode. There are some common things that can be done to correct this type of failure.

Using the "nofail" option in /etc/fstab

This option works well when you have multiple filesystems besides the root filesystem. For example you might want to use the nofail option for a block device dedicated to backups or an NFS share.

When this option is used the boot process will not be interrupted even if the filesystem fails to mount properly. This cane be more desirable than the boot process stopping. Especially if you do not have console access to your server.

Here's a usage example:

nfs.linuxbucket.local:/backups	/mnt/backups	nfs	defaults,nofail	0 0

Commenting Out fstab Entries

Another thing people often do is simply comment out the troubled filesystem so that the server will boot. This allows further troubleshooting to take place once the server is booted up. All that needs to be done is to add a # at the beginning of the entry.

Example:

#nfs.linuxbucket.local:/backups	/mnt/backups	nfs	defaults	0 0

Repairing EXT and XFS Filesystems

Probably the most common reason a filesystem would fail to mount is due to corruption. Depending on the extent of the damage to the filesystem, you might be able to repair it using the following commands.

First you will need to make sure the filesystem is unmounted using the umount command. If you're working with a root filesystem you'll need to attach the block device to another computer or boot from a liveCD, etc. In most cases it is also a good idea to take a snapshot of the block device if possible incase the repair process actually causes further damage to the filesystem.

Once you've made sure it's unmounted and you have a backup you can perform the following:

# List block devices and their file path
$ lsblk -p
NAME         MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
/dev/xvda    202:0    0   8G  0 disk
└─/dev/xvda1 202:1    0   8G  0 part /
/dev/xvdf    202:80   0   8G  0 disk
└─/dev/xvdf1 202:81   0   8G  0 part

# In this example I'm working with /dev/xvdf1. Check the filesystem type with file $ sudo file -sL /dev/xvda1 /dev/xvda1: Linux rev 1.0 ext4 filesystem data, UUID=1afa51d9-105a-4c14-8f32-8b8b424b1c81 (needs journal recovery) (extents) (large files) (huge files) # Here is an example of an XFS file system on another server $ sudo file -sL /dev/mapper/centos-root /dev/mapper/centos-root: SGI XFS filesystem data (blksz 4096, inosz 256, v2 dirs) # If it's either ext[2,3,4] or XFS you can use one of the following to attempt a repair. $ sudo fsck -r /dev/xvdf1 $ xfs_repair /dev/xvdf1

With ext filesystems they usually automatically perform a fsck after a certain number of mounts or after a certain amount of time. This information can be found using a command like sudo dumpe2fs -h /dev/xvdf1 | grep "Next check after". For XFS filesystems the default behavior is to perform a check on a file system every time the server is booted.

Conclusions

If you are not able to repair the filesystem you may need to rely on backups to recover your data. Once the filesystem has been restored hopefully you will be able to successfully mount without further issue. The nofail option will come in handy the next time a filesystem sees failures like this.