AIX: Creating new file systems in AIX? Don't edit /etc/filesystems by hand!

Don’t edit the /etc/filesystems file manually when creating file systems.

When you import a volume group, the importvg command will populate the /etc/filesystems file based on the logical volume minor number order (which is stored in the VGDA on the physical volume/hdisk). If someone manually edits the /etc/filesystems, then its contents will no longer match the order contained in the VGDA of the physical volume. This can become a problem the next time someone attempts to export and import a volume group. Essentially they may end up with file systems over-mounted and what appears to be the loss of data!

Here’s a quick example of the problem.

Let’s create a couple of new file systems; /fs1 and /fs1/fs2. I’ll deliberately create them in the “wrong” order.

# mklv -tjfs2 -y lv2 cgvg 1

lv2

# crfs -vjfs2 -dlv2 -Ayes -u fs -m /fs1/fs2

File system created successfully.

65328 kilobytes total disk space.

New File System size is 131072

# mklv -tjfs2 -y lv1 cgvg 1

lv1

# crfs -vjfs2 -dlv1 -Ayes -u fs -m /fs1

File system created successfully.

65328 kilobytes total disk space.

New File System size is 131072

Hmmm, lv2 appears before lv1 in the output from lsvg. The first indication of a potential problem!

# lsvg -l cgvg

cgvg:

LV NAME             TYPE       LPs     PPs     PVs  LV STATE      MOUNT POINT

lv2                 jfs2       1       1       1    closed/syncd  /fs1/fs2

loglv00             jfs2log    1       1       1    closed/syncd  N/A

lv1                 jfs2       1       1       1    closed/syncd  /fs1

Whoops! /fs1 should be mounted before /fs1/fs2!!! Doh!

# mount -t fs

# mount | tail -2

         /dev/lv2         /fs1/fs2         jfs2   Jul 19 23:07 rw,log=/dev/loglv00

         /dev/lv1         /fs1             jfs2   Jul 19 23:07 rw,log=/dev/loglv00

Data in /fs1/fs2 is now hidden and inaccessible. The /fs1 file system has over-mounted the /fs1/fs2 file system. This could look like data loss i.e. someone removed all the files from the file system.

# df -g | grep fs

/dev/lv2              -         -    -         -     -  /fs1/fs2

/dev/lv1           0.06      0.06    1%        4     1% /fs1

The file systems are listed in the wrong order in /etc/filesystems as well. Double Doh!

# tail -15 /etc/filesystems

/fs1/fs2:

        dev             = /dev/lv2

        vfs             = jfs2

        log             = /dev/loglv00

        mount           = true

        type            = fs

        account         = false

/fs1:

        dev             = /dev/lv1

        vfs             = jfs2

        log             = /dev/loglv00

        mount           = true

        type            = fs

        account         = false

No problem. I’ll just edit the /etc/filesystems file and rearrange the order. Simple, right?

# vi /etc/filesystems

/fs1:

        dev             = /dev/lv1

        vfs             = jfs2

        log             = /dev/loglv00

        mount           = true

        type            = fs

        account         = false

/fs1/fs2:

        dev             = /dev/lv2

        vfs             = jfs2

        log             = /dev/loglv00

        mount           = true

        type            = fs

        account         = false

Let’s remount the file systems in the correct order.

# umount -t fs

# mount -t fs

# df -g | grep fs

/dev/lv1           0.06      0.06    1%        5     1% /fs1

/dev/lv2           0.06      0.06    1%        4     1% /fs1/fs2

That looks better now, doesn’t it!? I’m happy now.....although, lsvg still indicates there could be a potential problem here…

# lsvg -l cgvg

cgvg:

LV NAME             TYPE       LPs     PPs     PVs  LV STATE      MOUNT POINT

lv2                 jfs2       1       1       1    open/syncd  /fs1/fs2

loglv00             jfs2log    1       1       1    open/syncd  N/A

lv1                 jfs2       1       1       1    open/syncd  /fs1

All is well, until one day someone exports the VG and re-imports it, like so:

# varyoffvg cgvg

# exportvg cgvg

# importvg -y cgvg hdisk2

cgvg

# mount -t fs

# mount | tail -2

         /dev/lv2         /fs1/fs2         jfs2   Jul 19 23:07 rw,log=/dev/loglv00

         /dev/lv1         /fs1             jfs2   Jul 19 23:07 rw,log=/dev/loglv00

Huh? What’s happened here!?  I thought I fixed this before!?

Try to avoid this situation before it becomes a problem (for you or someone else!) in the future. If you discover this issue whilst creating your new file systems, remove the file systems and recreate them in the correct order. Obviously, try to do this before you place any data in the file systems. Otherwise you may need to back up and restore the data!

# mklv -tjfs2 -y lv1 cgvg 1

lv1

# crfs -vjfs2 –dlv1 -Ayes -u fs -m /fs1

File system created successfully.

65328 kilobytes total disk space.

New File System size is 131072

# mklv -tjfs2 -y lv2 cgvg 1

lv2

# crfs -vjfs2 –dlv2 -Ayes -u fs -m /fs1/fs2

File system created successfully.

65328 kilobytes total disk space.

New File System size is 131072

You may be able to detect this problem, prior to importing a volume group, by using the lqueryvg command. Looking at the output in the “Logical” section, you might be able to ascertain a potential LV and FS mount order issue.

# lqueryvg -Atp hdisk2 | grep lv

0516-320 lqueryvg: Physical volume hdisk2 is not assigned to

        a volume group.

Logical:        00f603cd00004c000000013ff2fc1388.1   lv2 1

                00f603cd00004c000000013ff2fc1388.2   loglv00 1

                00f603cd00004c000000013ff2fc1388.3   lv1 1

Once you’ve identified the problem you can fix the issue retrospectively (once the VG is imported) by editing /etc/filesystems. Of course, this is just a temporary fix until someone exports and imports the VG again, in which case the mount order issue will occur again.

The essential message here is do NOT edit the /etc/filesystems file by hand when creating file systems.

 Creating new file systems in AIX? Don't edit /etc/filesystems by hand!