LVM

Logical Volume Manager allows Linux systems to decouple the physical layout of our disks from the filesystems. This allows us to replicate data across a series of striped physical volumes, create filesystems that are larger than the underlying disks and perform online resizes on them as we grow.

The basics

LVM thinks in terms of a few different types of objects:

• Physical Volumes (PVs) represent the underlying storage on e.g. physical disks or page blobs, containing:
  • Metadata, a snapshot of the composition of the volume groups hosted on the PV.
  • Extents, into which physical volumes are broken into for allocation to groups.
• Volume Groups (VGs) are collections of physical volumes.
• Logical Volumes (LVs) are volumes contained within a volume group, backed by one or more physical disks.

Configuration

LVM is configured in /etc/lvm/lvm.conf. For a complete, documented copy of the configuration file you can run the following command:

lvmconfig --typeconfig full --withcomments

Further information can be found in the lvm.conf(5) man page:

man 5 lvm.conf

Note that the file is made up of blocks containing key-value pairs. LVM's command line tools usually support overriding individual settings on the command line, e.g.:

sudo pvs --config 'global/use_lvmetad=0'

Troubleshooting

Duplicate PV UUIDs

LVM physical volume metadata includes a UUID value which is used to uniqely identify each LVM physical volume. This is important to ensure LVM doesn't inadvertantly write data to the wrong volume.

Errors like the following when running LVM commands (e.g. lvs, pvs) indicate that LVM can "see" two devices with the same UUID:

WARNING: Not using lvmetad because duplicate PVs were found.
WARNING: Use multipath or vgimportclone to resolve duplicate PVs?
WARNING: After duplicates are resolved, run "pvscan --cache" to enable lvmetad.
WARNING: PV 000000-0000-0000-0000-0000-0000-000000 on /dev/mapper/A-LV--A--1 was already found on /dev/drbd0.
WARNING: PV 000000-0000-0000-0000-0000-0000-000000 prefers device /dev/mapper/A-LV--A--1 because device is used by LV.

There are a few possible causes:

• Device mapper is exposing the same device with multiple names and LVM's devices/multipath_component_detection option is disabled. Try enabling multi-path component detection.
• You're using a piece of software (e.g. DRBD) to do block-level replication which exposes an additional device containing the same data. In this case the devices/filter and devices/global_filter options must be set to exclude the backing device and include only the device exposed by DRBD.
• You've attached a cloned copy or snapshot of the disks to the same server. You should either attach the disks to a different server or alter their UUIDs.

Recovering from a lost physical volume

The commands below involve hand-editing the metadata for an LVM volume group. Proceed carefully only if you're sure the issue with the missing device is resolved.

LVM stores a copy of the entire volu me group's metadata (including a map of all of the physical volumes it's backed by) on each individual physical volume within the VG. Attempting to activate a volume group with a missing disk (e.g. because you simply forgot to attach it) will cause LVM to mark the disk as missing in this metadata.

This will manifest itself in errors such as the following:

$ sudo vgscan
Reading all physical volumes.  This may take a while...
Couldn't find device with uuid '000000-0000-0000-0000-0000-0000-000000'.
Found volume group "A" using metadata type lvm2

$ sudo vgchange -a y A
Couldn't find device with uuid '000000-0000-0000-0000-0000-0000-000000'.
Refusing activation of partial LV LV-A-1. Use --partial to override.
Couldn't find device with uuid '000000-0000-0000-0000-0000-0000-000000'.
0 logical volume(s) in volume group "A" now active

First, back up the existing metadata:

sudo vgcfgbackup -f ~/broken-vg-meta A

Make a copy of it we can edit -- we want to preserve the original in case we're unable to resolve it:

sudo cp ~/broken-vg-meta{,.fixing}

Edit ~/broken-vg-meta.fixing with your preferred editor and locate the MISSING flags within the block for the volume group you're recovering (A in our case). For example, alter the flags line here such that the value is []:

# [snip]
A {
    # [snip]
    physical_volumes {
        # [snip]
        pv0 {
            # [snip]
            flags = ["MISSING"]
            # [snip]
        }
        # [snip]
    }
    # [snip]
}
# [snip]

Exit your editor, saving changes to the file. We can then ask LVM to apply these changes to the physical volumes in our volume group:

sudo vgcfgrestore -f ~/broken-vg-meta.fixing A

If all's worked you should now be able to activate the VG:

$ sudo vgchange -a y A
3 logical volume(s) in volume group "A" now active

---

Backlinks

• Device Mapper (public)