fstrim on ext4 on LVM on LUKS

[doktor5000]

Picking up from thread viewtopic.php?f=7&t=8599

You don't use LVM over LUKS, no?

*I* use ext4 partitions on LVM on LUKS.
So there is some known issue / method then?

For all partitions in that LVM, fstrim fails with "the discard operation is not supported"
While on the /boot partition which is a separate ext4 physical partition, fstrim succeed.

NOTE: I am running mga5 cauldron, 64 bit. Maybe something changed?

I followed https://wiki.mageia.org/en/Installation ... im_command :

In i /etc/lvm/lvm.conf i have the line

Code: Select all

   issue_discards = 1

Code: Select all

# cat /etc/crypttab
crypt_sda5 UUID=1683a090-76a1-4a2a-99c6-82821bbe2ab8 - discard

As it did not work after reboot, i also tried running # dracut -f and reboot, after reading this thread also dracut -f -I /etc/crypttab and reboot.

Code: Select all

# lsinitrd | grep crypttab
-rw-r--r--   1 root     root           84 Apr 20 08:49 etc/crypttab

I could not find a god idea so this far i have only checked the variants:
I note the line in /etc/crypttab may have "none" or "-". tried both, (incl dracut -f, reboot) no change.
Also there, tried with or without "luks," before "discard"
Some systems (debian derived) want "allow_discard" instead of "discard" : tried that too as i see in backups i had that on mga 3, but do not remember if it worked.

( Other SSD settings checked/made: the physical partition for luks is correctly aligned, kernel use correct scheduler, in fstab noatime on partitions (excluding /home because of kmail), and discard on swap partition only. )
( Another question is if that fstab trim option on that swap partition on LVM on LUKS is working or not. Anyway i may better move it to the mechanical drive i have and set separate encryption for it there. )

I tried the links you suggested. Best bet is this bug at redhat especially https://bugzilla.redhat.com/show_bug.cgi?id=890533#c38
( Weird they in that bug use "allow-discards" instead of "discards" in /etc/crypttab )
I need to leave this for work now, and do a backup before trying too hard...

So i ask in the mean time: Any better idea / known method for mageia 5 ?

Here is a nice one btw: http://blog.neutrino.es/2013/howto-prop ... d-dmcrypt/

[petedan10]

I wonder if this has been resolved, I mean if there are some definite steps to follow or if the new Mageia 6 takes care of it.

[doktor5000]

Probably not, as it has never been reported as a bug. Only loosely related report I can see is about discard as diskdrake option: https://bugs.mageia.org/show_bug.cgi?id=8701

[petedan10]

I tried following the steps in various guides on the net, and couldn't get fstrim to work either. I think these steps are distro-specific, or even distro-version-specific, and so there must be some variation of those steps to fit Mageia specifically.

Alternatively, I wonder if one could mount the lvm/luks partition using a livecd and issue fstrim from the livecd.

[doktor5000]

The only major difference being debian-like distros vs. fedora-like distros for the option in crypttab (allow-discards vs. discards)
Other then that there's nothing distro-specific about this.

FWIW, as this was already discussed in other thread, best have a look at existing setups:
viewtopic.php?f=7&t=10260 and viewtopic.php?f=7&t=8599

Did you check the contents of etc/crypttab in the initrd that you rebuilt?

[petedan10]

I have read the various pages on the subject and so far I have failed miserably to implement this properly and I wonder what am I missing.

The steps I tried were:

1. Edit /etc/lvm/lvm.conf and set

Code: Select all

issue_discards = 1

2. Edit /etc/crypttab and add

Code: Select all

UUID = .......    none luks, discard

3.run

Code: Select all

dracut -f -I /etc/crypttab

[doktor5000]

Did you check the contents of etc/crypttab in the initrd that you rebuilt?

[petedan10]

Yes, and they look normal:

Code: Select all

crypt_sda5 /dev/disk/by-uuid/blah-blah-blah none luks, discard

where sda5 is the lvm containing / and /home
(btw, viewing the contents of inird now requires skipcpio like

Code: Select all

/usr/lib/dracut/skipcpio initrd.img | gunzip -c | cpio -idv

instead of just cpio)

EDIT: I found that Fedora users have similar issues about enabling FSTRIM on encrypted LVM, and the grub2 config requires some modification, so I believe that something similar is missing:

https://ask.fedoraproject.org/en/answer ... revisions/
https://ask.fedoraproject.org/en/questi ... fedora-23/
http://blog.christophersmart.com/2016/0 ... revisited/

[petedan10]

Checking the contents of initrd.img, I noticed that etc/cmdline.d/90crypt.conf says

Code: Select all

rd.luks.uuid=luks-de023401-ccec-4455-832bf-e5ac477743dc

Now if instead it said

Code: Select all

rd.luks.uuid=luks-de023401-ccec-4455-832bf-e5ac477743dc rd.luks.options=discard

would the problem be solved?

(If this is correct, then the next question is how to put the modified 90crypt.conf back into initrd.img, as the format of initrd.img seems to have changed lately and older posts and guides don't apply)

I see that various distributions have different ways to set rd.luks.options=discard, for example http://blog.christophersmart.com/2016/0 ... revisited/ suggests editing grub config, so I wonder if there is some simpler way than editing initrd.img. I tried adding it to GRUB_CMDLINE_LINUX_DEFAULT but it didn't help.

[doktor5000]

Well you should try to add it interactively to the kernel commandline, in bootmenu just press F3 select -> Default and then append rd.luks.options=discard in the input line that appears.
Does that help? Then you could simply add it to grub options, do you use grub legacy or grub2 ?

[petedan10]

I use grub2.

I tried adding it on startup at the "quiet" point using various variations like

Code: Select all

rd.luks.uuid=luks-uuid rd.luks.options=discard
rd.luks.allow-discards=uuid
luks.options=discard

and nothing worked. I wonder if this is something I am missing or a bug.

[doktor5000]

Maybe I'll try reproduce with a fresh mga6 installation with LVM over LUKS

[petedan10]

I tried the steps at viewtopic.php?f=7&t=9732#p64808 with a fresh installation of Mageia 6 sta1 and the result is the same.

[petedan10]

After trying researching it as much as I could, I finally opened a bug report at https://bugs.mageia.org/show_bug.cgi?id=19086

[petedan10]

I have found a workaround which might even be a good solution.

I will try to explain this workaround along with some thoughts on the subject so that if something is wrong someone can step in and offer corrections.

SSD drives are somewhat new technology that still evolves (remember the Samsung firmware issues couple of years ago and even new controller editions substituting misbehaving older ones) and maybe we don't know yet the full extend of what their proper maintenance means. TRIM is such an area, but my understanding is that trimming once in a while is a good thing to do otherwise the ssd slows down and maybe suffers data integrity issues in the long run, which can be a couple of years or more depending on usage.

As pointed before, getting TRIM to work in encrypted volumes is not that simple. Developers don't enable it by default as it can lead to security issues and actual data areas can be differentiated from empty areas. But never using fstrim? Who knows what that means in the long run. So you have to take some extra steps to enable it by editing lvm.conf and crypttab.

Now there may be some issue/bug doing so under Mageia. Still this shouldn't deter anyone from using lovely Mageia

Here are the steps I used:

Boot into a livecd and open a terminal with root access. Then do the following:

1)

Code: Select all

# blkid | grep crypto

let's say this gives you /dev/sda2, if you have more encrypted volumes find the one you need to trim.

2)

Code: Select all

# cryptsetup luksOpen --allow-discards /dev/sda2 crypttest

enter the password and partition becomes available in /dev/mapper, please note the --allow-discards part.

3)

Code: Select all

# blkid

and find the names of the partitions inside the LVM, for example:

Code: Select all

/dev/mapper/vg--mga-root
/dev/mapper/vg--mga-home

and so on.

4)

Code: Select all

# mkdir /mnt/crypttest1 && mount /dev/mapper/vg--mga-root /mnt/crypttest1
# mkdir /mnt/crypttest2 && mount /dev/mapper/vg--mga-home /mnt/crypttest2

basically repeat for every partition you want

5)

Code: Select all

# fstrim - v crypttest1
# fstrim - v crypttest2

and again repeat for every partition you want

You will get the result of the fstrim operation. In my case, a test partition of 100GB with about 2GB of test data in it gave a result of about 80GB trimmed. I had used this partition for various tests and had never trimmed it.

So you have to boot into a livecd every few months and run fstrim from there. While a minor nuisance I think it might be a better solution as you don't have to mess with lvm.conf and crypttab and even GRUB or fstab settings that might even have other implications like discarding in real time.

As for swap I don't know if swap can or even needs to be trimmed manually. I think kernel takes care of that but I have read conflicting opinions on this and it might even be out of the scope of the discussion. Actually I am not sure a swap partition is even necessary but this definitely is another issue

[doktor5000]

TRIM is such an area, but my understanding is that trimming once in a while is a good thing to do otherwise the ssd slows down and maybe suffers data integrity issues in the long run, which can be a couple of years or more depending on usage.

But never using fstrim? Who knows what that means in the long run.

You only need trim for performance reasons, if your SSD controller doesn't do it automatically. It's not about data integrity, it's just that without trim for all writes to the SSD the blocks need to be deleted first before they are written. trim removes the requirement for deletion. And as you have noticed, when using encryption using trim actually increases possible attack vectors so yes there is an impact on data integrity when using trim with LVM and encryption.

And you can run completely without trim, tested that on my first SSD, OCZ Vertex 3 for nearly 3 years, without running trim once on the linux side.
No performance impact at all, and I did use the SSD heavily regarding the Mageia partitions.

So don't put too much effort into that our of vague assumptions, better get measurements with and without trim after several months of usage.