(solved) Installer on Live DVD (and disk mgt) dies: dmraid

[waferhead]

Solved by ditching fake/dmraid and just using mdadm like I should have in the beginning.
***************

Relocating here from the MGA5 feedback thread
Keywords: RAID dmraid installer

Finally bit the bullet and upgraded my rock stable Mageia 4 installation to Mageia 5.
This is my 'main" box, mythtv backend w/tuners etc, 3 monitors driven by a 2GB ATI 6950
2 raid arrays: /home is on a 1tb raid1, 1gb, /video (mythtv recordings etc) on a 2tb raid0.

After the regular installers choked, I tried an online update via urpmi, which was ~fast but failed installing the new kernel, fortunately the mga4 ones still worked.

On MGA5, the arrays are apparently no longer started automatically on boot?
(this setup has worked before Mageia existed on Mandriva, then MGA1/2/3/4 with ~no issues.)

If I run "''dmraid -ay" it finds the arrays/partitions, then "mount -av" mounts everything fine.
THEN I can log into X11 per usual. (/home is on a RAID1)

If i try to boot/install off live iso, it will fail to run the installer or mcc disc manager (open//close w/no errors shown) until the same dmraid procedure is run. the mcc boot loader borks in a similar fashion.
(will start working after running dmraid -ay to enable the arrays, which should me automatic)

This used to work.

Is dmraid no longer run in the initrd?
Why are we even using dmraid?
I thought it was deprecated ages ago with mdadm as the preferred soft RAID subsystem.

The box is "up", the update to 5 ran fine after I figured this out, but it still requires manual intervention every time I reboot.

Didn't see anything obvious in bugzilla so i figured I'd ask here.
(seaching for RAID seems to return all bugs?)

I'm going to attempt a clean install from the traditional installer DVD later.

[jiml8]

The upgrade certainly should have picked up the required RAID modules. However, if it didn't, then you can add the modules to the initrd yourself.

Run the command "dracut --list-modules" to see what modules are currently in your initrd. If you don't see the required modules, then you can edit the file /etc/dracut.conf to add the ones you need on the line add_dracutmodules= .

I will note that my own initrd includes the dmraid (and the mdraid) modules, even though I am not using them.

After you specify the required modules, execute the command "dracut" to build a new initrd. You may need to use the -f option to force your current initrd to be overwritten, or you might want to rename your old initrd prior to executing dracut, to preserve it in case something goes wrong.

In your situation, I would create a new initrd anyway, even if the modules you need are already present, because obviously something is wrong with the current initrd. By creating a new one, working within the system this initrd is supposed to be starting, you give the system another chance to get it right.

edit:

After re-reading your post, I see that "the regular installers choked" and you could not update the kernel. I have no idea what went wrong for you, but I suggest you make another try to update the kernel using drakconf. This should cause a new initrd to be written and will solve your problem.

[waferhead]

dmraid et. al. were in initrd, but tried dracut -f anyway (after raid was running and drives mounted) to no effect.

Started over...
Had a backup image of mga4 / and /usr: tried:
1)update online via urpmi: failed to detect/configure dmraid or pick up existing configuration.
(really need to add partclone to the repos again, I keep a binary statically linked copy handy)

(reinstalled backup image)

2)Update via traditional installer: Installer saw RAID partitions, asked for confirmation to configure, install went normally, raid still does not autoload on reboot and dumps you to the (previously useless) emergency shell
(which admittedly seems a lot more useful than last time I used it, as / is mounted and real vi etc are now available)
Everything but / really should be to be "nofail" in the fstab due to that bug.
(the update attempts could not install the new kernel after reboot as the RAIDs were not online and something chokes during the new kernel install)

3) clean install from traditional installer: same results as 2)

4) clean install from Livedvd: did not autodetect RAID, and installer or MCC disk config would pop up and die unless dmraid -ay was first run from the shell forcing it. Remembered to edit /etc/fstab so everything but / has "nofail" option this time before rebooting.

Did final reinstall sans swap to fully avoid the "nofail" bug via killing hibernation, with option "nofail" on remainder of partitions in fstab.

In ALL cases ~same results.

I'd simply stick "dmraid -ay" in a config file if I could find one that ran early enough, but systemd is a bit inscrutable so far, will poke at it a bit more tonight. I think there's a "redhat-(something)" systemd script somewhere that's supposed to start RAID that isn't quite making it.

Its pretty trivial to set everything for "nofail" and just manually bring it up, then log in as user, but it still just `aint right.
System works great otherwise, boot and shutdown times much improved.

If memory serves, I ran into something similar on MGA4 and installed sans the RAID drives, then added them after updating etc, worked.
Also tried that from live DVD, Didn't work on MGA5.

[waferhead]

Ah:(headslap)
From dmesg:
[ 1.331036] md: Skipping autodetection of RAID arrays. (raid=autodetect will force)
I guess I'll try adding that to the grub2 config, assuming I can parse it correctly.
Need to figure out what's causing that absolutely brilliant default option and kill it.

Also need to figure out where to add that to the grub2 configuration scripts so it won't get removed from grub.cfg on next update.

Result: No effect, still need to manually run dmraid -ay and mount -av.

With added "raid=autodetect" in kernel command line, Now dmesg says:
[ 1.992758] md: If you don't use raid, use raid=noautodetect
[ 1.992919] md: Autodetecting RAID arrays.
[ 1.992959] md: Scanned 0 and added 0 devices.
[ 1.992998] md: autorun ...
[ 1.993036] md: ... autorun DONE.

So much for the RAID autodetect routine in the kernel or initrd (probably kernel that early)
... Linux RAID wiki says in-kernel RAID autodetect is deprecated anyway, and the partitions are on AMD fake raids so no joy there.
https://raid.wiki.kernel.org/index.php/Autodetect

I need to get "dmraid -ay" run early enough so /home can automount at least, any suggestions welcome.

Then all I have to do is get mythbackend to run as a service, again.
(runs fine from shell, but dies as a service, even set uid root, but I digress)

[waferhead]

I think I have to play with dracut, it appears that it may be the key to doing something intelligent with dmraid as things get rolling during the early boot process...

Unfortunately, adding the kernel cmd line arg rd.dm=1 (which is passed to dracut, and should theoretically force dmraid to autoconfigure things) has no effect.

In the meantime I have added (as a quick and dirty work around):

Code: Select all

dmraid -ay >/dev/null
mount -a >/dev/null


...to the beginning of /etc/init.d/partmon, and it seems to get things configured and mounted early enough in the boot process.

[waferhead]

Continuing beating on this, I think i have a correct, sane cure, and it involves a one byte change to one systemd config file...
/usr/lib/systemd/fedora-storage-init

Can I have an "i"?

The problem looks like

Code: Select all

if ! strstr "$cmdline" nodmraid && [ -x /sbin/dmraid ]; then
        modprobe dm-mirror >/dev/null 2>&1
        dmraidsets=$(LC_ALL=C /sbin/dmraid -s -c -i)


I need to break my system again to test something...

[waferhead]

I was wrong.

Turns out, the root issue appears to be that the systemd module "fedora-storage-init" is not being loaded. At all.

Lacking the kernel boot arg "nodmraid" or such, if /sbin/dmraid exists, and "dmraid -s -c -i" returns any arrays (it does) it should eventually run "dmraid -ay..." assuming it is run. It's apparently not happening.
(It also seems like dmraid should have some sort of config file, but i digress)

Why "fedora-storage-init" is not being loaded remains a mystery, but there were a few "fedora-"services running on MGA4....

It appears it should be automagically loaded by local-fs-target.wants, via fedora-storage-init.service (which annoyingly doesn't seem to be a service you can call/enable directly)

Systemd has given me a headache, so I'll stick with the dirty hack to /etc/init.d/partmon for tonight.

[doktor5000]

Could you please report that as a bug, after searching if it hasn't been reported yet already: https://wiki.mageia.org/en/How_to_report_a_bug_properly
For completeness's sake and also to allow others to follow up on that bug report, please also post the link to it here in the thread, thanks.

Only things that are loosely related to dmraid and upgrade from mga4 are https://bugs.mageia.org/show_bug.cgi?id=6180
and the followup https://bugs.mageia.org/show_bug.cgi?id=16158

[waferhead]

I'm somewhat hesitant too call "bug" on this quite yet, esp as I'm not quite sure what package to blame, if any.
I have been doing a little more digging.

When i run the dmraid command in fedora-storage-init, it generates a weird error, and does not autobuild.

Code: Select all

/sbin/dmraid -ay -i --rm_partitions -p "$dmname"

returns (only pdc will return non-error state as its the controller in use, other ommited for clarity)

ERROR: pdc: wrong # of devices in RAID set "pdc_biihegaai" [1/2] on /dev/sdd
ERROR: pdc: wrong # of devices in RAID set "pdc_bghbbjbci" [1/2] on /dev/sdc
ERROR: pdc: wrong # of devices in RAID set "pdc_bghbbjbci" [1/2] on /dev/sdc
ERROR: pdc: wrong # of devices in RAID set "pdc_biihegaai" [1/2] on /dev/sdb
ERROR: pdc: wrong # of devices in RAID set "pdc_bghbbjbci" [1/2] on /dev/sda
ERROR: pdc: wrong # of devices in RAID set "pdc_bghbbjbci" [1/2] on /dev/sda
ERROR: either the required RAID set not found or more options required


Odd errors on pdc, but dmraid -f reports: (before or after manually running "dmraid -ay")

[code
/dev/sdd: pdc, "pdc_biihegaai", stripe, ok, 1953394048 sectors, data@ 0
/dev/sdc: pdc, "pdc_bghbbjbci", mirror, ok, 1953394048 sectors, data@ 0
/dev/sdb: pdc, "pdc_biihegaai", stripe, ok, 1953394048 sectors, data@ 0
/dev/sda: pdc, "pdc_bghbbjbci", mirror, ok, 1953394048 sectors, data@ 0

[/code]

Which indicates the errors from the command in fedora-storage-init are bogus.

Of couse dmraid -ay (by itself) returns

Code: Select all

RAID set "pdc_biihegaai" already active
RAID set "pdc_biihegaai" was not activated
RAID set "pdc_bghbbjbci" already active
RAID set "pdc_bghbbjbci" was not activated
RAID set "pdc_biihegaai1" was activated
RAID set "pdc_biihegaai1" was not activated
RAID set "pdc_biihegaai2" was activated
RAID set "pdc_biihegaai2" was not activated
RAID set "pdc_biihegaai5" was activated
RAID set "pdc_biihegaai5" was not activated
RAID set "pdc_bghbbjbci1" was activated
RAID set "pdc_bghbbjbci1" was not activated
RAID set "pdc_bghbbjbci5" was activated
RAID set "pdc_bghbbjbci5" was not activated
RAID set "pdc_bghbbjbci6" was activated
RAID set "pdc_bghbbjbci6" was not activated
RAID set "pdc_bghbbjbci7" was activated
RAID set "pdc_bghbbjbci7" was not activated


Which oddly actually results in the arrays being activated and mountable

Also fedora-init-service and parents do not make it into the initrd or come up on an installed system if system is installed with the default boot args including dm,rd=0 md.rd=0 etc....
Fedora-storage-init and friends ARE loaded if those boot args are set =1 during install, but they don't work properly due to the above bogus "wrong number of devices" error from dmraid.

[waferhead]

I think i'm going to image /home and /video, nuke it from orbit, and set it up from scratch using mdraid.

Its not like I love Windows and really needed to use the fakeraid, I just have it on some free space on a RAID0.

Its really about time I replaced the discs anyway, the WD Green 1TBs are long past their "use by" date running 24/7.
3-4TB drives are about the same cost as the "greens" were new, and a lot better.

It's annoying as the same raid set was working for so many distro releases, until the upgrade to mga5, but its just not worth the hassle.
Its perfectly usable now with the work around, until I start over.

[waferhead]

Deleted and recreated the RAIDS in bios, seen by seen/enabled in installer, mounted/accessible during install, but cannot mount on installed MGA5 system.
(says it mounts, but it silently fails, nothing in logs. dmraid is not my friend.)

Did the whole sequence again using Mint 17.2, no issues seeing the RAIDS, or installing.

Subsequently retried installing mgaeia5 from live DVD then classic installer using tested fresh arrays, same issue.

Apparently stuck running Mint17 until this gets resolved, hopefully Xboxboy can get mdadm auto config working, it has been the preferred tool for many years now.

[waferhead]

Finally sucked it up, erased everything and set it up using mdadm. Took 30 seconds from the live DVD.
Created the basic partitions in MCC, then

Code: Select all

mdadm --create /dev/md0 --level=5 --raid-devices=4 /dev/sd[abcd]1 --name=HOME


After rebooting to the liveDVD to verify it was still picked up, and doing a fresh install,(again verifying it came back up ssd boot) I imaged over the old /home partition with partclone, ran a fsck and resized to full capacity.

Now have a 4 disc 2.7TB (formatted) RAID5 soon to grow to 5-6disc raid 6, researching converting to btrfs, converting in-place is trivial, and I still have a backup image if all else fails.

Set the partition sizes ~1gb smaller than required just as disk sized vary and that's enough room to be possibly useful... formatted in the installer, it instantly found array and cruised through with no issues.

Mageia5 seems to have no issues seeing mdadm-created soft raid partitions during install or afterwards.

Just Say No to fake RAID and dmraid.

IIRC mandriva 2007-2010 (maybe older?) could create a md soft raid setup entirely from the installer.

I'll just install another disc for Win10.

[xboxboy]

hopefully Xboxboy can get mdadm auto config working, it has been the preferred tool for many years now.

Sorry, don't know how I missed this thread. Ironically mga5 worked out of the box for me, where as 4 didn't and I don't think three did either. But when I say it worked, it is using dmraid, not mdadm. So yes it works, is it the correct way, no not these days, as you're right, mdadm is supposed to run our raid devices.

I've created a feature request for Mageia 6: https://wiki.mageia.org/en/Feature:Use_mdadm_for_raid_devices
Please add to the request and help if we can get it passed.

[waferhead]

hopefully Xboxboy can get mdadm auto config working, it has been the preferred tool for many years now.

Sorry, don't know how I missed this thread. Ironically mga5 worked out of the box for me, where as 4 didn't and I don't think three did either. But when I say it worked, it is using dmraid, not mdadm. So yes it works, is it the correct way, no not these days, as you're right, mdadm is supposed to run our raid devices.

I've created a feature request for Mageia 6: https://wiki.mageia.org/en/Feature:Use_mdadm_for_raid_devices
Please add to the request and help if we can get it passed.

It surprised the hell out of me as well, have been running that same setup for ages now, but stuff happens.
The mdraid setup is a lot more flexible and now I can toss a new motherboard or add discs in and just not worry about it.

Your wiki entry prompted the note about old Mandrake/Mandriva supporting creating md raids in MCC... those scripts are likely to work with only minor massaging. mdadms' name may have changed and it has more options, but it still works about the same.

dmraid was simply a little too flake for my setup on the AMD//Promise pdc "controller", even after creating clean dm arrays in Mint17.2, MGA5 would see them, I could configure them in the installer, but they would silently fail to mount in Mageia (and I wasn't digging Mint--- a lot of subtle things don't work right)...even provided a "bla bla mounted on /home or /video" in the logs, but no mounts actually happened, it was worse than the original borked arrays, at least I was able to get a work around with the original setup.