[SOLVED] how to create lost /boot partition?

[magfan]

Due to "historical" reasons I have a separate boot partition (/dev/sda1) which is mounted under the root partition (/dev/sda2). Because of a hardware failure some parts within /boot are corrupted now: the grub directory is unreadable as well as many files inside /boot. So, I cannot boot the system any more. Any ideas how I could re-create a valid /boot directory?

GPT partitions on hardware RAID 10:

Code: Select all

/dev/sda1 = /boot
/dev/sda2 = /
/dev/sda3 = swap
/dev/sda4 = /tmp
/dev/sda5 = /home
/dev/sda6 = /data


The RAID is OK, no disk failures.

From /etc/fstab:

Code: Select all

# Entry for /dev/sda1 :
UUID=... /boot ext4 acl,relatime 1 2


[doktor5000]

Check which files you need from which packages, e.g. via

Code: Select all

urpmf '^/boot'

and compare those packages to what you have installed. The installed packages should be listed in /var/log/rpmpkgs which is created daily via cronjob, so should be recent enough (the DB files in /var/lib/rpm/ cannot be easily queried)

Then boot a Mageia installation or Live media, mount your broken Mageia installation (/ and /boot below it) and reinstall the affected packages via

Code: Select all

urpmi --root=/path/to/other/mageia/install --replacepkgs --replacefiles options

You probably need grub, dracut and the kernel packages you have currently installed.

Hope this is what you're asking for.

[magfan]

I was not very successful. After installing the /boot-related packages grub, dracut and kernel-server from within Mageia3 live I tried to boot the system. But it did not work...

[ghmitch]

Well, I think you are now getting farther than you were before? In any case, you will have to set up your grub configuration file. As I recall there is a way to do that with the rescue feature on the install media. Or you could simply edit the grub configuration file with a text editor. I suspect that is where the problem lies since that is the key to the whole boot process and is unique to every system.

[doktor5000]

Also, didn't you adjust the contents of the initrd to include some drivers for your RAID setup or something like that IIRC?
You have to repeat that, as currently you only have a standard initrd.

[magfan]

Including the modules for the hardware RAID (megaraid_sas) did not help much. What I did was the following:

Code: Select all

starting Mageia 3 rescue
chroot /mnt
cd /boot
dracut -f --add-drivers "megaraid_sas" initrd-3.10.24-server-2.mga3.img 3.10.24-server-2.mga3
lsinitrd initrd-3.10.24-server-2.mga3.img | grep sas       # -> sas modules are included !
exit
re-install boot-loader from Mageia 3 rescue menu
reboot


Well, after a couple of days it is not only the kernel panicking..

[magfan]

Today I am a bit further. At least Mageia starts to boot. But then it fails with the following message:

Code: Select all

dracut Warning: /sysroot has no proper rootfs layout, ignoring and removing offending mount hook
dracut Warning: Can't mount root filesystem


Well, I did not succeed with re-installing grub from the rescue disk (see previous post). So I tried to "update" Mageia with the Mageia 3 installation dvd. Unfortunately, it may not know about the new root layout (/bin -> /usr/bin, ...). I do not remember when that changed. Anyway, now I have a working kernel image but no proper root filesystem. How could I repair that? It does not look strange to me:

Code: Select all

total 428K
drwxr-xr-x  26 root root 4.0K Feb  5 08:51 .
drwxr-xr-x  11 root root  180 Feb  5 08:46 ..
-rw-------   1 root root 3.1K Feb  5 08:02 .bash_history
lrwxrwxrwx   1 root root    7 Jul 23  2013 bin -> usr/bin
drwxr-xr-x   3 root root 4.0K Feb  5 06:17 boot
drwxr-xr-x   2 root root 4.0K Aug 15  2012 .config
drwxr-xr-x   2 root root 4.0K Aug 15  2012 data
drwxr-xr-x   2 root root 4.0K Sep 18  2012 data1
drwxr-xr-x   2 root root 4.0K Sep 18  2012 data2
drwx------   3 root root 4.0K Jul 23  2013 .dbus
drwxr-xr-x   2 root root 4.0K Jan 27 09:37 dev
drwxr-xr-x 143 root root  12K Feb  5 07:13 etc
drwxr-xr-x   2 root root 4.0K Aug 15  2012 home
drwxr-xr-x   2 root root 4.0K Sep 10  2012 imagepool
drwxr-xr-x   2 root root 4.0K Apr  1  2013 initrd
lrwxrwxrwx   1 root root    7 Jul 23  2013 lib -> usr/lib
lrwxrwxrwx   1 root root    9 Jul 23  2013 lib64 -> usr/lib64
drwx------   2 root root  16K Aug 15  2012 lost+found
drwxr-xr-x   4 root root 4.0K Jul 25  2013 media
drwxr-xr-x   6 root root 4.0K Jul 30  2013 mnt
drwxr-xr-x  31 root root 4.0K Jan 14 08:57 opt
drwxr-xr-x   2 root root 4.0K Jan 27 09:38 proc
-rw-r--r--   1 root root 297K Dec 18 10:03 .readahead
-rw-r--r--   1 root root    0 Jan  1 03:42 .readahead_collect
drwxr-x---  43 root root 4.0K Feb  5 07:13 root
-rw-r--r--   1 root root    0 Feb  5 08:51 root_layout.txt
drwxr-xr-x   4 root root 4.0K Jan 27 12:27 run
lrwxrwxrwx   1 root root    8 Jul 23  2013 sbin -> usr/sbin
drwxrwxr-x   3 root root 4.0K Sep 13  2012 share
drwxr-xr-x   2 root root 4.0K Apr  1  2013 srv
drwxr-xr-x   2 root root 4.0K Aug 15  2012 sys
drwxrwxrwt   7 root root 4.0K Jan 27 12:27 tmp
drwxr-xr-x  18 root root 4.0K Jul 23  2013 usr
drwxr-xr-x  23 root root 4.0K Jul 23  2013 var
-rw-------   1 root root 2.7K Feb  5 07:56 .viminfo


[doktor5000]

From what I can see, that message from dracut is about the contents of the initrd. This has nothing to do with your actual / filesystem,
but with the root filesystem in the initrd or something related to it.

You may want to google for that error message, there are quite some results for this:
https://bugs.mageia.org/show_bug.cgi?id=5518
https://bugzilla.redhat.com/show_bug.cgi?id=730579

[magfan]

Thank you, doktor5000. Again I am a step further. If I replace "root=UUID=..." with "root=/dev/sda2" the system starts and reaches at least some kind of console. Until then a lot of messages of failed services appeared. Unfortunately, also the network does not work. If I call "ifup eth0" I get some kind of "memory access error". But "ifup" is only a script. I would love to re-install all packages from /var/log/rpmpkgs, but for this I need a network connection...

OK, I am a bit lost now. The booting device is /dev/sda. The first partition on this device is "/boot" (/dev/sda1). The second partition is the root partition "/" (/dev/sda2). I have only one single kernel installed (3.10.24-server-2.mga3). What can I do in which order to configure a proper /boot partition?

[magfan]

After another attempt I got stuck again (see image below). But on a terminal I could check the block ids of all partitions. I am very surprised that for the boot partition I got an extra field (PTTYPE="dos") and also UUID and PARTUUID are not identical! For the other partitions both UUIDs are identical. How can I correct this?

Code: Select all

# blkid /dev/sda1
/dev/sda1: LABEL="boot" UUID="...b1..." TYPE="ext4" PTTYPE="dos" PARTUUID="...b2..."
# blkid /dev/sda2
/dev/sda2: LABEL="system" UUID="...r1..." TYPE="ext4" PARTUUID="...r1..."


[doktor5000]

Well actually your system should still be untouched and working if you get it to boot normally if you didn't change anything.
And looking at the screenshots it seems everything except /boot has been mounted correctly already, I can see /tmp, /data, swap and /home all fscked and mounted.

For the UUIDs that don't work, you can temporarily change the fstab entries to use devicenames or Labels too, whatever works for you. And to find out the UUIDs, i'm normally using

Code: Select all

lsblk -f

as the format is nicer to read. You can also have a look at output of

Code: Select all

ls -al /dev/disk/by-uuid/

to see if there's maybe some mapping gone wrong, duplication or something like that.

My advice would be to get to another box, preferrably put it next to your broken one, and start reading and following down the systemd depedency chain and poke heere and there to see where it failed and how. That how I'd approach it. Seeing that the problem is with local filesystem target or one of its dependencies, start there:

Code: Select all

$ systemctl | grep local-fs
local-fs-pre.target                                                                       loaded active active    Local File Systems (Pre)
local-fs.target                                                                           loaded active active    Local File Systems

Get the defining file for each

[doktor5000@Mageia3 ~]$ systemctl status local-fs.target
local-fs.target - Local File Systems
Loaded: loaded (/usr/lib/systemd/system/local-fs.target; static)
Active: active since Mo 2014-02-03 23:54:46 CET; 1 day 15h ago
Docs: man:systemd.special(7)

Feb 03 23:54:46 Mageia3 systemd[1]: Starting Local File Systems.
Feb 03 23:54:46 Mageia3 systemd[1]: Reached target Local File Systems.
[doktor5000@Mageia3 ~]$ systemctl status local-fs-pre.target
local-fs-pre.target - Local File Systems (Pre)
Loaded: loaded (/usr/lib/systemd/system/local-fs-pre.target; static)
Active: active since Mo 2014-02-03 23:54:45 CET; 1 day 15h ago
Docs: man:systemd.special(7)

[doktor5000@Mageia3 ~]$

Then look in there to see relevant documentation or dependencies.

[doktor5000@Mageia3 ~]$ cat /usr/lib/systemd/system/local-fs.target
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.

[Unit]
Description=Local File Systems
Documentation=man:systemd.special(7)
After=local-fs-pre.target
DefaultDependencies=no
Conflicts=shutdown.target
OnFailure=emergency.target
OnFailureIsolate=no

[doktor5000@Mageia3 ~]$ cat /usr/lib/systemd/system/local-fs-pre.target
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.

[Unit]
Description=Local File Systems (Pre)
Documentation=man:systemd.special(7)
RefuseManualStart=yes

Or directly go to the documentation and read up how it's supposed to work:

Code: Select all

[doktor5000@Mageia3 ~]$ man systemd.special