[SOLVED] is mpathconf missing or renamed?

[magfan]

I'm trying to get Linux Multipath to work. A first step should be to call

Code: Select all

mpathconfig --enable

. However, this program cannot be found after installing the

Code: Select all

multipath_tools

package. Is there an alternative way to get it working?

[doktor5000]

Ususally the only thing you need is to install multipath-tools and then start multipathd service.
Adjust main config /etc/multipath.conf according to the device class you want to configure.

Although some details may be helpful for what you want to configure multipath I/O in particular, e.g. what storage vendor.

[magfan]

I'm trying to connect a QXcubeSAN with two controllers via ISCSI. The multipathd.service is running, but I can't find any configuration files:

Code: Select all

[root@cortex etc]# systemctl status open-iscsi.service
● open-iscsi.service - Open iSCSI Daemon
     Loaded: loaded (/usr/lib/systemd/system/open-iscsi.service; enabled; preset: disabled)
     Active: active (running) since Wed 2023-09-27 15:24:00 CEST; 16h ago
      Tasks: 2 (limit: 629145)
     Memory: 5.1M
        CPU: 1.291s
     CGroup: /system.slice/open-iscsi.service
             ├─2899 /sbin/iscsid
             └─2901 /sbin/iscsid

Sep 27 15:24:00 cortex systemd[1]: Starting open-iscsi.service...
Sep 27 15:24:00 cortex iscsid[2883]: iSCSI logger with pid=2899 started!
Sep 27 15:24:00 cortex systemd[1]: Started open-iscsi.service.
Sep 27 15:24:01 cortex iscsid[2899]: iSCSI daemon with pid=2901 started!
Sep 27 15:26:07 cortex iscsid[2899]: Connection1:0 to [target: iqn.2004-08.com.qsan:xs3324-000044bc0:dev1.ctr1, portal: 192.168.100.11,3260] through [iface: qsan_iscsi] is operati>
Sep 27 15:26:18 cortex iscsid[2899]: Connection2:0 to [target: iqn.2004-08.com.qsan:xs3324-000044bc0:dev1.ctr2, portal: 192.168.200.11,3260] through [iface: qsan_iscsi] is operati>
[root@cortex etc]# systemctl status multipathd.service
● multipathd.service - Device-Mapper Multipath Device Controller
     Loaded: loaded (/usr/lib/systemd/system/multipathd.service; disabled; preset: disabled)
     Active: active (running) since Wed 2023-09-27 15:48:35 CEST; 16h ago
TriggeredBy: ● multipathd.socket
    Process: 211750 ExecStartPre=/sbin/modprobe -a scsi_dh_alua scsi_dh_emc scsi_dh_rdac dm-multipath (code=exited, status=0/SUCCESS)
   Main PID: 211759 (multipathd)
     Status: "up"
      Tasks: 7
     Memory: 18.9M
        CPU: 2.902s
     CGroup: /system.slice/multipathd.service
             └─211759 /sbin/multipathd -d -s

Sep 27 16:10:39 cortex multipathd[211759]: sdd: path already removed
Sep 27 16:10:39 cortex multipathd[211759]: sde: path already removed
Sep 27 16:28:01 cortex multipathd[211759]: sdb: path already removed
Sep 27 16:28:03 cortex multipathd[211759]: sdc: path already removed
Sep 27 16:28:03 cortex multipathd[211759]: sdd: path already removed
Sep 27 16:28:03 cortex multipathd[211759]: sde: path already removed
Sep 28 08:06:07 cortex multipathd[211759]: sdb: path already removed
Sep 28 08:06:08 cortex multipathd[211759]: sdc: path already removed
Sep 28 08:06:08 cortex multipathd[211759]: sdd: path already removed
Sep 28 08:06:08 cortex multipathd[211759]: sde: path already removed
[root@cortex etc]# ls *multipath*
bindings  wwids
[root@cortex etc]# cat multipath/bindings
# Multipath bindings, Version : 1.0
# NOTE: this file is automatically maintained by the multipath program.
# You should not need to edit this file in normal circumstances.
#
# Format:
# alias wwid
#
[root@cortex etc]# cat multipath/wwids
# Multipath wwids, Version : 1.0
# NOTE: This file is automatically maintained by multipath and multipathd.
# You should not need to edit this file in normal circumstances.
#
# Valid WWIDs:
[root@cortex etc]# multipath
[root@cortex etc]# ll multipath.conf
ls: Zugriff auf 'multipath.conf' nicht möglich: Datei oder Verzeichnis nicht gefunden
[root@cortex etc]#


[magfan]

I have just created a bug report (https://bugs.mageia.org/show_bug.cgi?id=32338)

[doktor5000]

Well /etc/multipath.conf you need to create yourself then. You can grab some default one somewhere, or generate one from the current defaults, which can be queried via

Code: Select all

multipathd show config

E.g. for Ubuntu you also need to create that yourself, see e.g. https://ubuntu.com/server/docs/device-m ... troduction

Also, what's the output from

Code: Select all

multipath -ll

[magfan]

The multipath program does nothing. I can use -h to display all the options, but when I select one of them nothing happens.

Code: Select all

#
# multipathd show config > multipathd_show_config.txt
#
# multipath -ll
#
# multipath -h
multipath-tools v0.8.8 (03/12, 2021)
Usage:
  multipath [-v level] [-B|-d|-i|-q|-r] [-b file] [-p policy] [device]
  multipath [-v level] [-R retries] -f device
  multipath [-v level] [-R retries] -F
  multipath [-v level] [-l|-ll] [device]
  multipath [-v level] [-a|-w] device
  multipath [-v level] -W
  multipath [-v level] [-i] [-c|-C] device
  multipath [-v level] [-i] [-u|-U]
  multipath [-h|-t|-T]

Where:
  -h      print this usage text
  -l      show multipath topology (sysfs and DM info)
  -ll     show multipath topology (maximum info)
  -e      enable foreign libraries with -l/-ll
  -f      flush a multipath device map
  -F      flush all multipath device maps
  -a      add a device wwid to the wwids file
  -c      check if a device should be a path in a multipath device
  -C      check if a multipath device has usable paths
  -q      allow queue_if_no_path when multipathd is not running
  -d      dry run, do not create or update devmaps
  -t      display the currently used multipathd configuration
  -T      display the multipathd configuration without builtin defaults
  -r      force devmap reload
  -i      ignore wwids file
  -B      treat the bindings file as read only
  -b fil  bindings file location
  -w      remove a device from the wwids file
  -W      reset the wwids file include only the current devices
  -R num  number of times to retry removes of in-use devices
  -u      check if the device specified in the program environment should be a
          path in a multipath device
  -U      check if the device specified in the program environment is a
          multipath device with usable paths, see -C flag
  -p pol  force all maps to specified path grouping policy:
          . failover            one path per priority group
          . multibus            all paths in one priority group
          . group_by_serial     one priority group per serial
          . group_by_prio       one priority group per priority lvl
          . group_by_node_name  one priority group per target node
  -v lvl  verbosity level:
          . 0 no output
          . 1 print created devmap names only
          . 2 default verbosity
          . 3 print debug information
  device  action limited to:
          . multipath named 'device' (ex: mpath0)
          . multipath whose wwid is 'device' (ex: 60051...)
          . multipath including the path named 'device' (ex: /dev/sda or
            /dev/dm-0)
          . multipath including the path with maj:min 'device' (ex: 8:0)
#


[doktor5000]

Does your system actually see the disks or ISCSI initiators as block devices at all ?

What's the output of

Code: Select all

lsblk -f -p
ls -al /dev/disk/by-*

[magfan]

After uninstalling open-iscsi and multipath-tools, I reinstalled both packages. This time it looks different:

Code: Select all

[root@cortex mer]# multipath -ll
3202e0013781103c0 dm-4 QSAN,XS3324
size=73T features='0' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 12:0:0:0 sdf 8:80 active ready running
`-+- policy='service-time 0' prio=50 status=enabled
  `- 13:0:0:0 sdg 8:96 active ready running
[root@cortex mer]# cat /etc/multipath/wwids
# Multipath wwids, Version : 1.0
# NOTE: This file is automatically maintained by multipath and multipathd.
# You should not need to edit this file in normal circumstances.
#
# Valid WWIDs:
/3202e0013781103c0/
[root@cortex mer]# cat /etc/multipath/bindings
# Multipath bindings, Version : 1.0
# NOTE: this file is automatically maintained by the multipath program.
# You should not need to edit this file in normal circumstances.
#
# Format:
# alias wwid
#
[root@cortex mer]#

Maybe it's just a timing issue? Multipath looks for iSCSI devices before they exist? How could I track this down? I have attached the output of

Code: Select all

lsblk -f -p > lsblk_f_p.txt; ls -al /dev/disk/by-* > ls_al_dev_disk_by.txt

Finally, what is the name of the multipath device? Something I can mount via fstab to work with...

[doktor5000]

Please just paste the contents and don't upload such short outputs. Also I don't think there's a timing issue. Either multipath can see block devices or it doesn't, that's more a question whether your ISCSI setup was working properly beforehand.

Just for clarity, do you know what multipath is intended for, and did you use or configure it at all beforehand?

If you want the multipath map to have a different name then the default one, which comes from the WWPN (3202e0013781103c0 in your case)
then you want to add a multipath section in /etc/multipath.conf - see "multipaths section" in the man page for multipath.conf.

Code: Select all

        multipath {
                wwid                    3202e0013781103c0
                alias                   iscsi_multipath_1
        }

Finally, what is the name of the multipath device? Something I can mount via fstab to work with...

See above for a user friendly name. Although how do you want to mount a block device ?
If you want to use the block device for something then it would be /dev/mpath/iscsi_multipath_1 or /dev/mapper/iscsi_multipath_1 for the above alias (or respectively /dev/mapper/3202e0013781103c0 without an alias).

[magfan]

OK, after reinstalling mga9, I installed open-iscsi and configured iscsi. Then I installed multipath packages. Now lsblk finds both devices (sdf1, sdg1), but multipath still doesn't see anything:

Code: Select all

# tree /var/lib/iscsi
/var/lib/iscsi
├── ifaces
│   ├── qsan_iscsi_1
│   └── qsan_iscsi_2
├── nodes
│   ├── iqn.2004-08.com.qsan:xs3324-000044bc0:dev1.ctr1
│   │   └── 192.168.100.11,3260,1
│   │       └── qsan_iscsi_1
│   └── iqn.2004-08.com.qsan:xs3324-000044bc0:dev1.ctr2
│       └── 192.168.200.11,3260,1
│           └── qsan_iscsi_2
└── send_targets
    ├── 192.168.100.11,3260
    │   ├── iqn.2004-08.com.qsan:xs3324-000044bc0:dev1.ctr1,192.168.100.11,3260,1,qsan_iscsi_1 -> /var/lib/iscsi/nodes/iqn.2004-08.com.qsan:xs3324-000044bc0:dev1.ctr1/192.168.100.11,3260,1
    │   └── st_config
    └── 192.168.200.11,3260
        ├── iqn.2004-08.com.qsan:xs3324-000044bc0:dev1.ctr2,192.168.200.11,3260,1,qsan_iscsi_2 -> /var/lib/iscsi/nodes/iqn.2004-08.com.qsan:xs3324-000044bc0:dev1.ctr2/192.168.200.11,3260,1
        └── st_config

12 directories, 6 files
# lsblk
NAME              MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINTS
sda                 8:0    0  6,5T  0 disk 
├─sda1              8:1    0   64G  0 part  [SWAP]
└─sda2              8:2    0  6,5T  0 part  /data
sdb                 8:16   1    0B  0 disk 
sdc                 8:32   1    0B  0 disk 
sdd                 8:48   1    0B  0 disk 
sde                 8:64   1    0B  0 disk 
sdf                 8:80   0 72,8T  0 disk 
└─sdf1              8:81   0   16M  0 part 
sdg                 8:96   0 72,8T  0 disk 
└─sdg1              8:97   0   16M  0 part 
sr0                11:0    1 1024M  0 rom   
sr1                11:1    1 1024M  0 rom   
nvme1n1           259:0    0  1,9T  0 disk 
├─nvme1n1p1       259:1    0    1G  0 part 
└─nvme1n1p2       259:2    0  1,9T  0 part 
  └─md0             9:0    0  1,9T  0 raid1
    ├─vg0-lv_root 252:0    0  128G  0 lvm   /
    ├─vg0-lv_var  252:1    0  128G  0 lvm   /var
    ├─vg0-lv_opt  252:2    0  128G  0 lvm   /opt
    └─vg0-lv_home 252:3    0  512G  0 lvm   /home
nvme0n1           259:3    0  1,9T  0 disk 
├─nvme0n1p1       259:4    0    1G  0 part  /boot/EFI
└─nvme0n1p2       259:5    0  1,9T  0 part 
  └─md0             9:0    0  1,9T  0 raid1
    ├─vg0-lv_root 252:0    0  128G  0 lvm   /
    ├─vg0-lv_var  252:1    0  128G  0 lvm   /var
    ├─vg0-lv_opt  252:2    0  128G  0 lvm   /opt
    └─vg0-lv_home 252:3    0  512G  0 lvm   /home
# multipath -ll
#


[magfan]

Finally, I found a working solution. I downloaded the rpm multipath packages listed in the QSAN document "How to implement iSCSI multipath on Linux OS". I also had to download the required libraries. Then I installed the packages. Now everything works perfectly.

Code: Select all

# multipath -ll
mpatha (3202e0013781103c0) dm-4 QSAN,XS3324
size=73T features='0' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 13:0:0:0 sdf 8:80 active ready running
`-+- policy='service-time 0' prio=50 status=enabled
  `- 14:0:0:0 sdg 8:96 active ready running
#

[doktor5000]

What does multipathd show config show now ?

[magfan]

See attached

Code: Select all

multipathd show config > multipathd_show_config .txt

[doktor5000]

Interesting this is the default settings are all the same - this is the only one that differs (with the Mageia packages this was set to no):

Code: Select all

user_friendly_names "yes"

As you have a working setup, and this seems to be reproducible (and other might also want to use iSCSI with multipathing), would be appreciated if you could report this as a bug.

[magfan]

Here is the reference for the bug report: https://bugs.mageia.org/show_bug.cgi?id=32338