Mageia 5 - PCI GE6200, PCI-ex Tesla c1060; only PCI OK

[ishkibibble]

The Nvidia driver, v304, is the only driver that is loaded. I have installed the Mageia CUDA 6.5 toolkit with its associated driver for the c1060 but it isn't loaded.
On other machines with Cauldron using 3 gpu's, Xinerama is invoked during installation and I can set each gpu independently. In this case the option did not appear.
What is the preferred manner of loading the drivers for both the GE 6200 and the C1060 please?
I am using a Dell 690.

[doktor5000]

I have installed the Mageia CUDA 6.5 toolkit with its associated driver for the c1060 but it isn't loaded.

Are you talking about the driver itself or CUDA?
Please show the output of

Code: Select all

grep -i driver /var/log/Xorg.0.log
lspcidrake -v | grep VGA

Also, what you you mean in particular by

What is the preferred manner of loading the drivers for both the GE 6200 and the C1060 please?

From the subject your Geforce 6200 seems OK, or what's the issue with it?

[doktor5000]

Code: Select all

[jw@localhost ~]$ grep -i driver /var/log/Xorg.0.log
[    30.959]    X.Org Video Driver: 18.0
[    30.959]    X.Org XInput driver : 21.0
[    31.151] (II) Loading /usr/lib64/xorg/modules/drivers/v4l_drv.so
[    31.192]    ABI class: X.Org Video Driver, version 18.0
[    32.178]    Module class: X.Org Video Driver
[    32.204] (II) v4l driver for Video4Linux
[    32.204] (II) NVIDIA dlloader X Driver  304.128  Wed Aug 26 10:39:32 PDT 2015
[    32.204] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[    35.268] (II) NVIDIA(0): [DRI2]   VDPAU driver: nvidia
[    35.853]    Module class: X.Org XInput Driver
[    35.853]    ABI class: X.Org XInput driver, version 21.0
[    35.853] (II) Using input driver 'evdev' for 'Power Button'
[    35.876] (II) Using input driver 'evdev' for 'Power Button'
[    35.876] (II) No input driver specified, ignoring this device.
[    35.876] (II) No input driver specified, ignoring this device.
[    35.877] (II) No input driver specified, ignoring this device.
[    35.877] (II) No input driver specified, ignoring this device.
[    35.877] (II) Using input driver 'evdev' for 'Logitech Optical USB Mouse'
[    35.942] (II) No input driver specified, ignoring this device.
[    35.942] (II) Using input driver 'evdev' for 'Dell Dell USB Keyboard'
[    35.943] (II) Using input driver 'evdev' for 'Dell Dell USB Keyboard'

Code: Select all

[jw@localhost ~]$ lspcidrake -v | grep VGA
Card:NVIDIA GeForce 6100 to GeForce 7950: NVIDIA Corporation|NV44A [GeForce 6200] [DISPLAY_VGA] (vendor:10de device:0221 subv:3842 subd:b402) (rev: a1)

Code: Select all

[jw@localhost ~]$
[jw@localhost ~]$
[jw@localhost ~]$ GPU 0 - (GeForce 6200) PCI
[jw@localhost ~]$ GPU 1  -(Tesla C1060) PCI Express x16 Gen1


i would like to use the c1060 for programming. The c1060 shows up in the Nvidia display as GPU 1 but it is not available due to the mismatched driver. Only the 304 driver is loaded and not the 346. The C1060 is the issue and I would like to know of the proper way to add both cards and drivers please.
Thank you for your very quick reply!

[doktor5000]

Hmmm, only one card shows up. Please provide the complete output of

Code: Select all

lspcidrake -v
lspci -nntv

Basically you would need to configure the xorg.conf so that the X server can distinguish the two cards.
That is usually done with the BusID option, see e.g. https://devtalk.nvidia.com/default/topi ... d-change-/

You should then have two such sections in your xorg.conf:

Code: Select all

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "GeForce 6200"
    BusID          "PCI:1:0:0"
EndSection

Section "Device"
    Identifier     "Device1"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "Tesla C1060"
    BusID          "PCI:2:0:0"
EndSection


or similar.

You can also have a look at http://us.download.nvidia.com/XFree86/L ... index.html

[doktor5000]

Code: Select all

[jw@localhost ~]$ lspcidrake -v
firewire_ohci   : Texas Instruments|TSB43AB22A IEEE-1394a-2000 Controller (PHY/Link) [iOHCI-Lynx] [SERIAL_FIREWIRE] (vendor:104c device:8023 subv:1028 subd:01c0)
Card:NVIDIA GeForce 6100 to GeForce 7950: NVIDIA Corporation|NV44A [GeForce 6200] [DISPLAY_VGA] (vendor:10de device:0221 subv:3842 subd:b402) (rev: a1)
tg3             : Broadcom Corporation|NetXtreme BCM5752 Gigabit Ethernet PCI Express [NETWORK_ETHERNET] (vendor:14e4 device:1600 subv:1028 subd:01c0) (rev: 02)
Card:NVIDIA GeForce 8100 to GeForce 415: NVIDIA Corporation|GT200GL [Tesla C1060 / M1060] [DISPLAY_3D] (vendor:10de device:05e7 subv:10de subd:066a) (rev: a1)
mptsas          : LSI Logic / Symbios Logic|SAS1068 PCI-X Fusion-MPT SAS [STORAGE_SCSI] (vendor:1000 device:0054 subv:1028 subd:1f08) (rev: 01)
shpchp          : Intel Corporation|6311ESB/6321ESB PCI Express Downstream Port E2 [BRIDGE_PCI] (vendor:8086 device:3514) (rev: 01)
shpchp          : Intel Corporation|6311ESB/6321ESB PCI Express Downstream Port E1 [BRIDGE_PCI] (vendor:8086 device:3510) (rev: 01)
shpchp          : Intel Corporation|6311ESB/6321ESB PCI Express to PCI-X Bridge [BRIDGE_PCI] (vendor:8086 device:350c) (rev: 01)
shpchp          : Intel Corporation|6311ESB/6321ESB PCI Express Upstream Port [BRIDGE_PCI] (vendor:8086 device:3500) (rev: 01)
i2c_i801        : Intel Corporation|631xESB/632xESB/3100 Chipset SMBus Controller [SERIAL_SMBUS] (vendor:8086 device:269b subv:1028 subd:01c0) (rev: 09)
unknown         : Intel Corporation|631xESB/632xESB SATA AHCI Controller [STORAGE_SATA] (vendor:8086 device:2681 subv:1028 subd:01c0) (rev: 09)
ata_piix        : Intel Corporation|631xESB/632xESB IDE Controller [STORAGE_IDE] (vendor:8086 device:269e subv:1028 subd:01c0) (rev: 09)
lpc_ich         : Intel Corporation|631xESB/632xESB/3100 Chipset LPC Interface Controller [BRIDGE_ISA] (vendor:8086 device:2670 subv:1028 subd:01c0) (rev: 09)
unknown         : Intel Corporation|82801 PCI Bridge [BRIDGE_PCI] (vendor:8086 device:244e) (rev: d9)
ehci_pci        : Intel Corporation|631xESB/632xESB/3100 Chipset EHCI USB2 Controller [SERIAL_USB] (vendor:8086 device:268c subv:1028 subd:01c0) (rev: 09)
uhci_hcd        : Intel Corporation|631xESB/632xESB/3100 Chipset UHCI USB Controller #4 [SERIAL_USB] (vendor:8086 device:268b subv:1028 subd:01c0) (rev: 09)
uhci_hcd        : Intel Corporation|631xESB/632xESB/3100 Chipset UHCI USB Controller #3 [SERIAL_USB] (vendor:8086 device:268a subv:1028 subd:01c0) (rev: 09)
uhci_hcd        : Intel Corporation|631xESB/632xESB/3100 Chipset UHCI USB Controller #2 [SERIAL_USB] (vendor:8086 device:2689 subv:1028 subd:01c0) (rev: 09)
uhci_hcd        : Intel Corporation|631xESB/632xESB/3100 Chipset UHCI USB Controller #1 [SERIAL_USB] (vendor:8086 device:2688 subv:1028 subd:01c0) (rev: 09)
shpchp          : Intel Corporation|631xESB/632xESB/3100 Chipset PCI Express Root Port 1 [BRIDGE_PCI] (vendor:8086 device:2690) (rev: 09)
snd_hda_intel   : Intel Corporation|631xESB/632xESB High Definition Audio Controller [MULTIMEDIA_AUDIO_DEV] (vendor:8086 device:269a subv:1028 subd:01c0) (rev: 09)
unknown         : Intel Corporation|5000 Series Chipset FBD Registers [BRIDGE_HOST] (vendor:8086 device:25f6 subv:8086 subd:8086) (rev: 12)
unknown         : Intel Corporation|5000 Series Chipset FBD Registers [BRIDGE_HOST] (vendor:8086 device:25f5 subv:8086 subd:8086) (rev: 12)
unknown         : Intel Corporation|5000 Series Chipset Reserved Registers [BRIDGE_HOST] (vendor:8086 device:25f3 subv:8086 subd:8086) (rev: 12)
unknown         : Intel Corporation|5000 Series Chipset Reserved Registers [BRIDGE_HOST] (vendor:8086 device:25f1 subv:8086 subd:8086) (rev: 12)
i7300_idle      : Intel Corporation|5000 Series Chipset FSB Registers [BRIDGE_HOST] (vendor:8086 device:25f0 subv:8086 subd:8086) (rev: 12)
i7300_idle      : Intel Corporation|5000 Series Chipset FSB Registers [BRIDGE_HOST] (vendor:8086 device:25f0 subv:8086 subd:8086) (rev: 12)
i7300_idle      : Intel Corporation|5000 Series Chipset FSB Registers [BRIDGE_HOST] (vendor:8086 device:25f0 subv:1028 subd:01c0) (rev: 12)
shpchp          : Intel Corporation|5000 Series Chipset PCI Express x4 Port 7 [BRIDGE_PCI] (vendor:8086 device:25e7) (rev: 12)
shpchp          : Intel Corporation|5000 Series Chipset PCI Express x4 Port 6 [BRIDGE_PCI] (vendor:8086 device:25e6) (rev: 12)
shpchp          : Intel Corporation|5000 Series Chipset PCI Express x4 Port 5 [BRIDGE_PCI] (vendor:8086 device:25e5) (rev: 12)
shpchp          : Intel Corporation|5000X Chipset PCI Express x16 Port 4-7 [BRIDGE_PCI] (vendor:8086 device:25fa) (rev: 12)
shpchp          : Intel Corporation|5000 Series Chipset PCI Express x4 Port 3 [BRIDGE_PCI] (vendor:8086 device:25e3) (rev: 12)
shpchp          : Intel Corporation|5000 Series Chipset PCI Express x4 Port 2 [BRIDGE_PCI] (vendor:8086 device:25e2) (rev: 12)
unknown         : Intel Corporation|5000X Chipset Memory Controller Hub [BRIDGE_HOST] (vendor:8086 device:25c0 subv:8086 subd:8086) (rev: 12)
hub             : Linux 4.1.15-server-1.mga5 uhci_hcd|UHCI Host Controller [Hub|Unused|Full speed (or root) hub] (vendor:1d6b device:0001)
hub             : Linux 4.1.15-server-1.mga5 uhci_hcd|UHCI Host Controller [Hub|Unused|Full speed (or root) hub] (vendor:1d6b device:0001)
hub             : Linux 4.1.15-server-1.mga5 uhci_hcd|UHCI Host Controller [Hub|Unused|Full speed (or root) hub] (vendor:1d6b device:0001)
hub             : Linux 4.1.15-server-1.mga5 uhci_hcd|UHCI Host Controller [Hub|Unused|Full speed (or root) hub] (vendor:1d6b device:0001)
usbhid          : Logitech|Optical USB Mouse [Human Interface Device|Boot Interface Subclass|Mouse] (vendor:046d device:c016)
hub             : Dell|Dell USB Keyboard Hub [Hub|Unused|Full speed (or root) hub] (vendor:413c device:1003)
usbhid          : Dell|Dell USB Keyboard [Human Interface Device|Boot Interface Subclass|Keyboard] (vendor:413c device:2010)
hub             : Linux 4.1.15-server-1.mga5 ehci_hcd|EHCI Host Controller [Hub|Unused|Full speed (or root) hub] (vendor:1d6b device:0002)
usb_storage     : General |UDisk            [Mass Storage|SCSI|Bulk-Only] (vendor:abcd device:1234)
hid_generic     : Logitech Optical USB Mouse
hid_generic     : Dell Dell USB Keyboard
hid_generic     : Dell Dell USB Keyboard

Code: Select all

[jw@localhost ~]$ lspci -nntv
-[0000:00]-+-00.0  Intel Corporation 5000X Chipset Memory Controller Hub [8086:25c0]
           +-02.0-[01-05]--+-00.0-[02-04]--+-00.0-[03]--
           |               |               \-01.0-[04]--
           |               \-00.3-[05]----0b.0  LSI Logic / Symbios Logic SAS1068 PCI-X Fusion-MPT SAS [1000:0054]
           +-03.0-[06]--
           +-04.0-[07]----00.0  NVIDIA Corporation GT200GL [Tesla C1060 / M1060] [10de:05e7]
           +-05.0-[08]--
           +-06.0-[09]--
           +-07.0-[0a]--
           +-10.0  Intel Corporation 5000 Series Chipset FSB Registers [8086:25f0]
           +-10.1  Intel Corporation 5000 Series Chipset FSB Registers [8086:25f0]
           +-10.2  Intel Corporation 5000 Series Chipset FSB Registers [8086:25f0]
           +-11.0  Intel Corporation 5000 Series Chipset Reserved Registers [8086:25f1]
           +-13.0  Intel Corporation 5000 Series Chipset Reserved Registers [8086:25f3]
           +-15.0  Intel Corporation 5000 Series Chipset FBD Registers [8086:25f5]
           +-16.0  Intel Corporation 5000 Series Chipset FBD Registers [8086:25f6]
           +-1b.0  Intel Corporation 631xESB/632xESB High Definition Audio Controller [8086:269a]
           +-1c.0-[0b]----00.0  Broadcom Corporation NetXtreme BCM5752 Gigabit Ethernet PCI Express [14e4:1600]
           +-1d.0  Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #1 [8086:2688]
           +-1d.1  Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #2 [8086:2689]
           +-1d.2  Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #3 [8086:268a]
           +-1d.3  Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #4 [8086:268b]
           +-1d.7  Intel Corporation 631xESB/632xESB/3100 Chipset EHCI USB2 Controller [8086:268c]
           +-1e.0-[0c]--+-02.0  NVIDIA Corporation NV44A [GeForce 6200] [10de:0221]
           |            \-0a.0  Texas Instruments TSB43AB22A IEEE-1394a-2000 Controller (PHY/Link) [iOHCI-Lynx] [104c:8023]
           +-1f.0  Intel Corporation 631xESB/632xESB/3100 Chipset LPC Interface Controller [8086:2670]
           +-1f.1  Intel Corporation 631xESB/632xESB IDE Controller [8086:269e]
           +-1f.2  Intel Corporation 631xESB/632xESB SATA AHCI Controller [8086:2681]
           \-1f.3  Intel Corporation 631xESB/632xESB/3100 Chipset SMBus Controller [8086:269b]
[jw@localhost ~]$

I have also downloaded the Nvidia/dev post and the 346.96 readme. Thank you for searching these out.
The same issue happened on another machine so I can generalize the solution to other computers running Cauldron.
To recap, I simply modify the Xorg configure file by adding the lines you provided (while in graphics mode) and then reboot?

An aside, how long has it taken you to learn how to troubleshoot these kinds of issues?

[doktor5000]

Please next time use code tags as explained in ftp://ftp5.gwdg.de/pub/linux/mandriva/m ... e_tags.ogv

This was what I'm looking for:

Code: Select all

[jw@localhost ~]$ lspcidrake -v
Card:NVIDIA GeForce 6100 to GeForce 7950: NVIDIA Corporation|NV44A [GeForce 6200] [DISPLAY_VGA] (vendor:10de device:0221 subv:3842 subd:b402) (rev: a1)
Card:NVIDIA GeForce 8100 to GeForce 415: NVIDIA Corporation|GT200GL [Tesla C1060 / M1060] [DISPLAY_3D] (vendor:10de device:05e7 subv:10de subd:066a) (rev: a1)


Code: Select all

[jw@localhost ~]$ lspci -nntv
           +-04.0-[07]----00.0  NVIDIA Corporation GT200GL [Tesla C1060 / M1060] [10de:05e7]
           +-1e.0-[0c]--+-02.0  NVIDIA Corporation NV44A [GeForce 6200] [10de:0221]
 [jw@localhost ~]$

To recap, I simply modify the Xorg configure file by adding the lines you provided (while in graphics mode) and then reboot?

Those were only simplified examples, they're not intended for your to be used directly.
Best do as in the nvidia forum post, change to runlevel 3 via as root

Code: Select all

init 3

then move away/rename your current xorg.conf and generate a fresh one using

Code: Select all

nvidia-xconfig --enable-all-gpus

or use the shorter from

Code: Select all

nvidia-xconfig -a

Then try to switch back to runelvel 5 via reboot or

Code: Select all

init 5

and if X server fails to start check /var/log/Xorg.0.log to see what's wrong.

An aside, how long has it taken you to learn how to troubleshoot these kinds of issues?

In general it's usually sufficient to read enough documentation on a particular topic, and crosscheck with various other sources to get a comprehensive understanding.
In particular, I'm doing this for a little over 10 years now overall.

[ishkibibble]

...use code tags. OK, sorry.
Thank you very much for your help; I will implement the change within the next 24h and post the result.

The quick attention to my issue was unexpected and solidifies my opinion as this being one of the best linux distributions.

[ishkibibble]

Nope, nope and nope. None of the above worked. I worked through a number of reboots trying different things.
With the nvidia-xconfig, black screen with a white x for a mouse pointer. I used the output which identified the Tesla device
and placed it manually into the new xorg.conf file and nothing new developed. Using drakx11 another 346.96 was downloaded and
installed (despite there already existing the 346.96 from the rpm install).

What happens is that either the 346.96 driver or the 304.xx driver is being used for both devices, which does not work. I either have a
non-working graphic display or the C1060 can't run due to the improper driver.

How can I specifically associate the appropriate driver to the appropriate graphics card either with a tool or by modifying a file manually?
Is there a way to invoke Xinerama directly to configure each head independently?

The Xinerama screen comes up during a fresh install when I have multiple gpu's loaded and I have been able to use the C1060 by configuring each head independently in the past. Unfortunately, this old workstation has only one PCI and one PCI-express slot with the rest being PCI-X slots.

Thanks again.

[doktor5000]

For the other cauldron machines with multiple graphics cards, do any use a combination which also requires two different nvidia driver versions?

Apart from that, can't help much more as I've no system where I could reproduce your setup.
You could create a bugreport for that, but I admit that's it's a pretty rare cornercase. But who knows, maybe someone from the other developers can provide a hint on how to get this setup working.
https://wiki.mageia.org/en/How_to_report_a_bug_properly
For completeness's sake and also to allow others to follow up on that bug report, please also post the link to it here in the thread, thanks.

[ishkibibble]

When Cauldron was going through its 4.1->5.0 beta release, a P9X79 build used a 750 TI, 970 and the C1060 without issue.
An X-99 E-WS build has a 760 and 780 so I will stuff the C1060 in there - this runs Cauldron also. I will do this
within the next 48h and post the result. I will be sincerely surprised if it does not work.

I'm speculating this issue exists due to the PCI, PCI-Express mix. I have another box that runs a FX5600 and a D870
(caps out at CUDA 6.0). Fedora 19 didn't take (graphics again) but Ubuntu 12.04 did. SuSe worked also but did not detect
the Ethernet connection.

I believe this is a simple issue and I shall resolve it as time permits. Thank you again for your help.

[doktor5000]

a P9X79 build used a 750 TI, 970 and the C1060 without issue.

An X-99 E-WS build has a 760 and 780 so I will stuff the C1060 in there - this runs Cauldron also.

Well, all of those use the nvidia-current driver, the 6200 is not supported by that driver.

See, this is the matching that MCC uses between the different nvidia models and available drivers:

Code: Select all

[doktor5000@Mageia5]─[09:45:10]─[~] grep -i nvidia -A4 /usr/share/ldetect-lst/Cards+
# NVIDIA

NAME NVIDIA RIVA 128
DRIVER nv
BAD_FB_RESTORE

NAME NVIDIA RIVA TNT to GeForce 2
DRIVER nouveau
#DRIVER2 nvidia71xx (not compatible with x.org server 1.6)

NAME NVIDIA GeForce 2 MX to GeForce 4
DRIVER nouveau
#DRIVER2 nvidia96xx (not compatible with X.org server 1.11)

NAME NVIDIA GeForce FX series
DRIVER nouveau
# DRIVER2 nvidia173 (not compatible with X.org server 1.16)

NAME NVIDIA GeForce 6100 to GeForce 7950
DRIVER nouveau
DRIVER2 nvidia304
#DRIVER2_NO_SSE nvidia96xx
DRIVER2_NEEDS_SSE

NAME NVIDIA GeForce 8100 to GeForce 415
DRIVER nouveau
DRIVER2 nvidia340
#DRIVER2_NO_SSE nvidia96xx
DRIVER2_NEEDS_SSE

# nvidia-current supports some GeForce 410 cards as well,
# but does not support some 415 cards. The higher bound
# is used in the name, since this way both auto-selection
# and manual user selection by model number always result in a
# working driver for 410/415, since all of those are also
# supported by nvidia340.
NAME NVIDIA GeForce 420 series and later
DRIVER nouveau
DRIVER2 nvidia-current
DRIVER2_NEEDS_SSE

NAME NVIDIA GeForce3 (xbox)
DRIVER nvxbox
LINE Option "UseFBDev" "1"
LINE Option "HWCursor" "0"


[ishkibibble]

I am completely aware. I was only swapping over the c1060.
I configure hardware, operating systems, firmware/software relative to their lifecycles and
capabilities and I then optimize such builds by profiling them.. rather than blindly and ignorantly throwing things together.

Preferentially, I would have studied the design specs then worked out and implemented a paper solution, one and done, but
I don't have that luxury presently.

I can't do the swap within the original timeline as this takes out two machines handling a time sensitive workload. When time
permits I shall post the result. I'll also be researching archived posts during the transition from PCI to PCI express when I'm
sure others have encountered similar issues. Note, most of my systems dual/multiboot so I will be testing a couple of compilation
programs when I can. I'll the post the result accordingly.

Cheers.