[SOLVED] cifs module kernel crash after upgrade to 4.14.121

[doktor5000]

Dear gurus,

after the upgrade to the kernel version 4.14.121 my cifs mounts went south. They seems to be mounted fine on systems startup, but became totally unresponsive if I initiate write to the remote filesystem. I can see the following in the syslog:

Code: Select all

Jun  8 22:14:14 wdatalinux kernel: [50090.436227] stack segment: 0000 [#17] SMP PTI
Jun  8 22:14:14 wdatalinux kernel: [50090.436230] Modules linked in: ipt_REJECT nf_reject_ipv4 snd_seq_dummy snd_seq nvidia_modeset(PO) rpcsec_gss_krb5 auth_rpcgss oid_registry nfsv4 nfs lockd grace cmac sunrpc arc4 md4 fscache nls_utf8 cifs ccm xt_multiport iptable_filter ip_tables x_tables fuse sctp af_packet ir_sony_decoder ib_core it87 hwmon_vid msr dm_mirror dm_region_hash dm_log dm_mod uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_core snd_usb_audio videodev snd_usbmidi_lib snd_rawmidi media snd_seq_device rc_technisat_ts35 tda10023 tda10021 nvidia(PO) joydev input_leds ir_rc5_decoder hid_generic usbhid hid rc_hauppauge ir_lirc_codec lirc_dev igorplugusb ppdev iTCO_wdt iTCO_vendor_support gpio_ich snd_hda_codec_hdmi coretemp kvm_intel kvm psmouse ipmi_devintf ipmi_msghandler snd_hda_codec_realtek irqbypass
Jun  8 22:14:14 wdatalinux kernel: [50090.436275]  snd_hda_codec_generic snd_hda_intel snd_hda_codec parport_pc snd_hda_core parport snd_hwdep snd_pcm ftdi_sio usbserial i2c_i801 mantis mantis_core snd_timer dvb_core skge snd rc_core soundcore shpchp lpc_ich acpi_cpufreq evdev sch_fq_codel ipv6 crc_ccitt autofs4 uhci_hcd serio_raw sr_mod ehci_pci ehci_hcd usbcore usb_common button ide_pci_generic jmicron ide_core ata_generic pata_acpi pata_jmicron
Jun  8 22:14:14 wdatalinux kernel: [50090.436301] CPU: 1 PID: 25525 Comm: file.so Tainted: P      D    O    4.14.121-desktop-1.mga6 #1
Jun  8 22:14:14 wdatalinux kernel: [50090.436302] Hardware name: Gigabyte Technology Co., Ltd. P43T-ES3G/P43T-ES3G, BIOS F7 08/20/2010
Jun  8 22:14:14 wdatalinux kernel: [50090.436305] task: ffff9503027ad280 task.stack: ffffbaedcab2c000
Jun  8 22:14:14 wdatalinux kernel: [50090.436310] RIP: 0010:kmem_cache_alloc+0x69/0x160
Jun  8 22:14:14 wdatalinux kernel: [50090.436311] RSP: 0018:ffffbaedcab2f978 EFLAGS: 00010206
Jun  8 22:14:14 wdatalinux kernel: [50090.436314] RAX: 0000000000000000 RBX: 0000000001011200 RCX: 0000000000080f8e
Jun  8 22:14:14 wdatalinux kernel: [50090.436315] RDX: 0000000000080f8d RSI: 0000000001011200 RDI: 000045e960003320
Jun  8 22:14:14 wdatalinux kernel: [50090.436317] RBP: 424d53fe30010000 R08: ffffdaedbfc83320 R09: 0000000000000000
Jun  8 22:14:14 wdatalinux kernel: [50090.436319] R10: ffffbaedcab2fb98 R11: 0000000000000002 R12: 0000000001011200
Jun  8 22:14:14 wdatalinux kernel: [50090.436320] R13: ffffffffa418c05a R14: ffff95044ac8b700 R15: ffff9503027ad280
Jun  8 22:14:14 wdatalinux kernel: [50090.436322] FS:  00007fbd2d676800(0000) GS:ffff95045fc80000(0000) knlGS:0000000000000000
Jun  8 22:14:14 wdatalinux kernel: [50090.436324] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun  8 22:14:14 wdatalinux kernel: [50090.436326] CR2: 00007fbd2d715000 CR3: 000000019f796000 CR4: 00000000000006e0
Jun  8 22:14:14 wdatalinux kernel: [50090.436328] Call Trace:
Jun  8 22:14:14 wdatalinux kernel: [50090.436334]  mempool_alloc+0x6a/0x180
Jun  8 22:14:14 wdatalinux kernel: [50090.436355]  cifs_small_buf_get+0x16/0x30 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436372]  small_smb2_init+0x5d/0xb0 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436376]  ? __find_get_block+0xb7/0x2d0
Jun  8 22:14:14 wdatalinux kernel: [50090.436393]  SMB2_open+0xb5/0xc20 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436395]  ? kmem_cache_alloc_trace+0x151/0x160
Jun  8 22:14:14 wdatalinux kernel: [50090.436411]  ? cifs_strndup_to_utf16+0xc4/0x110 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436428]  ? smb2_open_op_close+0x9c/0x270 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436444]  smb2_open_op_close+0x9c/0x270 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436447]  ? __alloc_pages_nodemask+0x122/0x290
Jun  8 22:14:14 wdatalinux kernel: [50090.436464]  smb2_query_path_info+0x6f/0xf0 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436479]  cifs_get_inode_info+0x66b/0xb90 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436482]  ? __kmalloc+0x133/0x1a0
Jun  8 22:14:14 wdatalinux kernel: [50090.436496]  ? build_path_from_dentry_optional_prefix+0xd6/0x3d0 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436511]  cifs_revalidate_dentry_attr+0x1ce/0x240 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436527]  cifs_getattr+0x5c/0x190 [cifs]
Jun  8 22:14:14 wdatalinux kernel: [50090.436530]  ? security_inode_getattr+0x42/0x60
Jun  8 22:14:14 wdatalinux kernel: [50090.436533]  vfs_statx+0x8b/0xe0
Jun  8 22:14:14 wdatalinux kernel: [50090.436535]  SYSC_newlstat+0x39/0x70
Jun  8 22:14:14 wdatalinux kernel: [50090.436539]  do_syscall_64+0x6e/0x120
Jun  8 22:14:14 wdatalinux kernel: [50090.436543]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
Jun  8 22:14:14 wdatalinux kernel: [50090.436545] RIP: 0033:0x7fbd2bc091f5
Jun  8 22:14:14 wdatalinux kernel: [50090.436547] RSP: 002b:00007ffdeb4c81a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000006
Jun  8 22:14:14 wdatalinux kernel: [50090.436549] RAX: ffffffffffffffda RBX: 00007ffdeb4c8340 RCX: 00007fbd2bc091f5
Jun  8 22:14:14 wdatalinux kernel: [50090.436551] RDX: 00007ffdeb4c8210 RSI: 00007ffdeb4c8210 RDI: 00000000015cc978
Jun  8 22:14:14 wdatalinux kernel: [50090.436553] RBP: 00007ffdeb4c8338 R08: 00000000015cd470 R09: 0000000000000001
Jun  8 22:14:14 wdatalinux kernel: [50090.436554] R10: 00000000015cc97b R11: 0000000000000246 R12: 00007ffdeb4c8348
Jun  8 22:14:14 wdatalinux kernel: [50090.436556] R13: 00007ffdeb4c8338 R14: 00000000015e22bb R15: 0000000000000002
Jun  8 22:14:14 wdatalinux kernel: [50090.436558] Code: 08 65 4c 03 05 a1 c6 e1 5b 49 83 78 10 00 49 8b 28 0f 84 be 00 00 00 48 85 ed 0f 84 b5 00 00 00 49 63 46 20 48 8d 4a 01 49 8b 3e <48> 8b 5c 05 00 48 89 e8 65 48 0f c7 0f 0f 94 c0 84 c0 74 ba 48
Jun  8 22:14:14 wdatalinux kernel: [50090.436589] RIP: kmem_cache_alloc+0x69/0x160 RSP: ffffbaedcab2f978
Jun  8 22:14:14 wdatalinux kernel: [50090.436592] ---[ end trace 67f999ec2ea0577a ]---

mount output example:

Code: Select all

//nas.local/soft on /var/archive/soft type cifs (rw,relatime,vers=3.0,cache=strict,username=guest,domain=,uid=0,noforceuid,gid=0,noforcegid,addr=192.168.11.33,file_mode=0666,dir_mode=0755,soft,nounix,serverino,mapposix,noperm,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1,user=guest,_netdev)

fstab entry:

Code: Select all

//nas.local/soft /var/archive/soft cifs iocharset=utf8,user=guest,password=,noperm,file_mode=0666,_netdev,nofail,vers=3.0 0 0

I know that there was change in 4.14.120 related to cifs, but it seem I am the only one, who has this problem. Could somebody help me to fix it? For a start, rsize and wsize looks too big, but I don't know where these defaults came from.
This configuration works fine on 4.14.119

[doktor5000]

FWIW, I only see once change in 4.14.121 regarding cifs, maybe that introduced the breakage ? See e.g. https://cdn.kernel.org/pub/linux/kernel ... g-4.14.120

For a start, rsize and wsize looks too big, but I don't know where these defaults came from.

For the fstab entry, please mention also how you added it there (manually ór via MCC or ...)

Also, please add some information on the device that exports this SMB share, might not be necessary but more context information is always helpful.

Totally apart from that, I'd suggest writing this as a bugreport as this seems to be breakage introduced upstream. See https://wiki.mageia.org/en/How_to_report_a_bug_properly

[wdata]

Thank you for looking into the problem. I did add the entries manually, there is the ZyXEL NAS-326 on the other side.
My point in creating the topic here first before opening the bug was to check whether it is the configuration issue (despite of course the configuration issue should not lead to the GPF in the kernel module). My configuration is indeed very old, the system is alive more than 10 years and was first set up as one of Mandriva's with yearly update to mga6. I admit I have been playing with Samba/CIFS configs a lot to gain better perfomance so it could be the consequences.

[doktor5000]

FWIW, regarding the mount options, those should come systemd-fstab-generator, see the documentation at https://www.freedesktop.org/software/sy ... rator.html and https://www.freedesktop.org/software/sy ... mount.html

[doktor5000]

Issue was solved with 4.14.122

commit ddbe4b02aeca52900bb6965c533044b59924e37c
Author: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Date: Thu May 23 10:47:04 2019 +0200

Revert "cifs: fix memory leak in SMB2_read"

This reverts commit c54a881d793e3eea2a1b1460c5778b22128821ea which is
commit 05fd5c2c61732152a6bddc318aae62d7e436629b upstream.

Lars writes:
This patch should not be in 4.14-stable because
088aaf17aa79300cab14dbee2569c58cfafd7d6e was for 4.18+.

Now we have a double-free crash in SMB2_read because there are 2
calls to cifs_small_buf_release in the error path.

It was a mistake to backport it this far, so let's revert it.

[doktor5000]

Good catch. You should really write a bugreport, to ensure this fix gets into one of the next kernel updates.

[wdata]

Thanks for suggesting, but no need to. The next kernel 4.14.127 has came already with the fix.