7zip multithreading deadlock on ~100Gb files

This forum is dedicated to testing early releases and cauldron : Howtos, tips, tricks and user global feedback and thoughts...

Helpful tip :
For bugs tracking we use : https://bugs.mageia.org = The Mageia Bug Tracker
In this bug tracker you'll find already reported bugs and you'll be able to report those you have found....

7zip multithreading deadlock on ~100Gb files

Postby Northsoft » Nov 9th, '20, 22:22

Hello, 7zip freezes on compressing big file when I specify -mx=9 -myx=9. If I call strace on running processes, I see single futex syscall in every process. If I specify -mmt=off, I face great decrease of compression speed, but no freeze. How can I troubleshoot that? 5.9.6-desktop-1.mga8, 7z 16.02 2016-05-21. Ryzen 5 3500X, 16 Gb RAM.
Northsoft
 
Posts: 18
Joined: Dec 18th, '12, 19:22
Location: Novosibirsk, Russia

Re: 7zip multithreading deadlock on ~100Gb files

Postby doktor5000 » Nov 9th, '20, 23:48

You may want to add some basic context information. What type of file are you running this on, where is it located (underlying filesystem and what physical type of storage).

Also, what do you expect to happen when you do this? Your box will basically be using 100% CPU which might be freezing e.g. your graphical desktop partially or fully, depending on what scheduler is being used for that device and if you're not using something like ionice or similar.
Cauldron is not for the faint of heart!
Caution: Hot, bubbling magic inside. May explode or cook your kittens!
----
Disclaimer: Beware of allergic reactions in answer to unconstructive complaint-type posts
User avatar
doktor5000
 
Posts: 16218
Joined: Jun 4th, '11, 10:10
Location: Leipzig, Germany

Re: 7zip multithreading deadlock on ~100Gb files

Postby Northsoft » Nov 10th, '20, 07:10

doktor5000 wrote:What type of file are you running this on

Disk image got by dd. AFAIR this issue also happened if I use 7z a -si to compress stdin.

doktor5000 wrote:where is it located (underlying filesystem and what physical type of storage).

NTFS, HDD. AFAIR this issue also happened on Ext4.

doktor5000 wrote:Also, what do you expect to happen when you do this? Your box will basically be using 100% CPU which might be freezing e.g. your graphical desktop partially or fully, depending on what scheduler is being used for that device and if you're not using something like ionice or similar.

Had no experience of freezing GUI, using default Mageia settings. Just saw that strace shows futex syscall being executed in every process, single call longs in a minute for every subprocess.
Northsoft
 
Posts: 18
Joined: Dec 18th, '12, 19:22
Location: Novosibirsk, Russia

Re: 7zip multithreading deadlock on ~100Gb files

Postby morgano » Nov 10th, '20, 09:45

Also, maybe check with a 7zip forum for similar experiences or tips.
Mandriva since 2006, then Mageia since 2011 at home & work. Thinkpad T42p T43 T60 T61 T400. Aspire 7. Workstation using LVM, LUKS, VirtualBox, BOINC, CAD
morgano
 
Posts: 642
Joined: Jun 15th, '11, 17:51
Location: Kivik, Sweden


Return to Testing : Alpha, Beta, RC and Cauldron

Who is online

Users browsing this forum: No registered users and 1 guest