High CPU load during file copy

[zugunder]

Hi,

Could anyone confirm what I see on my machine?
During copy of any big file (300-700MB) via Dolphin or Krusader the both cores of CPU are 100% loaded. I can see it in Ksysguard and via CPU plasma applet. However, top does not show anything unusual - maybe 10-15% load.
It seems to happen during copying from any FS to any FS (I checked ext4->fat, ext4->ext4, ntfs->ext4). I have not tried a set of smaller files though.
Mageia 3 64bit, KDE.

Thank you.

[wintpe]

while we are waiting for comparisons it would be nice to know what
hardware you are copying from and to.

you might think its all the same, but specifics will help, as there could be issues with a
particular sata driver, quite a few chips from various era's have this issue.

also disks could have bad blocks try running badblocks across both your drives.

how big are they what spin speed, all relevant stuff.

lastly iotop might also sheds some light on it, but before you do that try the following.

dd if=/dev/zero of=/mountpoint/afileoneachdisk count=1000000000

this will tell you the transfer speed of each disk

regards peter

[jiml8]

I think that ksysguard does not accurately take into account processor wait states.

The processor will wait on I/O of course and if it doesn't have anything else to do, it will show as waiting. Ksysguard is considering this as the processor being active (as opposed to idle) and reports it as busy.

[zugunder]

wintpe

Thank you for the reply.

I was not attentive enough, I noticed that top was actually giving rather high CPU readings during file copy too, but with some delay.
The most consuming processes are:

kswapd0 66%
X 35%
kwin 15%
kworker/ 1:2 12%

If I turn off job notifications in KDE tray, kswapd0 jumps to 80%, the second is kworker, X and kwin do not use any CPU at all.
As far as I understand, kswapd is a process managing memory buffers during file operations. So why does it consume that much? Why X and kwin are eating lots of CPU?

Transfer speed seems to be OK, at least judging from the timing of file transfer. I have only one HDD (it is a laptop) with installed Win 7 (NTFS) and Mageia 3 (ext4).
CPU load reaches 100% if I copy a file from NTFS to ext4 or within the same ext4 partition (from /home/user to /home/user/). I also observe the same 100% load if I copy a file from ext4 to a flash stick (4GB Kingston Traveler).
My HDD seems to be in a decent shape, at least the test made by GSmartContol tell me that (I attached a couple of images). I also checked it for bad blocks under Windows - negative as well. And no, I do not observe high CPU load or low transfer speeds in Windows at all.
The laptop is on Intel 4 Series chipset with built-in video:

Code: Select all

lspci
00:00.0 Host bridge: Intel Corporation Mobile 4 Series Chipset Memory Controller Hub (rev 09)
00:02.0 VGA compatible controller: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller (rev 09)
00:02.1 Display controller: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller (rev 09)
00:1a.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 03)
00:1a.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 03)
00:1a.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 03)
00:1a.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 03)
00:1b.0 Audio device: Intel Corporation 82801I (ICH9 Family) HD Audio Controller (rev 03)
00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 (rev 03)
00:1c.1 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 2 (rev 03)
00:1c.5 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 6 (rev 03)
00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03)
00:1d.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03)
00:1d.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03)
00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev 93)
00:1f.0 ISA bridge: Intel Corporation ICH9M LPC Interface Controller (rev 03)
[b]00:1f.2 SATA controller: Intel Corporation 82801IBM/IEM (ICH9M/ICH9M-E) 4 port SATA Controller [AHCI mode] (rev 03)[/b]
02:00.0 Network controller: Atheros Communications Inc. AR9285 Wireless Network Adapter (PCI-Express) (rev 01)
03:00.0 Ethernet controller: Qualcomm Atheros AR8121/AR8113/AR8114 Gigabit or Fast Ethernet (rev b0)

[zugunder]

I think that ksysguard does not accurately take into account processor wait states.
The processor will wait on I/O of course and if it doesn't have anything else to do, it will show as waiting. Ksysguard is considering this as the processor being active (as opposed to idle) and reports it as busy.

Thank you for the comment. It makes sense, but it would not this mean that ksysguard and top take their raw readings from different sources?

[jiml8]

Thank you for the comment. It makes sense, but it would not this mean that ksysguard and top take their raw readings from different sources?

I am not sure of the answer to that, but my thought is that it need not mean that at all. If you look at top, it provides details about processor usage broken down into idle, i/o wait, userspace, and supervisor space usage where ksysguard only gives a line graph or a cpu% value. Thus, top is giving a more detailed breakdown while ksysguard is giving an aggregated value.

I would contend that the aggregated value should not include i/o waits as processor active time, but this is what ksysguard is doing, apparently.

[zugunder]

jiml8

Thank you for explanations. Sorry, I actually found out that the discrepancy between ksysguard and top readings were not so dramatic (though it definitelly exists). I was not attentive enough. I corrected the second post and pyut there some numbers from top. But the question why file transfer requires so much CPU power is still open.

[zugunder]

UPD

I have no idea why it is happening, but the inconsistency between top and ksysguard (and an applet) are back - kswapd0 does not consume much.
I understand what jiml8 said about ksysguard peculiar way of calculation of CPU load, but one simple thing makes me doubt - how did it happen that no one (including developers, etc) has noticed such a big difference during very popular and simple operation like file transfer?
So, is there any other way to reliably establish the actual CPU load (except measuring real wattage of the box)?

I attached 2 representative images taken during copying a file from my HDD to a flash stick - top and iotop outputs.

[zugunder]

Hi,

Returning to my very first posting - can anyone confirm that high CPU load (as reported by Ksysguard or plasma applet) during big files copy? Or does it happen to me only?

[doktor5000]

High CPU load can generally be observed in the following cases:

Writing a large number of small files onto flash-based partitions (SD cards, usb flash drives)
Writing onto NTFS partition with either a high fill state or which are highly fragmented

BTW: Writing to NTFS partitions uses much more CPU ressources in general compared to native unix filesystems like ext4.

[zugunder]

High CPU load can generally be observed in the following cases:

Writing a large number of small files onto flash-based partitions (SD cards, usb flash drives)
Writing onto NTFS partition with either a high fill state or which are highly fragmented

BTW: Writing to NTFS partitions uses much more CPU ressources in general compared to native unix filesystems like ext4.

Well, all from above applies to my case AND copying a single 300-500MB file between the folders within my home directory on an ext4 partition. I am interested in confirmation of the latter.

[eor2004]

Hi everybody, I can confirm this behavior not only in Mageia but in PCLinuxOS too when using KDE DE, so I think Mageia is not the problem but KDE itself, for any info on my hardware specs, see my signature!

P.S. By the way my 250gb and my 500gb external hard disks that I use as backup use XFS Filesystem too!

[eor2004]

I made a bug report about this issue, here's the link: https://bugs.mageia.org/show_bug.cgi?id=11077