Slow file copy issue

[pnbalaji]

Hi,

I am trying to copy some videos from windows partition into a USB flash drive. While copying a file of 438.8, the file got copied fast until 415.7MB and then the copy process got stuck for at least 2 minutes. See the screenshot below.

Stuck copy process

copy_issue.png (20.9 KiB) Viewed 1700 times

I don't think I had this issue before. It appears that the copy process is stuck after copying 95% of the file. If I copy 5 different files, at the end of 95% of each file, the copy process hangs and then proceeds after few minutes.

Can some one help? Is there a special copy software (like ultra copier, or teracopy for windows) that I can use?

I ran the ps -ef command to see what is running and I see that gvfsd-fuse is handling the file copy.

bneelka 1760 1 0 07:16 ? 00:00:00 /usr/libexec/gvfsd-fuse /home/bneelka/.gvfs -f -o big_writes


Thanks,
Balaji.

[doktor5000]

That is normal behaviour for most flash storage, first large bit of the copy progress will be cached, and then it takes the flash storage quite some time to actually write the stuff and provide feedback that it's synced, then the progress will continue.

[pnbalaji]

Returning to this post after a long time.

I have dumped the clonezilla ISO into a flash drive using the dd command, and the behaviour is same. The control came to the command line prompt almost immediately and every one will think that the ISO image has been written into the drive. However, when I try to eject the flash drive by unmounting it, thunar displays a message saying that "Data is being written into the drive".

It appears to me to consider this as a bug. However, I am not sure who will be responsible for this. XFCE, perhaps?

Code: Select all

[bneelka@localhost Linux]$ sudo dd if=clonezilla-live-20160529-xenial-amd64.iso of=/dev/sdc bs=1M
242+0 records in
242+0 records out
253755392 bytes (254 MB) copied, 0.394108 s, 644 MB/s
[bneelka@localhost Linux]$


Any thoughts?

Thanks,
Balaji.

[jiml8]

It is not a bug. Flash storage is notoriously slow to write. Also, most filesystems employ caching on write in order to avoid locking up an interface while the write actually occurs.

Commonly, the data to be written is cached in memory and actually written as opportunity presents (but always within 30 seconds or so, based on default Linux settings).

You can modify this behavior if you wish. Take a look at the sysctl mibs vm.dirty_ratio and vm.dirty_background_ratio to get started. You also can specify "nocache" on writing for various devices on the system (though offhand I am not sure how to do it with a USB stick...take a look at udev rules) but I suspect you will not like the user interface performance hit that results.

[pnbalaji]

I agree with the caching part. However, the control should not return to the command line prompt as the user might think the write is actually complete. If the user doesn't pay attention to the message that file manager is showing, he/she will unplug the flash drive and this will eventually damage the flash drive itself.

Thanks,
Balaji.

[doktor5000]

However, the control should not return to the command line prompt as the user might think the write is actually complete.

Then you need to change it so that it's synchronously written. But in that case you end up with other issues due to synchronous write, like this: https://lwn.net/Articles/572911/

For how to change that for dd, use the appropriate options for that, see e.g. http://askubuntu.com/questions/318186/w ... o-the-disk
and also the man page for dd, there are several options to control this behaviour, and you can even change them separately for input and output device.

See also https://www.kernel.org/doc/htmldocs/usb ... usbfs-sync for mor information.

[pnbalaji]

Thanks for the detailed info. I will take a look at the articles sometime today/tomorrow.

Thanks,
Balaji.

[wintpe]

always follow your dd or copy with the command sync.
when that returns you can be more sure that all cached data is flushed.

regards peter