I owe RapidElectronic for his excellent comment regarding the new
dd command. Beginning with version 8.24, you can specify the parameter, status=progress, to the dd command. By using this new parameter, you no longer need to send an explicit USR1 signal to the dd process to request an update of the disk copy statistics; it will automatically print periodic updates in the standard output.
$ sudo df if=/dev/sda of=/dev/sdb status=progress
Note that sending the USR1 signal will continue to work for the new dd.
(Original article)
dd is a popular, generic command-line tool for copying files from 1 location to another. It is often used to copy entire disk images.Like many Linux command line tools, it operates silently unless something unexpected happens. Its lack of visual progress feedback is a nice feature for scripting. However, it can leave you wondering about its progress if you are interactively dd-copying a large disk.
To illustrate, you run the following (valid, but perhaps not very useful) dd copy:
$ dd if=/dev/random of=/dev/null bs=1K count=100
It will run for a few minutes as it copies (and immediately discards) 100 blocks of randomly generated data, each of size 1 KB.
To get a progress report while dd is running, you need to open another virtual terminal, and then send a special USR1 signal to the dd process.
First, find out the process id of the dd process by running the following in the new virtual terminal.
$ pgrep -l '^dd$' 8789 dd $
To send the USR1 signal to the dd prcoess:
$ kill -USR1 8789 $
Note that as soon as the USR1 signal is detected, dd will print out the current statistics to its STDERR.
$ dd if=/dev/random of=/dev/null bs=1K count=100 0+14 records in 0+14 records out 204 bytes (204 B) copied, 24.92 seconds, 0.0 kB/s
After reporting the status, dd will resume copying. You can repeat the above kill command any time you want to see the interim statistics. Alternatively, you can use the watch command to execute kill at a set interval.
$ watch -n 10 kill -USR1 8789
P.S.
Other articles from this blog on the dd command:
Create files of a given size
71 comments:
Thank U!
thanks too. i was trying SIGINFO but that isnt implemented on linux. a wiki article said SIGPWR is SIGINFO - after 2 hrs of cloning my new HDD i managed to kill dd because of a power problem. Gutted. SIGPWR wiki article is a troll :( nice watch idea - i was going to script it :D
Thanks. This was very helpful!
Use "pv" command from "pv" package.
Try for example:
pv -ptre FILE | dd of=FILE bs=1M
Gives a nice output with a progressbar, good for writing cf cards from images.
Just use dcfldd instead. It is an "enhanced version of dd for forensics and security" and is readily available in most *nix package mangers!
you can use killall -s USR1 dd
can be ever better, using only tr as an extracommand:
dd if=/dev/random of=/dev/null bs=1K count=1000
then:
pgrep -l '^dd$' | tr -d [:alpha:]
12320
watch -n 1 kill -USR1 $(pgrep -l '^dd$' | tr -d [:alpha:])
and output will be:
dd if=/dev/random of=/dev/null bs=1K count=1000
0+85 records in
0+85 records out
709 bytes (709 B) copied, 93.5436 s, 0.0 kB/s
0+113 records in
0+113 records out
933 bytes (933 B) copied, 119.764 s, 0.0 kB/s
0+113 records in
0+113 records out
.
.
and so on ...
Handy tip, thanks!
/hERB
Thanks alot
If you're the only one on the system, just do:
pkill -USR1 dd
Saves yourself a step!
By the way, using SIGUSR1 on MacOS will kill dd. You need to use SIGINFO instead.
Careful, I did this on Ubuntu 8.10 and it just killed the process >:(
Thank you! Very useful piece of advice! :)
Thanks a lot. Nice tip! Just helped me to see the status if my prolong running 'dd'!
Dipak
1. Nice hack :)
2. Ubuntu doesn't really count as Linux :)
Thanks. This was exactly what I was looking for.
Ubuntu definitely counts as Linux.
You are a genius, and I thank you so very much. I am glad that I don't have to kill dd and start it over again with a --verbose flag!
Neat!
thanks, very helpfull - I've used:
while sleep 10; do sudo kill -USR1 [PID]; done
Good stuff, very helpful while low-level copying a 750Gb drive. Thanks!!
thanks!!! was very helpfull...
You made my day. Thanks.
very helpful, thank you
I did the following:
$dd if=/dev/sda of=/dev/sdb conv=sync
and on another terminal:
$watch -n 480 kill -USR1 4292
The final message was:
488397168+0 records in
488397168+0 records out
250059350016 bytes (250 GB) copied, 53809 s, 4.6 MB/s
53809 s = 14 h 56 mn 49 s
Before copying I got :
$ fdisk -l
Disk /dev/sda: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x35e920cb
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 6092 48829567+ 83 Linux
/dev/sda3 6093 13387 58597087+ 7 HPFS/NTFS
/dev/sda4 13388 30401 136664955 f W95 Ext'd (LBA)
/dev/sda5 13388 21036 61440561 7 HPFS/NTFS
Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xe8cd189d
Device Boot Start End Blocks Id System
/dev/sdb1 1 60801 488384001 c W95 FAT32 (LBA)
After copying I got :
$ fdisk -l
Disk /dev/sda: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x35e920cb
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 6092 48829567+ 83 Linux
/dev/sda3 6093 13387 58597087+ 7 HPFS/NTFS
/dev/sda4 13388 30401 136664955 f W95 Ext'd (LBA)
/dev/sda5 13388 21036 61440561 7 HPFS/NTFS
Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x35e920cb
Device Boot Start End Blocks Id System
/dev/sdb1 * 1 13 104391 83 Linux
/dev/sdb2 14 6092 48829567+ 83 Linux
/dev/sdb3 6093 13387 58597087+ 7 HPFS/NTFS
/dev/sdb4 13388 30401 136664955 f W95 Ext'd (LBA)
/dev/sdb5 13388 21036 61440561 7 HPFS/NTFS
Thanks, i knew about the USR1 trick, but learnt about the watch command from your blog. It's really useful.
Thx! :)
Exactly what I needed. Thank you!
Thanks. but you need to explain what STDERR means. You don't explain what STDERR is
STDERR is error message destination, a console/monitor by default. Thanks for the article.
Typing ^T (CTRL-T) in the console where dd is running should achieve the same result as the USR1 signal.
Regards
Just thank you for this tip.
Thumb up! Thank you mate!
Really.. nice job! This is what I wanted.
Thank You.
http://webhoststore.tk
Thanks for the help!
Excellent, thank you Master!
Warning: using the USR1 signal on Mac OSX kills dd!
Thank you So Much! I was in pain because I didn't knew if the command was working correctly or not.... i am zeroing my hard drive.Thank you again for your advice.
Warning: using the USR1 signal on Mac OSX kills dd! Use this on OSX:
killall -SIGINFO dd
or (if dd was started as sudo)
sudo killall -SIGINFO dd
dd if=/dev/zero of=/dev/null & pid=$! ; watch -n 3 kill -USR1 $pid
:c)
Thanks. This was very helpful!
I just did
sudo kill -USR1
on Mac, It killed the process!! Damn! it was running more than 1 hour :( Lost it!
I just used it after booting my machine from System Rescue CD and using the dd command to clone my disk to another disk of the same size. Worked like a charm. Thank you for the tip!
Thank you!!! Had many months that I was seeking for this tip!!! tks :))
@iambaskar
yeah, I just did that too, and it killed the process w/o restarting. Also on a Mac.
@Joel Riggs and Mac OS friends: check the man page of your dd implementation. In Linux, "dd" is part of the GNU coreutils package. The man page in a recent Linux system says:
Sending a USR1 signal to a running 'dd' process makes it print I/O sta‐
tistics to standard error and then resume copying.
$ dd if=/dev/zero of=/dev/null& pid=$!
$ kill -USR1 $pid; sleep 1; kill $pid
Other "dd" imprementation may not handle an USR1 signal in which case the process will terminate.
Everything is in the manuals (TM).
Cool stuff. Thank you.
dd if=/dev/sda1 bs=1M |pv -brtp -s 500M| gzip -1 > /home/boot-backup.gz
Cool, and thank you!
Thanks for the info!
I'm in the process of dd a 300GB disk. With help of
$ pgrep -l '^dd$'
8789 dd
$
and
$ kill -USR1 8789
I find I have 10 hours still to wait to complete dd from the 5 hours so far.
I am hindered by the fact I am booting from USB HDD and copying files to another HDD from laptop HDD.
@ 5.9MB/s
As a result I can now kick back and get on with something else rather than staring at what appeared to be a Terminal window doing nothing :)
I forgot to add.
I had to use sudo kill -USR1 8789
Try out **pv** or pipeviewer :D
You could just do
watch -n 1 sudo kill -USR1 `pidof dd`
Dooooood. Thats awesome. THANKS!
Excellent info - really useful - I'm trying low level format a flash drive, and this really helps to tell me how far its got.
err, can't you simply press Ctrl+T in the terminal running dd?
That works on FreeBSD and other BSDs.
you could also use:
watch -n 10 kill -USR1 `pidof dd`
If your device has no activity lights, you've no idea that
anything's even happening with "dd" :(
The progress info that "dcfldd" shows is just what you need.
I've now got this in my .zshrc: -
## Conditional Aliases
## dd -> dcfldd
if type dcfldd 2>&1 >/dev/null ; then
alias dd='dcfldd'
fi
Thanks!
On OpenSUSE this seems to work only partially. When I was reading from a HDD to file it worked like a charm but when I was writing it back to another HDD it killed the process. Very strange.
thank u!
Thank you.
Clearest description and works.
I had "bash" errors so had to add "sudo" to the front.
Very useful. Thank you for this post!
Thank you very much friend, that really works :-)
Thank you man, this is very usefull!!! :) -USR1 and watch will become my new friends.
Hope you check my project, it has information about technology and virtual education in Argentina. hugs!!
I don't know if you still maintain this page and if you know that this is page #1 when your searching "dd progress".
So maybe you could enhance your post that your post is more useful than the comments below. ;)
I would recommend this e.g. command:
while true; do sudo kill -USR1 $(pgrep '^dd$'); sleep 5m; done
where sudo is just needed if you e.g. dd-ing from /dev/sda and the loop just if you want it to repeat by itself. Sure it's also possible with watch... but at least
kill -USR1 $(pgrep '^dd$')
is a really useful combination. This maybe will have some issues when using multiple dd's, but whatever. :) Guess that's easy to fix then on the fly.
Just press CTRL+t to see the interim statistics
New version of coreutils (8.24) adding a status progress to dd tool:
Usage on Xubuntu 15.10:
Open terminal shell and type these commands:
wget ftp://ftp.gnu.org/pub/gnu/coreutils/coreutils-8.24.tar.xz
tar -xf coreutils-8.24.tar.xz
cd coreutils-8.24
./configure && make -j $(nproc)
Run dd as root:
sudo ./dd if=/dev/sdc of=/dev/sda conv=noerror status=progress
You will see: Bytes, Seconds and Velocity (Bytes/seconds)
To check versions of dd:
Native:
dd --version
New (cd coreutils-8.24/src):
./dd --version
Thanks, Rapid Electronic. I've updated this article with the new information you provided.
Thanks, I don't use linux much often and everytime I do I wonder how to check dd status :-)
Anyway, you should mention that you're talking about GNU version of dd, which is pure linux thing. BSD and other unices (including OS X, which is based on Darwin, which is Apple derivate of FreeBSD) use another version of dd and kill and everything. So the INFO kill signal and Ctrl+T keypress are lethal for linux/GNU version of dd!
Little flame about Ubuntu:
Ubuntu definitely is linux, but.. It's linux for (dumb) masses. Folks who know linux rarely install it on their machines. Ordinary linux guy uses Debian, Slackware, RedHat or OpenSUSe. Some maniacs use Arch. I'd like to watch Ubuntu user installing Arch linux or set up devices on any other distro :-)
Shouldn't it be $ sudo dd if=/dev/sda of=/dev/sdb status=progress instead of $ sudo df if=/dev/sda of=/dev/sdb status=progress ?
You can add watch to the kill command to repeat the command.
For exemple every 60 seconds :
watch -n 60 kill -USR1 2334
Post a Comment