Linux/Unix File Copy Trick

I have a need for this hack every now and then, and I *always* forget it for some reason, so I’m putting it here for safe keeping. I’ll start a “hacks” category, too, so I can locate these quickly, and so you can too :) So, here’s the hack:

[jonesy@cranford testing]$ ls
bar  foo
[jonesy@cranford testing]$ ls foo
1  2  3
[jonesy@cranford testing]$ ls bar
1  2  3  4  5  6
[jonesy@cranford testing]$ yes n | cp -i bar/* foo 2>/dev/null
[jonesy@cranford testing]$ ls foo
1  2  3  4  5  6
[jonesy@cranford testing]$ ls bar
1  2  3  4  5  6

I’m piping a constant “no” to the “cp -i” command, which asks you if you want to overwrite (or not) any files that already exist in the target directory. You don’t have to send STDERR to /dev/null — there’s just some messy output if you don’t. The end result is that the copy command will only copy files into the destination directory if they don’t already exist. It’ll skip files that exist in both directories (Well, technically the *copy* command won’t, but the operation will when you do it this way).

Of course, I could just forcibly overwrite the directory contents, but I don’t know if the files that exist in both directories are identical. Or I could move one sideways and the other into place, but the same issue exists.

  • Billings

    Why not use the -n switch to cp?

  • m0j0

    I think you mean the ‘-u’ switch. There is no ‘-n’ switch. The reason I don’t use ‘-u’ is because I don’t want to worry about checking dates and ages or running touch or any of that nonsense — I *do not* want files that already exist in the destination directory to be overwritten regardless of their age. I just want files that aren’t already there added.

  • Billings

    I mean the -n switch, which both my BSD cp and GNU cp systems have. I suppose it’s not POSIX so it’s not portable.

  • jinks

    from cp –help:
    -n, –no-clobber do not overwrite an existing file (overrides a previous -i option)

    this is the gnu version. If you are on Solaris, the gnu-tools should be available iirc. Try gcp, gnu-cp, …

  • m0j0

    Ok, well, redhat apparently doesn’t ship gnu cp, although the man page seems to indicate that it is:
    /> cp –version
    cp (GNU coreutils) 5.97

    If you log into a RH box with the coreutils package installed, you’ll find that the RH-supplied /bin/cp does not have a -n flag.

  • http://blog.friocorte.com/ goozbach

    try this:

    rsync -r –ignore-existing bar/. foo/.

    Here’s what happens when I do it on my box:

    dcarter@ord-mandell:/tmp/test$ ls foo/
    1 2 3
    dcarter@ord-mandell:/tmp/test$ cat foo/*
    foo
    foo
    foo
    dcarter@ord-mandell:/tmp/test$ ls bar/
    1 2 3 4 5 6
    dcarter@ord-mandell:/tmp/test$ cat bar/*
    bar
    bar
    bar
    bar
    bar
    bar
    dcarter@ord-mandell:/tmp/test$ rsync -r –ignore-existing bar/. foo/.
    dcarter@ord-mandell:/tmp/test$ ls bar/
    1 2 3 4 5 6
    dcarter@ord-mandell:/tmp/test$ cat bar/*
    bar
    bar
    bar
    bar
    bar
    bar
    dcarter@ord-mandell:/tmp/test$ ls foo/
    1 2 3 4 5 6
    dcarter@ord-mandell:/tmp/test$ cat foo/*
    foo
    foo
    foo
    bar
    bar
    bar

  • http://saintaardvarkthecarpeted.com/blog Saint Aardvark

    I’m with goozbach; this is the sort of thing I usually invoke rsync for.

  • m0j0

    rsync is great, but if it’s not installed, that’s a problem, and it’s not a given that it’ll be installed wherever I find myself today. It’s also not usually the case that I’m given free reign to install anything. In addition, rsync’s flags have never cemented themselves in my brain very well. I have to use the man page to do even really trivial things with rsync. I know. I’m a dunce with rsync.

  • Oubliette

    I’m here because I want to do exactly this and I have already tried rsync. The source files are on a samba share and rsync can’t see them. Well, actually, it can see them, it builds the copy list fine, but when it goes to get them it hits a “file has vanished” error for every single file. I think this might be a unicode issue, the files are written to the samba share by iTunes from a Mac, but I can’t fiddle with the code page because the share is on a dumb NAS device. I’m left with cp and this trick to not overwrite files…

    … or -n now I know it exists. It wasn’t on any of the man pages I looked at on the net and I don’t have my Ubuntu box with me :(

  • Redbrick

    Thanks for this tip, it’s an easy and quick way to sync to directories that don’t support rsync properly (e.g. certain SMB shares) and on a platform without the -n switch to cp (e.g. Debian). I sometimes like to add the -v switch to cp to make it list the files it’s copying.

  • dimangi

    I had also problems with rsync, but the cp hack worked just fine…

  • Marcos Jacoby

    What you would do (I think) is do cp -i –reply=no

    That way, it will ask if you want to overwrite the file and automatically say “no.” Check your version and give it a test run as this option will be removed very soon!

  • Marcos Jacoby

    Or you try this:
    -n, –no-clobber do not overwrite an existing file (overrides a previous -i option)

  • Mike G

    GNU coreutils version 5.97 (2006) doesn’t have the -n option available. In version 8.4 (2010) it exists.
    I never thought I’d run into a problem where I used a feature of copy that didn’t exist in an older version…
    I figured cp had been around since they invented unix/linux go figure that a new option came into being between 2006 and 2010. Times they are a changing.