I'm backing up my work NAS to a 3 TB USB drive using "cp -Rpnv /src /dest". There are a ton of duplicates (~200G at last count). If I delete the duplicates from the backup, next time I run it, it will fetch them all over again (this is terribly slow over 100BT, and it makes the NAS almost unusable for the duration).

Is there a way to exclude deleted duplicate folders from future backups by using eg a text list (I suppose I could just paste the list into the terminal), such as with tar?

cp? What is this, 1980? How about rsync or rsnapshot?

-u

Update. Only updates the dest file if the source file is newer.

edit: or maybe I didn't read your post properly, that might not be what you want.

Haven't needed to work with those, so I don't know them. A quick scan indicates rsync may not be appropriate for our needs:

1. No files are edited, new files are created. Mostly image and graphics formats.

2. I'm not sure our aging Linkstation NAS supports rsync over afp (or at all). The compression thing would pile overhead on the pitifully underpowered NAS cpu. A lot of the files are already compressed (jpg, pdf etc).

3. I'm not sure this wouldn't turn out to be a much bigger headache to deal with. I'm looking for a qwikfix(TM).

4. We will probably need to to replace (and upgrade) the NAS sometime in the fairly near future (I've advised against replacing drives in it). At that point I will review rsync and other backup options.

Seconding rsync. It can very easily only copy what has changed, even if in your case this is new files.

AFP being Apple File Protocol? If the NAS also allows SSH use that, otherwise rsync will happily copy files as long as you can mount the AFP volume on a Linux box. I've never done it, but I'd be very surprised if you can't mount the AFP volume in Linux

Finally, rsync is the quick fix:
 

rsync -arz /src /dest

Only thing to note is to pay attention of trialling slashes on the source and destination. If you leave off the traling slash, rsync will create the last folder of path in the destination, but if you include the trailing slash on the source, rsync will skip that folder and copy subfolders and files of that folder directly to the destination.

Both volumes are formatted HFS+. Doing this on a linux box isn't going to work. Weirdo file and folder names that include eg "/".

HFS+ mounts (readonly) fine on Linux. Read-write is not as easy, but as you're copying data from the volume you shouldn't need to worry, except of course when restoring the backup. Natch.

Forward-alashes, like any other special character (space etc.), shouldn't be a problem as long as you escape then with a backslash.

If and when you replace the NAS, probably best to pick a common filesystem for the volumes.

http://unix.stackexchange.com/questions/118521/copy-files-modified-after-specific-date-using-cp-switches

Is that  supposed to mean naturally?

Natch.

Snatch.

Saying natch makes you guys sound like hipster rednecks.

http://celebrityrepresentation.com/wp-content/uploads/2012/11/MAX1.jpg

Fucking Jethro & the seement pond.

Mister Drysdale!