• Alexios
  • recipes
  • unix

You have a deep directory tree of files, and you need to flatten it into a single directory, so that there are no files in subdirectories, and all the files are in the same place.

all the filenames are unique. If they're not, some will get overwritten (using the cp command), or not linked (using the ln command).

Solution

cd /ROOT/DIRECTORY/OF/TREE
find . -type f -print0 | xargs -0 -I{} cp -n {} /FLAT/DIRECTORY

You obviously need to set $ROOT_DIRECTORY_OF_TREE to the root directory with the files you want to flatten (or just replace it in the cd command).

The same holds for the $FLAT_DIRECTORY, which is the destination.

Discussion

The command find constructs a list of all files, and feeds it to xargs, which builds a suitable command line and executes it for each file (one at a time, in this example).

The dot (.) indicates that find should start with the current directory. On GNU find (used on Linux), this is the default and can be left out. On every other version of the utility (e.g. on MacOS), you have to include it.

Options -print0 (in find) and -0 (in xargs) ensure all special characters are handled properly (except the two characters that no Unix can deal with in a filename, namely NUL and /).

The -I%%% option does all the magic — it replaces subsequent instances of the string %%% with files from the standard input, one by one.

If your destination directory is on the same filesystem as the source directory tree, you're better off using ln(but not ln -s) instead of cp. It's much faster and uses up just one inode and not a single byte more.

An added bonus of ln is it won't overwrite existing files by default, so you'll get an error message for each duplicate filename (i.e. filenames breaking the precondition above).