How to create a large number of files in Linux (thousands or millions)

Do you need an ultra fast way to create a large number of files in Linux? Perhaps, you are doing some load testing for an application and you need to create 1000 or even 1,000,000 files in the matter of seconds. Well, here is how to do it!

There are two parts to creating these files. First, is creating a single master file that contains the data that the thousands/millions files are based on. Second, splitting this master file into the total number of files that you require.

How to create the master file

1. Determine the number of files and the size of each file that you require
2. Multiply the total number of files times the size (in bytes). For example: If you want to create 10000 files that are 10 bytes each, do 10000 * 10 = 1,000,000. This represents the size of the master file that is needed.
3. To create this master file, run the command:

dd if=/dev/zero of=masterfile bs=1 count=1000000

4. The above command will create a 1 MB file called ‘masterfile’. This file contains all 0’s. If you would like it to contain random binary data, use /dev/urandom

To split the master file into thousands of pieces

1. Now that the master file is created, you can now use this to generate the desired 10,000 files that are 10 bytes each.
2. Run the command:

split -b 10 -a 10 masterfile

The -b option specifies the size in bytes of each file. The -a option defines the length of the filename of the new files +1 (-a 10 means create a 11 character long filename)

And that’s all there is to it! If you need help with any variation to this, please post it as a comment.