Tag Archives: tar.gz

Speeding up TAR.GZ compression with PIGZ

Posted on by

Most of you probably noticed TAR.GZ compression isn’t very fast. Recently, during routine system backup I have realised TAR.GZ is limited not by disk read/write, but GZIP compression (98% of computation).

time sudo tar cpfz backup/ubuntu1404.tgz --one-file-system /

real 6m20.999s
user 6m1.800s
sys  0m19.043s

GZIP in its standard implementation is single CPU bound, while most of modern computers can run 4-8 threads concurrently. But there are also multi-core implementations of GZIP ie. PIGZ. I have decided to install PIGZ and plug it with TAR as follows:

sudo apt-get install lbzip2 pigz

time sudo tar cpf backup/ubuntu1404.pigz.tgz --one-file-system --use-compress-program=pigz /

real 1m43.693s
user 8m34.168s
sys  0m20.243s

As you can see, TAR.GZ using PIGZ on 4-cores i7-4770k (using 8 threads) is 2.5 times faster than GZIP! And you get standard TAR.GZ archive as output:)

The same applies to BZIP2 compression using LBZIP2.