Comments on: Practical Compressor Test http://blog.grandtrunk.net/2004/07/practical-compressor-test/ Mon, 06 Feb 2012 08:58:31 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: nctritech http://blog.grandtrunk.net/2004/07/practical-compressor-test/comment-page-1/#comment-4310 nctritech Tue, 21 Dec 2010 17:39:32 +0000 http://blog.grandtrunk.net/2009/12/practical-compressor-test/#comment-4310 I noticed that this article is quite old, but I have some comments that would probably be very helpful to any future benchmarks. You should have combined more varied file formats, particularly executables, before drawing the conclusions here. In my own independent tests when checking each of these algorithms for suitability in creating software packages for our Tritech Service System distribution of Linux, I compressed the entire in-RAM filesystem tree for the OS into a .tar archive, then compressed that .tar with a variety of formats and compression levels, and used an automated script to count the raw over-the-network decompression times involved from an unloaded server's tmpfs exported and mounted over CIFS; here's the general sort of command I used: time lzop -dc /mnt/cifs/fs_lzop-1.tar.lzo > /dev/null Because I am compressing a combination of binary files, text files, image files/pixmaps, and small media files such as .wav files, I'm generating a somewhat more realistic average this way. Having done this from a tmpfs instance on a TOTALLY unused server and un unused network is extremely important, since this eliminates the chance that disk issues or network usage will taint the tests. What I found is that lzop -1 is by far the best way to get anything shuffled over any LAN and immediately decompressed very quickly, even if the decompressor was an old Pentium II, and regardless of 10/100/1000 Mbit negotiated link speed. However, for minimizing space consumption, or for very slow links, xz wins hands down despite taking about 11x-12x the time of lzop for decompression, specifically xz -e -2. bzip2 is useless. lzma is pointless since xz is very slightly better and is exactly the same on time. Various gzip levels make a wonderful compromise between the two. In the end, it comes down to the purpose. If you're shooting data across a LAN, lzop is almost guaranteed to drastically increase the performance of that LAN transfer; however, if you're sending a 110MB filesystem tree over the Internet to some guy with 1.5Mbit DSL, you're better off accepting a 10s decompression penalty if it means a 20MB (about 2 minutes of transfer time at 1.5Mbit) download savings. I noticed that this article is quite old, but I have some comments that would probably be very helpful to any future benchmarks.

You should have combined more varied file formats, particularly executables, before drawing the conclusions here. In my own independent tests when checking each of these algorithms for suitability in creating software packages for our Tritech Service System distribution of Linux, I compressed the entire in-RAM filesystem tree for the OS into a .tar archive, then compressed that .tar with a variety of formats and compression levels, and used an automated script to count the raw over-the-network decompression times involved from an unloaded server’s tmpfs exported and mounted over CIFS; here’s the general sort of command I used:

time lzop -dc /mnt/cifs/fs_lzop-1.tar.lzo > /dev/null

Because I am compressing a combination of binary files, text files, image files/pixmaps, and small media files such as .wav files, I’m generating a somewhat more realistic average this way. Having done this from a tmpfs instance on a TOTALLY unused server and un unused network is extremely important, since this eliminates the chance that disk issues or network usage will taint the tests.

What I found is that lzop -1 is by far the best way to get anything shuffled over any LAN and immediately decompressed very quickly, even if the decompressor was an old Pentium II, and regardless of 10/100/1000 Mbit negotiated link speed. However, for minimizing space consumption, or for very slow links, xz wins hands down despite taking about 11x-12x the time of lzop for decompression, specifically xz -e -2. bzip2 is useless. lzma is pointless since xz is very slightly better and is exactly the same on time. Various gzip levels make a wonderful compromise between the two. In the end, it comes down to the purpose. If you’re shooting data across a LAN, lzop is almost guaranteed to drastically increase the performance of that LAN transfer; however, if you’re sending a 110MB filesystem tree over the Internet to some guy with 1.5Mbit DSL, you’re better off accepting a 10s decompression penalty if it means a 20MB (about 2 minutes of transfer time at 1.5Mbit) download savings.

]]> By: Wim http://blog.grandtrunk.net/2004/07/practical-compressor-test/comment-page-1/#comment-3986 Wim Sun, 12 Dec 2010 09:15:44 +0000 http://blog.grandtrunk.net/2009/12/practical-compressor-test/#comment-3986 It's a free theme call MistyLook (with some of my own CSS tweaks...) It’s a free theme call MistyLook (with some of my own CSS tweaks…)

]]> By: Georgetta Rosazza http://blog.grandtrunk.net/2004/07/practical-compressor-test/comment-page-1/#comment-3714 Georgetta Rosazza Mon, 06 Dec 2010 00:19:26 +0000 http://blog.grandtrunk.net/2009/12/practical-compressor-test/#comment-3714 I appreciate this wordpress theme, i hope you don't mind me asking but is it a free design or did you buy it? Thanks! I appreciate this wordpress theme, i hope you don’t mind me asking but is it a free design or did you buy it? Thanks!

]]>