Lossless audio formats comparison

A comparison of six lossless audio formats.

This is page two of a three-page article. This page covers lossless audio formats. Page one is the introduction page, which sets out the test method and lists the CDs used in the comparison. Page three covers lossy formats.

Lossless codecs and settings

This comparison examines six lossless audio formats, five of which use lossless compression. Compression modes were chosen based on recommendation by the codec's creator. Failing such a recommendation, I looked for the consensus on audio discussion forums. Failing that, I settled on the mode I felt produced the best compromise between speed and resultant file size.

Wave (.wav)

Uncompressed Wave files are some people's idea of an ideal lossless format when archiving their CD collection digitally. Because the audio files are uncompressed, the file size is larger, but the rip-and-encode process should take less time.

FLAC (.flac)

dBpowerAMP codec release 5.3, using FLAC 1.1.2.

Compression mode: 4 (medium).

FLAC is an open source codec that produces lossless compressed files. It has been around a while, and it's the format I use when I'm archiving my CD collection. Even though lossless files are relatively large, there are a small number of portable music players that support playback of FLAC files. There is also support for FLAC in several digital hi-fi systems. FLAC files support tagging to store track details.

Monkey's Audio (.ape)

dBpowerAMP codec release 7, using Monkey's Audio 3.99.

Compression mode: High.

When I published my last audio format comparison, I was worried that development on Monkey's Audio had dried up. It turns out I was wrong; the Monkey's Audio codec has seen version updates twice this year, so it's still very much in the game. Monkey's Audio files use their own APE tagging system to store details about the audio file.

OptimFROG (.ofr)

Using OptimFROG 4.506.

Compression mode: Fast.

This codec has a reputation for strong compression, so I decided to give it a try.

Shorten (.shn)

dBpowerAMP codec release 3, using Shorten 3.5.1.

Compression mode: lossless default.

Shorten is seemingly very popular with people who trade in recordings of live music, and there are whole discussion groups dedicated to the subject of where to find SHN files of particular artists.

WavPack (.wv .wvc)

dBpowerAMP codec release 4, using WavPack 4.3.1.

Compression mode: Fast.

Performance tables seemed to put this codec's speed in the same neighbourhood as FLAC's, and I wanted to see for myself. WavPack is also an open source codec.

Raw data

Rip-and-encode times

Firstly, the time taken to rip each CD album and encode it using each lossless codec. The time recorded was that reported by dBpowerAMP after the rip-and-encode process of each album was complete.

Lossless file sizes

Next, the total file size of each album stored in each lossless format.

Lossless compression rates

Finally, a look at how well each album compressed on average. This is measured by dividing the average file size of each album encoded to a compressed lossless format (so we aren't including the uncompressed Wave format) and dividing it by the number of seconds in that album's running time. This gives us the average lossless compression rate for each album, in bytes per second.

Analysis of the data

Average rip-and-encode times

Overall, Wave is still the fastest lossless format to rip and encode a CD to. But I was shocked to see FLAC finish faster than Wave on three of the albums. FLAC took only seven percent longer than Wave, on average. A great improvement over the performance I saw from FLAC in my previous audio format comparison.

WavPack was next fastest, but couldn't keep up with FLAC. An extra twenty-five seconds taken for each album isn't too bad, unless you've got a big stack you want to rip and encode all in one go.

Shorten came in next, at an average of over twenty seconds longer than WavPack.

Monkey's Audio came in fifth, taking almost fifty percent longer to rip and encode the albums than it took to rip and encode to uncompressed Wave.

OptimFROG took an awful long time to encode each album. It was the slowest on every album tested, and slowest by quite some time. On average, it took OptimFROG two-and-a-half minutes longer to rip and encode a CD than Monkey's Audio, and over two-and-a-half times the amount of time taken to rip and encode to Wave. If OptimFROG had offered a faster compression mode, I'd have taken it. But Fast mode is the fastest on offer. Five-and-a-half minutes feels like a long time to rip and encode a single album.

Average album sizes

Monkey's Audio produced the smallest lossless file size for every one of the ten albums tested. And its performance seems to have improved a little since my previous format comparison. On average, an album encoded into Monkey's Audio takes up only fifty-nine percent of the space that the same album encoded into uncompressed Wave would.

OptimFROG comes in second, producing average album sizes only a little larger than Monkey's Audio, at sixty percent of the size used by Wave.

FLAC comes in third, and also seems to have improved its ability to compress since my last format comparison, this time producing an average album file size only sixty-two percent that of uncompressed Wave.

WavPack is very close behind FLAC, producing an average file size of sixty-three percent that of Wave. One interesting observation is that WavPack files took up slightly more space than FLAC for every album except The Essential Classics Collection. The smooth nature of classical music perhaps agrees more with WavPack than it does FLAC. The difference was still slight, however.

Shorten offered the least effective compression in this comparison. On average, the albums compressed with Shorten took up sixty-seven percent of the space that Wave used. This is roughly the performance that FLAC turned in during my last comparison.

Wave, being uncompressed, is not surprisingly the least space-efficient format. The albums tested in this comparison ended up using an average 678MB per album when stored in uncompressed Wave format, which would take up almost seventy percent of a 1GB flash disk in a portable music player. (If anyone were mad enough to use uncompressed Wave on a portable player.)

Each format held the same order of effectiveness for every album except The Essential Classics Collection, which caused WavPack to be slightly more effective than FLAC for just that one album.

Average compression rates

The Essential Classics Collection (Volume 1) is by far the most successfully compressed of the ten albums. Its average lossless compression rate is 76KB per second of the album. I assume this is because classical music is much slower and smoother than modern studio music, and because classical music features much more silence and near-silence. Silence ought to be extremely easy to compress for any codec.

Endtroducing is not as compressible as the classics, but it still needs fewer bytes per second than the other eight albums, and by some margin. This album features fairly orderly and laid-back tracks, as well as sparse verbal tracks that ought to be easy to compress.

Most of the remaining albums loiter in the 110–120 kilobytes per second range. But three albums demand more. Barrio Fino, Resident Evil: Apocalypse, and The W all need over 123KB per second. These albums all feature tracks that are full of noise that doesn't let up. Barrio Fino is thick with dance sounds, Resident Evil: Apocalypse vibrates with wailing guitars, and The W is an album throbbing with varying basslines and samples. Such densely-packed audio is apparently difficult to compress losslessly.

Conclusion

In my opinion, it's a choice between FLAC and Monkey's Audio. And the choice depends on whether you value time or disk space more.

FLAC is now so fast that the time saving you'd make by ripping and encoding to uncompressed Wave is barely worth thinking about. FLAC took the same amount of time or less than Wave on four of the ten albums. Combine that with the fact that encoding to FLAC will save you almost forty percent of the space that Wave uses, and you'd be mad to keep using Wave to rip and encode CDs on the basis of saving time.

If disk space is more important to you, then Monkey's Audio may be the codec for you. Ripping and encoding to Monkey's Audio will take you a third longer than doing so with FLAC, but Monkey's Audio produced the smallest file sizes for each of the ten albums tested here. The space saving is only five percent of the space taken by FLAC files, an average of 21MB per album, but if you're on a tight disk budget, or if you want to serve the files from a web server, twenty megabytes per album might mean a lot to you.

OptimFROG and WavPack show promise. OptimFROG is clearly capable of compression of the same strength as Monkey's Audio, but it needs to become a lot faster before it will look attractive. WavPack offers the same sort of compression strength as FLAC, but isn't as fast. The speed of a software process usually improves as the software gets developed, as optimisations allow the same work to be done more efficiently. So these lossless codecs may well offer faster ripping and encoding times in the future.

Shorten performed, in my opinion, the least impressively of all the compressed lossless codecs, being the slowest and least effective at reducing file size. To all the people who trade in live music recordings stored in the Shorten format, I recommend they take a look at FLAC or Monkey's Audio, both of which will save space and bandwidth compared to SHN files. [Shorten's creators, SoftSound, now seem to have gone offline.]

Of course, if you need even bigger space savings, and couldn't care less how long it takes to encode your audio, then you should give the slower compression modes of some of the codecs a try. In this comparison I've gone with the compression mode I think will give the best mix of speed and size for each codec, but some people will care only about size.

Whichever codec you choose to encode your music with, make sure you test a few codecs for yourself first. That way you can decide which one produces the best compromise between speed and size.

For me, though, it's going to continue to be FLAC that I use to rip and encode albums to.

Evidence that conversions really are lossless

To make sure that the audio stored in each lossless format really is the same as the original audio source, I used an uncompressed Wave file of Canon in D major from The Essential Classics Collection. This Wave file was examined using the handy HashTab Shell Extension, and the MD5, SHA1 and CRC-32 hash values for the file were noted down, as was the size of the Wave file in bytes. This Wave file was converted using dBpowerAMP into a FLAC file (using the same compression settings as used in the big comparison above), and the original Wave file was deleted. The FLAC file was converted into a Monkey's Audio file, which was converted into an OptimFROG file, which was converted into a Shorten file, which was converted into a WavPack file. Finally, the WavPack file was converted into an uncompressed Wave file.

The resulting Wave file was exactly the same size as the original Wave file, and the MD5, SHA1 and CRC-32 hash values matched exactly with the original. This is very strong evidence that each of the lossless formats tested preserves the original audio perfectly.

Note that this process only succeeds because Wave does not support meta tags, it only stores the audio in the track. The conversion process above will change meta tags (that describe the track name, year the track was released, etc), so these are not necessarily preserved exactly between lossless formats. This is because not all audio file formats support the same meta tags. However, as far as audio is concerned, these formats really are lossless.