Hypothetical: Alternative to FLAC and IT-Compressed samples?

Kitsune_Phoenix · March 19, 2015, 00:03:53

I have been looking at WavPack and Mp3HD, and have been wondering if these would be ideal replacements for FLAC or IT Compression in the next format OpenMPT will take in the future in order to minimize the size of the module. Thoughts?

Here is a comparison of various lossless codecs for further reference.

Saga Musix · March 19, 2015, 00:52:25

QuoteWavPack

And what exactly is your rationale behind this? FLAC is the best-supported lossless format out there, and literally the only advantage WavPack has over FLAC is that it supports floating-point samples, which OpenMPT doesn't support at the moment anyway - they are all converted down to 16-bit. All the current lossless codecs have a compareable compression ratio, you can verify that by just looking at the comparison table you linked.
Once OpenMPT actually supports floating-point samples, we can talk about this again, but until then it remains a mostly unnecessary feature.

QuoteMp3HD

Supporting anything that has MP3 in the name always brings stupid patent trouble that I am not going to touch. How is mp3HD even supposed to help with your goal of minimizing module size if it both contains a backwards-compatible lossy MP3 stream and a lossless variant? Why on earth would you ever want this in a module where you would always want to use the lossless version?! Mp3HD is only useful if you need to support legacy clients that only can play regular MP3 files, which is not even a scenario that would apply in the case of a new MPTM format.
OpenMPT is not supposed to be an all-in-one monster that supports every format ever created (no matter if lossy or lossless) just because someone out there thinks that it's a cool format.

Besides, it's not like the new format is going to happen tomorrow or even next month. I want to get this as right as possible, and that requires some proper design - and no offense, but if I / we need any help on that, I'll ask the people of which I know that they have a competent and well-founded opinion on things like this.
And to shed some light what the actual future MPTM format could do about this: It will most likely support any arbitrary audio codecs in principle, but the number of actually supported codecs in the application itself will be fairly limited.

Kitsune_Phoenix · March 19, 2015, 08:22:09

I had intended for this thread to be a discussion about possible alternatives to FLAC, and only brought up those two codecs as examples.

Saga Musix · March 19, 2015, 14:26:15

Quite honestly, it's too early for that. And as said, I know who to ask when I need input on file formats.

Kitsune_Phoenix · March 19, 2015, 15:53:11

Regardless, I would like this topic to remain open in case anyone else wants to provide their own input. Even if this is all just hypothetical.

LPChip · March 19, 2015, 20:14:19

I understand you like a discussion, but understand that whatever the outcome of this topic is, it is far too early for any of this to be used as development guide.

Then again, I'm not really that interested in how the sample format is being developed, given that I don't use them. I use SF2 files through a VSTi if I need samples, and for the rest, all synths.

The exception here is when I make a chiptune, but then the song has to be back-wards compatible, so they're IT files anyway, and again this topic does not apply.

Kitsune_Phoenix · March 20, 2015, 20:37:21

SF2? I haven't heard of that.

FreezeFlame(Alchemy) · March 20, 2015, 21:22:04

SF2 is an soundfont format. Its an collection of samples, that are handled by MIDI hardware/software.
https://en.wikipedia.org/wiki/SoundFont

Rakib · March 24, 2015, 21:00:10

How about using vst samplers, if you need access to more sample format?

Saga Musix · March 24, 2015, 21:43:19

That would certainly reduce the size of the modules because the sample file would not be part of the actual module anymore.

Kitsune_Phoenix · March 24, 2015, 22:39:28

Quote from: Rakib on March 24, 2015, 21:00:10
How about using vst samplers, if you need access to more sample format?

I mean a hypothetical standardized codec for internal samples. In case you want to send modules to other people... and preferably without requiring platform-dependent plugins.

On that note however, once you can cram multiple modules into a compressed archive and have OpenMPT actually detect all of them at once when you open the archive (at the moment, it can only open one module at a time when multiple are contained in an archive), you could have a bunch of modules that share external instruments that operate on a relative path thingamajig.

LPChip · March 25, 2015, 10:18:22

You can already do that by using one module and subsongs.

In the past, there was the Itp (.it project) file which allowed to link to .iti files rather than embedding them which was the idea of doing the same, but different.

Anyway, as long as sample data is not compressed in the module itself, having the same sample in 10 songs, then compressing those 10 songs should make the file 9 times as small, because a good compressor will identify the same data multiple times and just store it once in the compression file.

Saga Musix · March 25, 2015, 13:40:39

Quote from: LPChip on March 25, 2015, 10:18:22
In the past, there was the Itp (.it project) file which allowed to link to .iti files rather than embedding them which was the idea of doing the same, but different.

Which is the same as external samples in the .mptm format, but less versatile. But that's exactly what she was talking about I guess, the whole "relative paths inside zip files" thing.

Quote from: LPChip on March 25, 2015, 10:18:22Anyway, as long as sample data is not compressed in the module itself, having the same sample in 10 songs, then compressing those 10 songs should make the file 9 times as small, because a good compressor will identify the same data multiple times and just store it once in the compression file.

You don't need a "good" compressor for that, you need an archive format that supports solid compression (i.e. not .zip), and a large enough dictionary size, which is the main bottleneck here (large dictionaries require a lot of RAM during both compression and extraction).

Kitsune_Phoenix · March 25, 2015, 17:27:52

Regardless, it is the year 2015. You'd think we should have perfected lossless audio compression by now, or at the very least a form of lossy compression that sounds good regardless of playback speed and that doesn't deteriorate with multiple encodings.

...once we have quantum computers, we wouldn't probably need lossy formats ever again, since 1's and 0's could be merged wherever necessary. But that would require a quantum processor, and quantum storage device and maybe even quantum RAM.

Saga Musix · March 25, 2015, 17:32:20

Quote from: Kitsune_Phoenix on March 25, 2015, 17:27:52
Regardless, it is the year 2015. You'd think we should have perfected lossless audio compression by now, or at the very least a form of lossy compression that sounds good regardless of playback speed and that doesn't deteriorate with multiple encodings.

We do have tons of good lossless formats, FLAC being one of them. What's your point again?