Huge MP3 samples (was: Apple Music)

[Wodd]

I have a question. Does OpenMPT not support audio files designed for iOS devices? Specifically, I purchased an audio file from Relaxing White Noise. But, ModPlug Tracker will not open it. I tried using conversion software; but, converting it to MP3 using that (as I figured that might drop any trigger tags or the like that might be causing it to not open in OpenMPT) did not work. I am in the process of having it converted to OGG as of this post. But, it is a bummer that it doesn't work in its original format. I had previously downloaded it from YouTube. But, that file, too, would not load into the sample editor. I did a test to see if the issue is the [slightly longer than] 10 hour length. Such is not the case.

[StarWolf3000]

Uhm, where does the site say it offers files in Apple's format? It only speaks of MP3. And OpenMPT does support MP3.

[Wodd]

Putting the files on Apple's devices is mentioned after purchasing one (or more) of them. You can download a copy of the file from here Download the file with a name beginning with Campfire.

[manx]

OpenMPT (or any of the module formats supported by OpenMPT) does not support samples of any type or format that are 10 hours long.

This has nothing to do with "Apple Music". "Apple Music" is a music streaming service by Apple.

Audio containers/formats common on Apple systems are m4a/AAC (supported by OpenMPT via MediaFoundation), caf/ALAC (not supported by OpenMPT).

As far as I can see, OpenMPT 64bit will (somewhat successfully, the result is likely unusable) load the file (allocating > 14GB of RAM in the process).
OpenMPT 32bit will just run out of memory loading the file.

There are 2 bugs here:
1. OpenMPT does not properly reject MP3 samples longer than MAX_SAMPLE_LENGTH.
2. OpenMPT 64bit does not stop allocating memory after MAX_SAMPLE_LENGTH of data has already been decoded (this applies equally to MP3, Vorbis, Opus samples).

Another related bug here is https://bugs.openmpt.org/view.php?id=1112, which is far from trivial to fix.

In any case, OpenMPT will probably never support samples as long or big as this example.

[manx]

1. OpenMPT does not properly reject MP3 samples longer than MAX_SAMPLE_LENGTH.

Fixed in r10760. Will be backported to 1.27 soon.

2. OpenMPT 64bit does not stop allocating memory after MAX_SAMPLE_LENGTH of data has already been decoded (this applies equally to MP3, Vorbis, Opus samples).

Fixed in r10762. Will be backported to 1.27 soon.

[Wodd]

In any case, OpenMPT will probably never support samples as long or big as this example.

Length is not the issue. To verify, change the MAX_SAMPLE_LENGTH to at least 7fffffff (2,147,483,647) and load Counting and Primes into the newly built mptrack.exe. (Use 64-bit) After that file loads, try again to load the campfire file. You should notice that Counting and Primes loads with no problems; but, the campfire file still fails to load, yielding the Unknown File Format dialog. I tested Windows Media Player, Groove Music, and a third player and all of them load the campfire file with no problems. But, OpenMPT will not properly read it. I looked at the sizes of Counting and Primes and the campfire file and Counting and Primes is larger in size, but loads into OpenMPT with no problems. Size is not an issue here.

[manx]

In any case, OpenMPT will probably never support samples as long or big as this example.

Length is not the issue. To verify, change the MAX_SAMPLE_LENGTH to at least 7fffffff (2,147,483,647) and load Counting and Primes into the newly built mptrack.exe. (Use 64-bit) After that file loads, try again to load the campfire file. You should notice that Counting and Primes loads with no problems; but, the campfire file still fails to load, yielding the Unknown File Format dialog. I tested Windows Media Player, Groove Music, and a third player and all of them load the campfire file with no problems. But, OpenMPT will not properly read it. I looked at the sizes of Counting and Primes and the campfire file and Counting and Primes is larger in size, but loads into OpenMPT with no problems. Size is not an issue here.

Length is the issue. OpenMPT cannot represent samples as long as either of the examples you have given. The only sane option is to reject those files, as the user would otherwise expect the complete file getting loaded when in fact in would absolutely NEED to be truncated to some arbitrary length.
The arbitrary length chosen by OpenMPT is MAX_SAMPLE_LENGTH. Increasing the length will not change anything about that fact (both examples still will not fit).

7fffffff makes no sense as a limit because samples that long can
1. not be loaded into a 32bit OpenMPT on 32bit host systems *at all* (not enough user virtual address space available on x86 Windows)
2. not be stored in any module format supported by OpenMPT (all of them are basically limited to 32bit for the whole module file due to width of on disk structure fields.

OpenMPT therefore limits MAX_SAMPLE_LENGTH to 0x10000000, which, for stereo/16bit, amounts to 1GB of data (which can still be handled by file formats and the limited amount of virtual address space available on 32bit x86 Windows).

[Wodd]

Since I am not a moderator for the source code, I don't any longer use the 32-bit build. I have been using the 64-bit build exclusively for a while now. Sadly, because my computer was recently reloaded, I do not have access to the code, as to attempt debugging why the file will not load. I did math for finding the approximate sample length of the file. Given that I did increase MAX_SAMPLE_LENGTH to 2,147,483,647 (as going beyond that did not work, even when the controlling integer was 64-bit), it should load at only approximately 1,587,601,191 samples. But, it does not. Even if accounting for doubling of the size for memory usage, that is still only 1.073 GiB (gibibytes) or 1.152 GB (gigabytes). I don't mean to anger you, manx. But, I am confused as to why the campfire file will not load when the Counting and Primes file, which is larger in size and more than twice the length, opens with no problems.

[manx]

Neither file did load anything near properly, both before and after my changes for me.

The required result is that they do not load, because samples that long are unsupported.

The precise amount of length a sample will be able to use is unknown upfront, which is why the arbitrary limit is lower than the theoretical maximum.
This depends on address space layout randomization, amount of memory occupied by things already loaded in OpenMPT, installed shell plugins (which run in every process using file selection dialogs), address space fragmentation, the way the standard library implements growing of the temporary std::vector memory buffer, and various other things. Also, OpenMPT maps all input files into its address space, which further reduces the amount of available address space by, in this case, 600MB.

Also, even in 64bit OpenMPT neither file can load properly because sample length is stored as a 32bit value.

tl;dr: The limit is there for a reason, and it will not go away (either anytime soon, or possibly at all for 32bit OpenMPT). Changing the limit to 0x7fffffff will not fix anything or even make it possible to load larger samples at all on 32bit. It also makes no sense even for 64bit because, for various file formats, saving could not represent that sample length either.

OpenMPT is not designed to be used with multiple-hours long audio tracks, and it does not want to be that. There is better software out there than a tracker for such use cases.

[Wodd]

Plugins are another thing that I hadn't considered, as I do not use them. The vendor for that file has a shorter version of the same audio. That worked with no problems.