.52 Cannot load a song of 1GB that has 59,920 patterns

Really Weird Person · March 08, 2009, 02:45:47

Full Version:
OpenMPT v1.17.02.52

Has the bug occured in previous versions? If yes, please specify version(s): (This option is optional, and doesn't need research. But in case you know...)

Description of the bug:
I am not sure if this a bug, but I will try to be clear-cut. I have a song that is 59,920 patterns long (straight through) (1, 2, 3, 4, 5, ..., 59,920) (1.04 GB in size). When I try to open it a second time, I cannot open it. I receive an "Out of memory" error. I tried opening it on a different [but beefier] computer and received the same error. My first thought was that, perhaps, it was too large. Here is the catch, however. I made the same song using 1,024 row patterns (3,745 of them to be exact) and was able to open another instance of it. [Note 1]

[Note 1]: I did not copy the patterns. I only inserted the number of patterns that were needed and the samples.

Is my report understandable? If this is indeed a bug, I would like it fixed ASAP, but, because life tends to make people busy, I am not sure when or if this will be resolved.

How often does it happen?:
Whenever I try to open a song that is 59,920 patterns long (straight through) (1, 2, 3, 4, 5, ..., 59,920) and 1.04 GB in size :lol: (This has only happened [essentially] once [literally twice, but I call it once because it was the same song both times].)

How to reproduce (step by step description):
1. Open MPT
2. Create a song that is 59,920 patterns long (going from either 1 - 59,920 or 0 - 59,919) (I suspect either way would produce the same result.)
3. Insert 37 samples.
4. Insert the patterns into their proper place (make sure that the LCM of yiour pattern numbers is 59,920.) (The easiest way to do this is to use two songs that are of lengths 428 patterns and 560 patterns.) (Order is irrelevant.)
If you follow those four [relatively] easy [but time consuming] steps, you should be well on your way to making a song that is at least 1.04 GB in size [Note 2].

[Note 2]: You may need more samples, more patterns, larger patterns, larger samples, or any combination of those to make the file 1.04 GB in size.

Edit by mod: Added bug subject to subject
Edit by mod: Set status to "closed" as newer versions of MPT don't support that many patterns anymore.

LPChip · March 08, 2009, 11:51:27

I'm not sure if people are able to create such file that easilly reproduces this.

In fact, I think its not a limitation of OpenMPT, but your computersystem. In such case, I wouldn't be able to reproduce this.

So the best way this can be tested is by sharing your 1GB song.

Can you zip the song in a .7z file and tell me how big it is? I don't mind giving you an FTP account so you can temporarilly store it on my server so others can see if they have the same problem loading this.

So in order to help you a bit further, I'm going to list a few possible causes and solutions:

Insufficient memory during save
When you saved the song, something went wrong, and the file got corrupted. The only solution here would be to recreate it.

Insufficient virtual memory
Windows has a virutal swapfile that it uses to store data that cannot be stored in memory. If the virtual memory is too low, an error like this can occur. If you go to the properties of "my computer" you should somewhere find an option to set this. (windows xp) Normally you want to set the minimum size to 1,5x your memory and the maximum size to 3x your memory. If you have 1 GB of system ram, you want 1500 - 3000.

Do note: if you have 1 GB of ram, but the system is set by its default 245-745 (or whatever the default is) you can easilly get problems like this. Secondly, even if you have the swapfile set right, make sure there's enough diskspace for the swapfile to expand. To solve this, change the swapfile settings and reboot.

Troubleshooting
Its a good thing to let us know how much ram OpenMPT is using when you work on this song. This can be seen in the taskmanager. Its also good to tell us how much RAM you have and what OS you're using.

Saga Musix · March 08, 2009, 13:22:47

QuoteI'm not sure if people are able to create such file that easilly reproduces this.

This is exactly what a test framework is for (maybe you remember our MSN conversation with rewbs). However, the MPT sourcecode doesn't really have a good test framework yet.

Really Weird Person · March 08, 2009, 13:54:02

Here are some answers:
1. Modplug Tracker's memory use according to Task Manager is ≈ 1.3 GB.
2. I am running the Windows 7 beta on the machine on which the song was created and my other machine that I tested it on runs Windows Vista Home Premium (both 64-bit). Perhaps I need a 128-bit OS? :lol: They both have 4 GB of RAM. The creator machine has a dual-core processor and the test subject has a quad-core processor. The creation machine's processors are manufactured by AMD and run at a total clock speed of 4.4 GHz. The other machine's processors are manufactured by Intel and run at a total clock speed of 10.68 GHz. As for the Virtual memory being too low, that is possible. I do not mess with that too often, so that may be my problem, thanks. As for the corruption, I suppose that is possible, but that happening twice in a row seems rather odd.

LPChip · March 08, 2009, 16:01:43

Would you be able to 7zip the file so it can be tested in a development environment?

And also, do you know how to work with FTP? Then I'll give you an ftp account for my server so you can temporarilly upload it.

Really Weird Person · March 08, 2009, 19:10:41

Here is something that I forgot to mention. I tried each of the files in Modplug Player (on the quad-core computer) and both of them worked just fine. They just would not open in Modplug Tracker for whatever reason.

LPChip · March 08, 2009, 21:51:03

Quote from: "Really Weird Person"Here is something that I forgot to mention. I tried each of the files in Modplug Player (on the quad-core computer) and both of them worked just fine. They just would not open in Modplug Tracker for whatever reason.

Great. Now, if you can get us your file so we can examine this... I sent you a PM with your FTP login details along with some instructions on how to use them.

Saga Musix · March 08, 2009, 22:04:55

QuoteHere is something that I forgot to mention. I tried each of the files in Modplug Player (on the quad-core computer) and both of them worked just fine. They just would not open in Modplug Tracker for whatever reason.

that's quite easy to explain. MPP uses an older version of the MPT 1.16 replay engine, which does not support that many patterns. thus, it won't load the complete tune.

Really Weird Person · March 08, 2009, 22:51:18

The song was successfully transferred according to FileZilla.

LPChip · March 09, 2009, 12:21:03

And it can be downloaded here: http://users.lpchip.com/rwp/Perhaps_the_Slime_Would_Miss_Rayman_Completely!.mptm

(I haven't tested it myself)

Rakib · March 09, 2009, 13:00:37

Code Select

http://users.lpchip.com/rwp/Perhaps_the_Slime_Would_Miss_Rayman_Completely!.mptm

hmm some problems with the forum.

Saga Musix · March 09, 2009, 16:10:27

It's not a problem with the forum, it's a problem with the user.

Special chars like ! have to be urlencoded.

Relabsoluness · March 10, 2009, 23:43:07

Are you sure that the memory simply doesn't run out(I haven't looked at the file)? By my understanding, when a module is loaded in MPT/OMPT, the file is first loaded to memory as whole, so if you're loading 1 GB file the peak memory consumption can be expected to be at least 2 GB and possibly much larger. And if so, perhaps it even requires 1 GB of contiguous memory(just an (un)educated guess).

And it's worth mentioning that because of various problems, the pattern limit will be lowered to 4000 in the new versions.

Saga Musix · March 10, 2009, 23:51:45

Quick look: I wasn't able to load the file either. ModPlug doesn't even seem to *try* opening it, because it stops directly after clicking on the file. My hex editor took considerably longer to realize that it can't handle it... (to big)

Really Weird Person · March 11, 2009, 23:56:21

Hmm, that will be a problem (the 4,000 limitation thing), as I have a song that is 6,420 patterns long (straight through). Remaking the song with extended patterns would be quite burdensome because I would be forced to paste a pattern and do several Page Down presses because not all of the patterns are filled. As for the lack of handling the song, I did not think of it taking up all of my memory (4 GB, minus any running programs).