Error while saving loses plugin data

[Techno.Jon]

Hi all,

So I just tried to save a module I'm currently working on and it threw me a whole bunch of exceptions from Kontakt Player 4 that caused it to lose all the data about the plugin.  I brought it back from the autosave, but I can't save it as anything because doing so corrupts the plugins.

I am running 1.22.07.00 on Windows 7 x64.  My machine is a custom build with an MSi motherboard and AMD CPU.  I am currently not using an auxiliary sound card.

More details:



Upon attempted save, MPT throws me one to three of these errors (up to one for each instance of Kontakt 4 (When working with a symphonic orchestra, the 16 available MIDI channels fill pretty quickly (if anyone could tell me how to tell MPT to send midi data to channels greater than 16, that'd be greatly appreciated))).  I have not received these errors in the past--it has saved fine with half a dozen Kontakts running simultaneously.  My active plugins include the following:



If anyone knows why this error is occurring, it would be nearly as helpful as knowing a solution.  I might try rebooting and hope it works.  Unfortunately, the autosave was not recent enough to catch all my changes, so I have to resurrect some stuff, but if anyone knows how to make it save again, that would be fantastic.

[LPChip]

If you open the auto-save song, and save it as a new file, does it immediately crash? or does it only crash after a certain info edited into the song?

If the auto-save song already crashes, can you upload it so the devs can look into it?

[Saga Musix]

Dispatch code 23 is effGetChunk, meaning that Kontakt maybe can't allocate enough memory to copy the plugin data, so it's only natural that the data is lost.
Are you maybe running out of memory? If that is the case, it might be a good idea to use jBridge to run Kontakt in a separate process, to circumvent the 32-bit process memory limit. That list of plugins (EWQL, several Kontakt instances...) suggests that this might be indeed the case, so you either have to try jBridge while OpenMPT doesn't have its own bridge yet, or you could also try the experimental OpenMPT 64-bit build if your plugins are available as as 64-bit plugins.
Besides, it might be a good idea to upgrade to Kontakt 5, I know that especially old versions of Kontakt have always caused trouble with OpenMPT, but with the latest versions NI has apparently worked on increasing host compatibility. I don't use Kontakt, but the few times I used it for testing stuff, Kontakt 5 always ran stable for me.

[Techno.Jon]

Are you maybe running out of memory?

Not that I can see.  I've got 8gb in this machine, and MPT and all things running inside it were taking a total of ~2gb, IIRC.  (That's the first thing I checked, but when I brought up Task Manager, it told me I was only using ~4gb total.)

it might be a good idea to upgrade to Kontakt 5

I tried that back when it came out, but for some reason it was bugging out with my current library, and since 4 had been working fine for forever, I just decided it wasn't worth it to pursue solving the problem and just kept working with 4.  I'm not using any library that requires a more recent version, so I probably won't bother until it's undeniably necessary.  That and Kontakt threw me no errors and continued to work fine in the open module after the failed save, so I don't think the problem lies inside it.

If you open the auto-save song, and save it as a new file, does it immediately crash? or does it only crash after a certain info edited into the song?

If the auto-save song already crashes, can you upload it so the devs can look into it?

It throws the errors if I try to resave the auto-saved piece.  I have no clue why the auto-save worked if this is the case.

I'll shoot you guys a rar with the current corrupt save, the working-fine autosave, and the nonworking resaved one.  It should be noted that Kontakt gives me no errors upon loading one of the borked files--it simply doesn't load any instruments.

EDIT: You may recognize (at least the title of) this tune--I have the old sample-only version on ModArchive.  I've finally gotten around to making it professional.  Yay.

[Saga Musix]

Not that I can see.  I've got 8gb in this machine, and MPT and all things running inside it were taking a total of ~2gb, IIRC.  (That's the first thing I checked, but when I brought up Task Manager, it told me I was only using ~4gb total.)

So you are running out of memory - remember that OpenMPT is still a 32-bit process, and I think the official builds are not compiled to be large address aware, so one instance of OpenMPT can only allocate as much as 2 GB of memory (instead of possible 3 GB or so).

As I said before, Kontakt throws an exception while OpenMPT asks it to provide the patch data, but since that fails, OpenMPT doesn't write any patch data into the module file, so it also doesn't restore any broken patch data that Kontakt could try to restore when loading that file. For the time being, using jBridge to run Kontakt (or any memory-hungry plugin) in a separate process might be required, or you can try that 64-bit version of OpenMPT if your plugins are available in 64-bit flavour. I hope to get OpenMPT's own plugin bridge working soon, so that this won't be necessary anymore.

Edit: I'll build an OpenMPT executable that is large address aware, so that OpenMPT can address more than 2GB of RAM, let's see if it helps.

[Saga Musix]

Here's a build of OpenMPT with large address awareness being enabled. It's experimental, but OpenMPT's own code shouldn't have any problems with large addresses. I have no idea how well this will play together with plugins, it's up to you to try that out now. If everything works out as expected, you should be able to use more than 2 GB of RAM in OpenMPT.
http://sagagames.de/stuff/mptrack.exe

[Techno.Jon]

So you are running out of memory - remember that OpenMPT is still a 32-bit process, and I think the official builds are not compiled to be large address aware, so one instance of OpenMPT can only allocate as much as 2 GB of memory (instead of possible 3 GB or so).

Ah, yes!  That makes sense.

So I downloaded your build, and it works perfectly!  Saves and plays fine, with no errors (at least in the first few minutes).  I'll respond with an update after working with it for a while.

Anyway, a final question as to the nature of the universe: why does the memory cap break MPT when saving but not when playing normally or when editing the patches?

[Techno.Jon]

Saga, you are the best.  That build's working fine--no saving problems, even after resurrecting the data lost to the borked save and playing back the whole piece.  I'd suggest building that into the next official release (or at least distribute it for testing), since it seems to be stable, at least over here on my end.  However, I will start using jBridge for future compositions, just to be safe, although I will most likely continue to use this build.

[Saga Musix]

Anyway, a final question as to the nature of the universe: why does the memory cap break MPT when saving but not when playing normally or when editing the patches?

Kontakt has to create a new memory chunk which holds the plugin settings as they are written to the module file; knowing Kontakt, this chunk is probably not very small, so Kontakt fails to allocate the required extra memory. Since you were bordering right on the memory limit, it could have been anything else that could have failed, it's probably just a coincidence that it happened with the plugin data - probably because Kontakt allocates a lot of temporary data for that, who knows.

I'd suggest building that into the next official release (or at least distribute it for testing), since it seems to be stable, at least over here on my end

Yeah, that's my plan (and the modification has already been committed to the code repository), but to be absolutely sure, I'd like to test it a bit more, as I don't know what would happen with plugins that are not large address aware in such a situation. I've tried to play around with that scenario (99% of my plugins are probably not large address aware), and so far it seemed to be stable.

However, I will start using jBridge for future compositions, just to be safe, although I will most likely continue to use this build.

Hopefully jBridge will also not be necessary anymore in the next version - as OpenMPT will come with a 64-bit build and its own plugin bridge. However, I have no idea when this will happen, probably early next year.

[Techno.Jon]

I see.  Take your time with any new version: you do awesome work and no one wants to rush it.  Thank you very much for your answers, that build, and of course all that you do!

[Saga Musix]

Also, one more thing...

When working with a symphonic orchestra, the 16 available MIDI channels fill pretty quickly (if anyone could tell me how to tell MPT to send midi data to channels greater than 16, that'd be greatly appreciated)

You can't, because that's simply how MIDI works - MIDI messages have only one nibble (4 bit value) of channel information, that's 16 channels in total. Yes, the Kontakt standalone version can handle more than 16 MIDI channels since it probably works differently internally, but the VST 2.x version can't. The VST 3 version probably can, but I will certainly not implement VST 3 support into OpenMPT.

[Techno.Jon]

Bumping for an interesting thing I've noticed.

This build has been the only build of MPT I've used to not crash after being left idling for a long period of time.  I know the bug was supposed to be fixed ages ago, but I simply stopped asking about it.  The idle-then-crash has been present in every version of MPT I've used except this build.  I don't know if you changed anything, but if you did, it works.

+9001 reliability, release this thing.

[Saga Musix]

The solution to that problem is to not use DirectSound output anyway. There have been no changes to the DSound code since 1.22.07.00 as far I'm aware.