Manipulations and feedbacks with libopenmpt.js?

[Edward256]

Hello.
First time coming here, but been playing with Modules for about 30 years. I've been recently looking at Javascript for making some games (made two so far) and other visuals (hopefully synced to music). Just two months ago I started looking into maybe playing Modules on the web in combination with visuals, most probably games. I saw on modarchive.org that it has a web player, which used to be Java and now is Javascript and uses the code from chiptune2.js which in turn uses libopenmpt.js.

Now, if I read the chiptune2.js code correctly, it would seem the libopenmpt.js provides chunks of raw audio data which chiptune2.js converts into playable audio using the Web Audio API. I have also seen some instructions sent to libopenmpt.js which seem to be configurations as to how everything is to sound.

But I am wondering if there are more instructions one can send, and at any time? For example can I change the tempo midway (running out of time in a game), or the pitch (slowmo mode), or mute certain channels or instruments (game paused and only baseline plays)? I am also wondering if one can addEventListener on libopenmpt.js that will trigger for every new note/instruction played to keep certain things synced?

I know this is all probably pointless because I saw some thread where someone said trying to play a Module with fancy visuals would lag like heck, but thought I'd ask anyways. The alternative if I want to mute certain channels would be to make stems, but I learned the hard way that web audio can't always sync 4 or more "channels". Was hoping to keep it all in one place and small for loading time.

/Edward

[Saga Musix]

Yes, you can change tempo, pitch and various other properties of module playback, please refer to the documentation for the libopenmpt C interface, as well as the documentation for the libopenmpt_ext C interface. libopenmpt_ext is an extension interface to libopenmpt which provides more experimental and thus subject-to-change APIs for direct module manipulation. chiptune2.js directly uses this C API, so you can just add more functionality to your software exactly the same way as chiptune2.js does.

I am also wondering if one can addEventListener on libopenmpt.js

There is no such thing because libopenmpt does not do any audio playback by itself, so any real-time notifications have to be generated by the user who knows when exactly the audio rendered by libopenmpt is going to be actually heard. You can retrieve pattern data from libopenmpt (see the documentation above) after rendering a chunk of audio to generate those notifications yourself.

[Saga Musix]

One more thing, there may be a small change required to use chiptune2.js with the current libopenmpt WASM builds. This is what I did in my custom fork to make it work: https://github.com/sagamusix/s3m.it/commit/b69bf732bc9e2a2a0d596f536d4282fe7c07f871#diff-3f278f073a08e8a2f0ade5bab09be2c9

[Edward256]

Ah. Thank you for pointing me in the right direction. I had seen the documentation page, but I did not know exactly where to look for those exact commands. After some comparing (and putting scripts in the right order) I am able to play a song and see exact row and position.

[Edward256]

Ok, I seem to be having a problem with setting the tempo...
Trying the direct approach

Code Select Expand


libopenmpt._openmpt_module_ctl_set(player.currentPlayingNode.modulePtr,"play.tempo_factor","2");

returns

Code Select Expand


openmpt: error: openmpt_module_ctl_set: ERROR: argument null pointer


Trying through chiptune2.js

Code Select Expand


player.module_ctl_set('play.tempo_factor', '2.0');

returns

Code Select Expand


TypeError: libopenmpt.ccall is not a function


... Maybe I picked the wrong libopenmpt... Straight from the official site version 0.4.11 and extracted the one in the js folder.

[Saga Musix]

Trying the direct approach

I suggest you read up a bit on how emscripten-translated code works in general, because you are directly interacting with the emscripten output here and there is absolutely no hand-holding that makes things easy and intuitive to use from the JS side. In this particular case, you are trying to pass a javascript string to emscripten, but that won't work because it has to be transformed into a string that emscripten can handle. For example, you could put it on the emscripten stack (as it's known to be a very small string, it's safe to put it on the stack):

Code Select Expand

libopenmpt._openmpt_module_ctl_set(player.currentPlayingNode.modulePtr, asciiToStack("play.tempo_factor"), asciiToStack("2"));

... Maybe I picked the wrong libopenmpt... Straight from the official site version 0.4.11 and extracted the one in the js folder.

As mentioned above, chiptune2.js is currently not compatible with our latest releases due to not waiting for the runtime to be initialized. Your error message sounds like the runtime is probably not initialized. See my suggested patch from the link above.

[Edward256]

Ok... asciiToStack is not defined. However, looking at the metadata part of chiptune2.js I figured I could probably use the same technique to set a key value.

Code Select Expand


var keyNameBuffer1 = libopenmpt._malloc(18);
writeAsciiToMemory("play.tempo_factor", keyNameBuffer1);
var keyNameBuffer2 = libopenmpt._malloc(2);
writeAsciiToMemory("2", keyNameBuffer2);
libopenmpt._openmpt_module_ctl_set(player.currentPlayingNode.modulePtr,keyNameBuffer1,keyNameBuffer2);
libopenmpt._free(keyNameBuffer1);
libopenmpt._free(keyNameBuffer2);

I managed to change tempo and pitch using this.

However, there don't seem to be any "keyNames" for other manipulations. I guess that is where libopenmpt_ext comes in, which I see also contains tempo and pitch change. But my current problem is "activating" it. I'm guessing it has something to do with openmpt_module_ext_get_interface() but not exactly sure what params I should be giving it, or how it should be "spelled" in js.

[Saga Musix]

Here's how using an interface function could look like in C:

Code Select Expand


openmpt_module_ext_interface_interactive interactive;
int success = openmpt_module_ext_get_interface(modulePtr, LIBOPENMPT_EXT_C_INTERFACE_INTERACTIVE, &interactive, sizeof(interactive));
if(!success)
{
  // error handling...
}
interactive.set_tempo_factor(modulePtr, 2.0);

I have never used libopenmpt_ext with emscripten so I cannot tell you the exact syntax to use on the JS side. You might want to look up how to use function pointers, as every interface is basically just a struct of function pointers. If you have any findings, please let us know here.

[Edward256]

Well, I haven't managed to figure out the interactive part yet, but thought I'd show what I've managed to do so far:
https://edwardleuf.org/music/Mods/showmod.php?file=hdmod.it
lags on mobile

[Saga Musix]

Looks nice