Author Topic: Manipulations and feedbacks with libopenmpt.js?  (Read 333 times)

Offline Edward256

  • Shy artist
  • Posts: 5
  • Gender: Male
    • Edward's Homepage
  • Operating System: Ubuntu LTS - Wine
Manipulations and feedbacks with libopenmpt.js?
« on: January 18, 2020, 00:34:51 »
Hello.
First time coming here, but been playing with Modules for about 30 years. I've been recently looking at Javascript for making some games (made two so far) and other visuals (hopefully synced to music). Just two months ago I started looking into maybe playing Modules on the web in combination with visuals, most probably games. I saw on modarchive.org that it has a web player, which used to be Java and now is Javascript and uses the code from chiptune2.js which in turn uses libopenmpt.js.

Now, if I read the chiptune2.js code correctly, it would seem the libopenmpt.js provides chunks of raw audio data which chiptune2.js converts into playable audio using the Web Audio API. I have also seen some instructions sent to libopenmpt.js which seem to be configurations as to how everything is to sound.

But I am wondering if there are more instructions one can send, and at any time? For example can I change the tempo midway (running out of time in a game), or the pitch (slowmo mode), or mute certain channels or instruments (game paused and only baseline plays)? I am also wondering if one can addEventListener on libopenmpt.js that will trigger for every new note/instruction played to keep certain things synced?

I know this is all probably pointless because I saw some thread where someone said trying to play a Module with fancy visuals would lag like heck, but thought I'd ask anyways. The alternative if I want to mute certain channels would be to make stems, but I learned the hard way that web audio can't always sync 4 or more "channels". Was hoping to keep it all in one place and small for loading time.

/Edward

Offline Saga Musix

  • OpenMPT Developers
  • *****
  • Posts: 6,862
  • aka Jojo
    • Download music, samples, VST plugins: Saga Musix Website
  • Operating System: Windows 10 x64
Re: Manipulations and feedbacks with libopenmpt.js?
« Reply #1 on: January 18, 2020, 10:23:10 »
Yes, you can change tempo, pitch and various other properties of module playback, please refer to the documentation for the libopenmpt C interface, as well as the documentation for the libopenmpt_ext C interface. libopenmpt_ext is an extension interface to libopenmpt which provides more experimental and thus subject-to-change APIs for direct module manipulation. chiptune2.js directly uses this C API, so you can just add more functionality to your software exactly the same way as chiptune2.js does.

Quote
I am also wondering if one can addEventListener on libopenmpt.js
There is no such thing because libopenmpt does not do any audio playback by itself, so any real-time notifications have to be generated by the user who knows when exactly the audio rendered by libopenmpt is going to be actually heard. You can retrieve pattern data from libopenmpt (see the documentation above) after rendering a chunk of audio to generate those notifications yourself.
» No support, bug reports, feature requests via private messages - they will not be answered. Use the forums and the issue tracker so that everyone can benefit from your post.

Offline Saga Musix

  • OpenMPT Developers
  • *****
  • Posts: 6,862
  • aka Jojo
    • Download music, samples, VST plugins: Saga Musix Website
  • Operating System: Windows 10 x64
Re: Manipulations and feedbacks with libopenmpt.js?
« Reply #2 on: January 18, 2020, 11:56:10 »
One more thing, there may be a small change required to use chiptune2.js with the current libopenmpt WASM builds. This is what I did in my custom fork to make it work: https://github.com/sagamusix/s3m.it/commit/b69bf732bc9e2a2a0d596f536d4282fe7c07f871#diff-3f278f073a08e8a2f0ade5bab09be2c9
» No support, bug reports, feature requests via private messages - they will not be answered. Use the forums and the issue tracker so that everyone can benefit from your post.

Offline Edward256

  • Shy artist
  • Posts: 5
  • Gender: Male
    • Edward's Homepage
  • Operating System: Ubuntu LTS - Wine
Re: Manipulations and feedbacks with libopenmpt.js?
« Reply #3 on: January 18, 2020, 19:16:47 »
Ah. Thank you for pointing me in the right direction. I had seen the documentation page, but I did not know exactly where to look for those exact commands. After some comparing (and putting scripts in the right order) I am able to play a song and see exact row and position.
« Last Edit: January 18, 2020, 19:27:16 by Edward256 »

Offline Edward256

  • Shy artist
  • Posts: 5
  • Gender: Male
    • Edward's Homepage
  • Operating System: Ubuntu LTS - Wine
Re: Manipulations and feedbacks with libopenmpt.js?
« Reply #4 on: January 18, 2020, 21:02:44 »
Ok, I seem to be having a problem with setting the tempo...
Trying the direct approach
Code: [Select]
libopenmpt._openmpt_module_ctl_set(player.currentPlayingNode.modulePtr,"play.tempo_factor","2");
returns
Code: [Select]
openmpt: error: openmpt_module_ctl_set: ERROR: argument null pointer

Trying through chiptune2.js
Code: [Select]
player.module_ctl_set('play.tempo_factor', '2.0');
returns
Code: [Select]
TypeError: libopenmpt.ccall is not a function

... Maybe I picked the wrong libopenmpt... Straight from the official site version 0.4.11 and extracted the one in the js folder.

Offline Saga Musix

  • OpenMPT Developers
  • *****
  • Posts: 6,862
  • aka Jojo
    • Download music, samples, VST plugins: Saga Musix Website
  • Operating System: Windows 10 x64
Re: Manipulations and feedbacks with libopenmpt.js?
« Reply #5 on: January 18, 2020, 23:35:49 »
Quote
Trying the direct approach
I suggest you read up a bit on how emscripten-translated code works in general, because you are directly interacting with the emscripten output here and there is absolutely no hand-holding that makes things easy and intuitive to use from the JS side. In this particular case, you are trying to pass a javascript string to emscripten, but that won't work because it has to be transformed into a string that emscripten can handle. For example, you could put it on the emscripten stack (as it's known to be a very small string, it's safe to put it on the stack):
Code: [Select]
libopenmpt._openmpt_module_ctl_set(player.currentPlayingNode.modulePtr, asciiToStack("play.tempo_factor"), asciiToStack("2"));
Quote
... Maybe I picked the wrong libopenmpt... Straight from the official site version 0.4.11 and extracted the one in the js folder.
As mentioned above, chiptune2.js is currently not compatible with our latest releases due to not waiting for the runtime to be initialized. Your error message sounds like the runtime is probably not initialized. See my suggested patch from the link above.
» No support, bug reports, feature requests via private messages - they will not be answered. Use the forums and the issue tracker so that everyone can benefit from your post.

Offline Edward256

  • Shy artist
  • Posts: 5
  • Gender: Male
    • Edward's Homepage
  • Operating System: Ubuntu LTS - Wine
Re: Manipulations and feedbacks with libopenmpt.js?
« Reply #6 on: January 19, 2020, 14:09:35 »
Ok... asciiToStack is not defined. However, looking at the metadata part of chiptune2.js I figured I could probably use the same technique to set a key value.
Code: [Select]
var keyNameBuffer1 = libopenmpt._malloc(18);
writeAsciiToMemory("play.tempo_factor", keyNameBuffer1);
var keyNameBuffer2 = libopenmpt._malloc(2);
writeAsciiToMemory("2", keyNameBuffer2);
libopenmpt._openmpt_module_ctl_set(player.currentPlayingNode.modulePtr,keyNameBuffer1,keyNameBuffer2);
libopenmpt._free(keyNameBuffer1);
libopenmpt._free(keyNameBuffer2);
I managed to change tempo and pitch using this.

However, there don't seem to be any "keyNames" for other manipulations. I guess that is where libopenmpt_ext comes in, which I see also contains tempo and pitch change. But my current problem is "activating" it. I'm guessing it has something to do with openmpt_module_ext_get_interface() but not exactly sure what params I should be giving it, or how it should be "spelled" in js.

Offline Saga Musix

  • OpenMPT Developers
  • *****
  • Posts: 6,862
  • aka Jojo
    • Download music, samples, VST plugins: Saga Musix Website
  • Operating System: Windows 10 x64
Re: Manipulations and feedbacks with libopenmpt.js?
« Reply #7 on: January 20, 2020, 19:51:52 »
Here's how using an interface function could look like in C:
Code: [Select]
openmpt_module_ext_interface_interactive interactive;
int success = openmpt_module_ext_get_interface(modulePtr, LIBOPENMPT_EXT_C_INTERFACE_INTERACTIVE, &interactive, sizeof(interactive));
if(!success)
{
  // error handling...
}
interactive.set_tempo_factor(modulePtr, 2.0);
I have never used libopenmpt_ext with emscripten so I cannot tell you the exact syntax to use on the JS side. You might want to look up how to use function pointers, as every interface is basically just a struct of function pointers. If you have any findings, please let us know here.
» No support, bug reports, feature requests via private messages - they will not be answered. Use the forums and the issue tracker so that everyone can benefit from your post.

Offline Edward256

  • Shy artist
  • Posts: 5
  • Gender: Male
    • Edward's Homepage
  • Operating System: Ubuntu LTS - Wine
Re: Manipulations and feedbacks with libopenmpt.js?
« Reply #8 on: February 15, 2020, 11:08:58 »
Well, I haven't managed to figure out the interactive part yet, but thought I'd show what I've managed to do so far:
https://edwardleuf.org/music/Mods/showmod.php?file=hdmod.it
lags on mobile
« Last Edit: February 15, 2020, 14:54:27 by Edward256 »

Offline Saga Musix

  • OpenMPT Developers
  • *****
  • Posts: 6,862
  • aka Jojo
    • Download music, samples, VST plugins: Saga Musix Website
  • Operating System: Windows 10 x64
Re: Manipulations and feedbacks with libopenmpt.js?
« Reply #9 on: February 16, 2020, 19:06:44 »
Looks nice :)
» No support, bug reports, feature requests via private messages - they will not be answered. Use the forums and the issue tracker so that everyone can benefit from your post.