potential features request for libopenmpt: get tempo/speed/row dur/row samples

[EthanF4D]

Hello, continue from my recent post.
A little background:
I am developing a music game. Now I am experimenting with the capability with interactive interface.
It works for the basics. Play note and stop note. But for a music game, the skill and fun comes from timing of playing a note.
In my experiment I know that, if the play_note() happens between rows, the beginning of notes will not be rendered (especially noticeable for drums).
If I timed it so the play_note() happen before rendering the next row, the sound is perfect. AWESOME! 

Hence my game loop would be, some play_note(), some stop_note(), then read_interleaved_float_stereo() for a row worth of samples.
my test songs is IT format and XM format, so classic tempo mode
for my sample rate of 48000, the #rowSamples is currently calculated by (int)(48000 * 2.5 * Speed / Tempo).
It may be off by 1 or rounding error, and not handling alternative/modern mode.
So here I would request for a function openmpt_module_get_row_samples()

Then I have another test song with variable tempo/speed. So I have to query for speed and tempo for each row (to calculate #rowSamples)
After I set_position_order_row(), the get_current_speed() and get_current_tempo() does not reflect the change of tempo/speed on current row.
They only change when I read_interleaved_float_stereo() for even 1 sample. Looking at the source code, I realize it is because the change happens after ProcessRow()
And peek with read_interleaved_float_stereo() is a quite demanding process.
So I would expect the proposed function openmpt_module_get_row_samples() to account for the tempo/speed change,
and get_current_speed/tempo() to reflect it. or another set of functions for get_next_row_speed() get_next_row_tempo()

[manx]

The plan is to add functionality to libopenmpt which allows querying the remaining samples until the next tick (and thus also row).
That way, I think all your use cases could be handled by issuing play note at exactly the right position.

If you can query directly how many sample frames until a tick or row is done, there would also be no need to do the tempo calculations yourself.

The precise API in this case might even involve 3 new API calls. Only querying the number of sample frames per or until tick leaves it unclear whether the internal state gets updated when rendering the last sample frame of a tick. I think there are use cases for both, updating and not updating.

See https://bugs.openmpt.org/view.php?id=1017.

As features in this area are amongst the most requested features, we might add something like that to libopenmpt 0.4 already, instead of 0.5 like originally planned (however, we do not plan features that precise anyway).

After I set_position_order_row(), the get_current_speed() and get_current_tempo() does not reflect the change of tempo/speed on current row.

Which is precisely the reason why we probably need more API calls than a single new one. In that case it would be up to the API user to initialize the current (in this case the new) row without even rendering anything.

[Saga Musix]

In my experiment I know that, if the play_note() happens between rows, the beginning of notes will not be rendered (especially noticeable for drums).

That doesn't sound quite right to me... new notes should be able to be "queued" (via play_note) at any point in time, but they should always start playing on the next tick - and in particular, the start of the sample should definitely not be missing. Do you have a minimal example where this happens?

[EthanF4D]

In my experiment I know that, if the play_note() happens between rows, the beginning of notes will not be rendered (especially noticeable for drums).

That doesn't sound quite right to me... new notes should be able to be "queued" (via play_note) at any point in time, but they should always start playing on the next tick - and in particular, the start of the sample should definitely not be missing. Do you have a minimal example where this happens?

Thanks for reply Saga. Previously I didn't lock the module for multithread (audio thread keeps pulling samples, main thread play/stop notes), so it could be the main cause of missing the beginnings. Now that I lock it, the problem seems fixed.
Good to know that the notes should always start playing on the next tick, does this mean it will not be row-boundary precise?
With simple 1024 sampled, the drum sound ends prematurely even I have tried to extend the note duration.
With row sampled and play/stop note at this boundary, if I peek the buffer (1 sample read) , then query for the current tempo speed and fill the rest of the buffer after that, the sound has problem.

Code Select Expand


            int channels = 2;
            OpenMPTWrapper.openmpt_module_read_interleaved_float_stereo(Module.ModulePtr, 48000, 1, PeekBuffer);
            Tempo = OpenMPTWrapper.openmpt_module_get_current_tempo(Module.ModulePtr);
            Speed = OpenMPTWrapper.openmpt_module_get_current_speed(Module.ModulePtr);
            NumberOfSampleRequested = (int)(SampleRate * 2.5 * Speed / Tempo);
            Array.Copy(PeekBuffer, Buffer, 1 * channels);
            OpenMPTWrapper.openmpt_module_read_interleaved_float_stereo(Module.ModulePtr, 48000, (ulong)NumberOfSampleRequested - 1, &Buffer[1 * channels]);

With row sampled and if I don't peek at all (I can make use of the tempo/speed I get when I preprocess the song first), the drum is perfect.

for the original song, say on channel 1, the drum is hit every 4 rows. If I mimic this part by current API play_note and stop_note, they may actually be played on 4 different channels and not undergoing the regular 'note cut/note fade' as if replaced by another note on the same channel. Will this part be subject to change in future?

The plan is to add functionality to libopenmpt which allows querying the remaining samples until the next tick (and thus also row).
That way, I think all your use cases could be handled by issuing play note at exactly the right position.

If you can query directly how many sample frames until a tick or row is done, there would also be no need to do the tempo calculations yourself.

Thank you for reply manx. I am excited to hear such great news for v0.4 and you guys are very enthusiast of this library. So if the API will support all of this, I guess the sound problem I encountered may go away

[Saga Musix]

Thanks for reply Saga. Previously I didn't lock the module for multithread (audio thread keeps pulling samples, main thread play/stop notes), so it could be the main cause of missing the beginnings. Now that I lock it, the problem seems fixed.

Good, that was my only idea how it could happen.

Good to know that the notes should always start playing on the next tick, does this mean it will not be row-boundary precise?

No, because that would not make sense at all. Why should the API only allow to trigger notes on row boundaries while modules themselves can trigger them on any tick?
Ticks are the smallest time unit in modules and I don't think it makes sense to have a lower granularity in the API for that reason. In particular, OpenMPT does not even know in advance how long a row is going last, for two reasons:
- in modern tempo mode, the tick duration is adjusted on every tick to compensate for rounding errors to get perfectly steady BPM
- tempo change effects can change the tempo mid-row (because again, the smallest time unit in modules are ticks, not rows).
For this reason, a potential openmpt_module_get_row_samples() function cannot exist, and if something similar will be introduced, then it would rather be openmpt_module_get_tick_samples().

for the original song, say on channel 1, the drum is hit every 4 rows. If I mimic this part by current API play_note and stop_note, they may actually be played on 4 different channels and not undergoing the regular 'note cut/note fade' as if replaced by another note on the same channel. Will this part be subject to change in future?

I am not sure I understand the question here. What is supposed to change? If you use the play_note API, the note is not triggered on any specific pattern channel but on a background channel, hence it is not subject to any note-off events from other channels. You can only stop it by passing play_note()'s return value to stop_note().

[EthanF4D]

Thanks for the detailed explanation regarding ticks. Because I can only query the API for commands in row granularity, and the fact that I could "play a row" in OpenMPT tracker, this led me to wrongly assume that it is row-boundary precise.

It seems to me that I may not have convey what my game would look like, so you might not understand some of my odd questions. Here is a game called drum mania. https://youtu.be/SkpxU7z3tVI
Traditionally, the music is composed and exported in wav. Notes of particular interest (usually drums and main melodic instruments) are spliced out and exported in many wav samples. This mean the content creation is very restricted and cannot change easily. And use up huge amount of space. This is why I find out the module music is perfect for the task.

Say I analyzed a song, Assuming all patterns has 64 rows
and for example a particular note of interest is:
Channel 5 (1-based)
Order 3 (0-based), row 8 (0-based), C-5 11 v64 —
Order 3, row 12, ==
I currently book-keep it to be [5] C-5 11 v64, startRow=3*64+8=200, durationInRow=12-8=4
So if I would know startTick and durationInTick I am more than happy to use that as well.

If the gamer hit a note (in previous posts, by "player" I meant the gamer, the person that plays a game. Not module player software), I will use play_note() and stop_note().
Suppose the original note sequence is
Channel 5
Row 8, C-5 11 v64
Row 10, C-5 11 v64
Row 12, C-5 11 v64
Row 16, ==

This is audible as 3 notes and because being on the same channel, will there be some processing such as stop the sustain of the previous note and/or cross fade into the new note's beginning?
If this is the case it would sound different than they play/stop individually on different background channels, although arguably may not be such a big deal. A play_note_on_channel() could be beneficial and it will be the programmer's duty to call it on same background channel returned from the first play_note().

I understand libopenmpt is meant for playback and not for games. The interactive interface could be only to fulfill the minimum feature and will probably not contain features like pitch bend. Worry not, I have a work around that if gamer hit almost perfect timing, the original note is played instead. The off notes should sound awful anyway

[Saga Musix]

Thanks for the detailed explanation regarding ticks. Because I can only query the API for commands in row granularity, and the fact that I could "play a row" in OpenMPT tracker, this led me to wrongly assume that it is row-boundary precise.

Pattern data retrieval is bound to rows because that's what patterns are - just a set of rows. However, I think that some additional APIs may be useful here to get the current tick and the actual row length (because there are commands that can add additional ticks so that get_current_speed() is only the minimum amount of ticks on a row).

This is audible as 3 notes and because being on the same channel, will there be some processing such as stop the sustain of the previous note and/or cross fade into the new note's beginning?

Whether the notes in your example fade out, stop abruptly or keep playing forever completely depends on how the instruments in the module are set up.

If this is the case it would sound different than they play/stop individually on different background channels, although arguably may not be such a big deal. A play_note_on_channel() could be beneficial and it will be the programmer's duty to call it on same background channel returned from the first play_note().

stop_note() simply uses the default note-off action of that instrument, see the above comment what that can imply. It wouldn't make a differnce whether this note is then placed on a pattern channel or a background channel. Also note that pattern commands can be used to change the actual note-off action, so that an instrument that would normall fade out is cut off instead. This is probably not at all what you want.