[DUPE] Revised IT Project (.ITP) functionality

[Skaven]

The current implementation of Impulse Tracker Projects (.ITP) has some problems due to the way it's designed. Primarily, since it's based on Instruments (.iti), any song where multiple instruments share the same sample creates duplicate data when the song's instruments are saved for project references. Also, I find the process to convert an existing .IT file to a .ITP a bit complicated.

Therefore I suggest, that:

• The existing .ITP functionality is scrapped
  
• Make the Samples to have relative file paths, which are saved with the song
  
• Implement "Set Path" command to Samples instead of Instruments. This allows the user to replace a sample if needed, or to fix a broken link.
  
• Implement a "Reload Sample" command, to refresh the sample if it has been edited in an external editor while the .ITP is opened.
  
• Implement functionality for automatically turning an existing .IT to an .ITP (automatically saves all the .IT's samples to a specified folder and creates relative paths to them).
  
• Implement "Save As.." functionality to easily save an .IT copy from an .ITP for distribution or further processing, such as MO3 encoding. (The user can also convert an .ITP back to .IT by just changing the song type, just like before.)
  
• For easily porting instruments between songs and for creating an instrument library, a new "Instrument with relative paths" file format which saves the parameter data but NOT the samples, just paths.
  

In other words, the basic idea of the new .ITP is just that instead of saving all the samples with the song file, only paths to the samples are saved. All parameter / instrument data is still saved into the .ITP file.

When the user opens an .ITP file, the samples are loaded from disc thru the paths. If a link is broken (the sample has been moved, deleted or renamed), the program will prompt for a replacement file or let the user Ignore and leave the sample blank for now.

The sample loop points are initially read from the .WAV metadata, but if the user has modified the loop points in the .ITP (to use BiDi loop for example), those values will be used instead. If the user decides to re-open a .WAV, OpenMPT would prompt whether to use the .WAV's loop points or the ones specified in the .ITP.

If a sample has been edited, OpenMPT would detect this when opening the .ITP, and prompt which metadata to use.

If a sample is destructively edited within OpenMPT (amplified, cropped, etc), the user will be prompted to either save the changes over the original .WAV file (which will affect all .ITPs that use the sample), or to create a new version of the sample, which the sample slot will then automatically point to.

If the user imports an Instrument (with relative paths) that uses a sample that already is used in the song (has the same relative path), the instrument will use the existing sample instead of loading a duplicate into another sample slot.

In a nutshell:
.IT - contains all the song data, including the samples
.ITP - contains all the song data, but the samples are referred to via relative paths

This would give a simple, robust way to save dozens of work versions of your song without filling your hard drive with redundant sample data. It will also make it easy to edit the samples in an external editor, as they don't need to be saved from IT, edited, then imported back to IT.

And there shall be much rejoice.

[bvanoudtshoorn]

Wow - that sounds as though you're looking for a very sampler-like facility within OpenMPT. You're actually wanting to take the whole concept of instrument abstraction to the next level - not only do you want the instruments divorced from the file, but you want the samples divorced from the instruments, right?

I did a quick search, and I found this product - a VSTi sampler which can (apparently) use ITIs and XIs. I personally use Kontakt, the only drawback of which is its inability to use ITIs and XIs.


Edit: I'm downloading Antiwave at the moment, and I'll report back in a little while.

[Skaven]

You're actually wanting to take the whole concept of instrument abstraction to the next level - not only do you want the instruments divorced from the file, but you want the samples divorced from the instruments, right?

I don't want to divorce the instruments from the song, I actually want to keep those in. I just want to divorce the sample data, because the samples represent the biggest bulk of data in an .IT file.

The biggest benefit of this would be that multiple work versions of the song would not use much disk space.

Any VST sampler with this functionality would not be very useful for what I do, because they are not .MO3 compatible (ie you cannot .MO3 encode a song that uses VST instruments). But thanks for the tip anyway.

[bvanoudtshoorn]

Well, the VSTi that I found brought down my entire system anyway, so I'd steer clear of it regardless. :/

I do wonder, though, how compatible with MO3 ITPs are, given that they're an OpenMPT-only type.

[Skaven]

ITPs would naturally not be .MO3 compatible. To create a .MO3 you would have to Save As.. an .IT copy of the project.

It would still be manageable. You would have a bunch of .ITP files as work versions, then one "current" .IT file which you can overwrite whenever you need to .MO3 encode it. (and which you can then delete if you REALLY want to save disk space)

[Skaven]

Doesn't anyone else find this useful? It would also act as a stepping stone to the other feature that would be cool: allowing several songs to share the same sample set (with relative paths). That way I could compose tons of songs for a game without having to put them one after another into one sequence list and messing around with Bxx commands.

Oh, but after that, the next step would have to be making BASS.DLL or FMOD compatible with this arrangement. And the MO3 encoder would need to work with this too (that, or make OMPT able to load Ogg encoded samples).

Okayyyyy, I guess it's a "never mind" then. :|

[LPChip]

I believe Jojo was going to try something in the direction of MO3 support, but I could be wrong.

[Skaven]

MO3 support implemented into MPT? I don't think that's necessary? But please explain, could be I understood wrong. How would MO3 support work with what I was explaining above?

[LPChip]

Uh... I kinda read MO3 and my mind did the rest

[Relabsoluness]

Doesn't anyone else find this useful?

I think the ideas of improving ITP functionality are great, and I've had it among the top entries in my todo-list for quite some time, but so far have never got around to do anything about it.

allowing several songs to share the same sample set (with relative paths). That way I could compose tons of songs for a game without having to put them one after another into one sequence list and messing around with Bxx commands.

What would you think about multiple sequences per module?

[Skaven]

Thank you for your replies. Let's discuss.

I think the ideas of improving ITP functionality are great, and I've had it among the top entries in my todo-list for quite some time

Coolness. I wouldn't call it improving the existing one, but rather revising and simplifying it. The key ingredients are:

- Relative links/paths to samples only
- A new instrument file format which saves the parameters and envelopes, but only contains a link/path to the sample, no sample data (for porting instruments between songs)

What would you think about multiple sequences per module?

That sounds interesting, but it also sounds like it would make things a bit complicated? I wonder how BASS.DLL or FMOD would handle these, they would also need to be updated to support this. I presume these would be managed in the Tree View? You could have a renameable 'folder' for each sequence there.

A multi-layered "two-dimensional" sequence editor could also be interesting. You could put the drum beats as short patterns in one column, the chords in another and the leads in yet another, would allow some snappy arrangements. But again, this would require an update from BASS.DLL / FMOD's part too. And it would require revising the GUI by giving the current sequence editor multiple horizontal rows and making them pattern length aware.... OK, scratch that, it would get pretty complicated and start looking like Buzz.

One nifty way to solve this would be to create instanced "clips" within a pattern. If you copy-paste an area in a pattern, it could be "instanced", and visually surrounded with a faint rectangle. If you edit the contents of any clip rectangle, all its instances would get updated. Any single instance could also be un-instanced and turned back into unique note data if needed.

An instance auto-detection tool could be handy. It would scan the patterns in the song and look for identical sequences of notes (copy-pasted drum beats, etc) and convert them to instances.

As a cool extra, any instance could also be given a unique Transpose value, so that you could transpose each instance relative to the original. (If you want to transpose ALL the instances by the same amount, you do it by editing the contents of the instance instead of giving it a transpose value). Hmm, while we're at it, why not unique Volume and Pan values as well.

Anyway, this is just a quick idea so I'm not sure what kind of complications it would introduce to song editing and how hard it would be to implement, but at least I suppose it would not require updated FMOD/BASS support if all the instancing could be converted back to note data at any point (that, or include redundant "backwards compatibility" note data in the .IT file that BASS / FMOD can read).

None of these would solve the "multiple bulky work versions filling my hard drive" problem, however, that requires relative paths to samples.

[LPChip]

I like the instance way of working, although the extra's you're also proposing might be a bit too much to start with.

I wouldn't want to have a box to represent the notes, but rather see the notes and make them visible on another way. Perhaps even a different location. A tracker is so powerfull because it gives overview. Working with instances and replacing the notes to a box would interfere with the overview. But making the notes a different color or perhaps the background too, like grayed out as when you see past the end of a pattern, that would work quite well.

[Skaven]

Oh, of course the notes inside any Instance would be visible, and also editable without switching views or anything. Anyhow, instances and their relations to each other need to be clearly visualized, otherwise the user will find them confusing.

But if you indicate instances by just highlighting/coloring the background of the instances and there are multiple instances in a row, it may be difficult to tell where one instance ends and the next one begins. A faint single-pixel dash rectangle around each instance would make them visually more separate, like blocks if you will. But again, I don't know if the pattern display actually supports this. If you use a colored BG, maybe a slightly different shade for consequent instances would make them appear separate. Or how about this: when you move the cursor / float the mouse over an instanced block, it would be highlighted, and all its related instances would also highlight, but a bit more faintly.

Another possible problem: if there are two short instances that are almost identical but still different (like two different short hi-hat sequences laid out densely), it may be difficult to tell them apart. Maybe some kind of color-coding could be applied there?

Also, if you select an area within an instance, would it be possible to display the area selection in all the instances within the pattern? This would also clearly indicate that if you edit these notes here, all those notes there will also change.

P.S. Maybe these note sequence instancing messages should be moved to a separate thread?

[LPChip]

Well, it all falls under .itp functionality, or perhaps even .mptm functionality. (both perhaps? )

[Relabsoluness]

What would you think about multiple sequences per module?

That sounds interesting, but it also sounds like it would make things a bit complicated?

Perhaps, but there would be just one sequence to work with at any given time. So instead of having the songs one after another in sequence, there would be own sequence for all songs; when one would like to work with some particular sequence, it could be chosen simply by, say, right-clicking on sequence and choosing the given sequence from the context menu. The same patterns, samples, instruments etc. would be available for all sequences.

I wonder how BASS.DLL or FMOD would handle these, they would also need to be updated to support this.

Having the feature implemented is already in distant future, having anykind of support for it elsewhere tests the limits of imagination.

I presume these would be managed in the Tree View? You could have a renameable 'folder' for each sequence there.

Tree view could indeed have part in this.

One nifty way to solve this would be to create instanced "clips" within a pattern. If you copy-paste an area in a pattern, it could be "instanced", and visually surrounded with a faint rectangle. If you edit the contents of any clip rectangle, all its instances would get updated. Any single instance could also be un-instanced and turned back into unique note data if needed.

Nice idea -- could prevent 'code copy'. Related to this, I've been pondering about possibility of having sort of 'background pattern data'. The idea is roughly that if one has for example drum channels in pattern 1 and the same drum part would be used in multiple patterns, then instead of copying it to every pattern, one could have the channels in other patterns as 'background channels': the channels would effectively have the pattern data just as in pattern 1, but background data could be edited only in pattern 1, and the modifications done there would be effective on all background instances. Compared to the somewhat arbitrarily chosable clips, this might be considerably easier to implement.