You can't. It's simple as that. I lost count how many times I told you this before but we do not offer support for increasing any quantity limits in the OpenMPT code. You will have to figure it out on your own and I can tell you it's not a trivial task that could be accomplished with a single line of code. It requires changes over the entire code base, including file format changes. None of the file formats supported by OpenMPT can technically support more than 255 instruments (or effect values greater than 255, etc.).
