Hanging Notes

Topics: Audio, Host Development, Host Processing, Midi
May 8, 2012 at 1:25 PM

Hi obiwanjacobi

I've had some time to resume work on my workstation and am getting a persistent problem on one or two synth vsts regarding hanging notes.

If i should hold down one key after the other so I end up with 4 or 5 notes togeteher I get hanging notes.

If I should use vsthost it works just fine.

Now I used to procject in the samples which allows me to proxy the calls and I monitored the calls from  vsthost this seemed to be slightly different when sending midi via calling  hostCmdStubs[vstKey].PluginContext.PluginCommandStub.ProcessEvents(ve);

The first difference was that vsthost never sends MIDIOn with a volume of 0, it always turns them into NOTEOffs.

I tried implementing the same but to no avail.

The next I noticed the delta argument was being populated in VstMidiEvent, although it was only ever 0 for note offs (this could just be a coincidence)

So I attempted to emulate that, however I think I did this incorrectly and this caused the note to hang for more predicably so I believe the problem is related to the delta argument when I'm playing (and holding) the notes one after the other.

This is how I am implenting my delta changes

My host

private static object lockDelta = new Object();
        private int delta = 0;
        public int Delta
        {
            get
            {
                lock (lockDelta)
                {
                    return delta;
                }
            }
            set
            {
                lock (lockDelta)
                {
                     delta = value;
                }
            }
        }

 

 

 void IVstHostAudioManager.IncrementSamplePos(int blockPos)
        {
            Delta = blockPos;
        }
 
public void SendMidiData(byte[] midiData, Guid vstKey)
        {
            VstEvent[] ve = new VstEvent[] { new VstMidiEvent(Delta, 0, 0, midiData, 0, 0) };
            hostCmdStubs[vstKey].PluginContext.PluginCommandStub.ProcessEvents(ve);

        }
.......................................
In the VstPlayback
 if (host != null && host.VstOutputBuffers != null)
                {
                    // This function fills vstOutputBuffers with audio processed by a plugin
                    host.ProcessReplacing(blockSize);

                    // Init loop indexes
                    i = 0;
                    j = 0;

                    unsafe
                    {
                        // Fill audio buffer for all channels, multiply samples by volume range [0, 1]
                        audioBuffer = (float*)audioOutputBuffer.ToPointer();

                        while (i < blockSize * AudioParameters.NbChannels)
                        {
                            for (k = 0; k < AudioParameters.NbChannels; k++)
                            {
                                audioBuffer[i++] = host.VstOutputBuffers[k][j] * AudioParameters.globalVolume;
                            }
                            host.IncrementSamplePos(j);
                            j++;
                        }
                    }
                }

So in effect
host.IncrementSamplePos(j) send the position within the buffer block that the playback is at

Now the questions, if you would be so kind:-
  • Does my implemenation look OK ?
  • Do you know of such an issue ?
  • My Delta value is always 511 despite thread locking?

Any help we be great, since I'm bouncing of the walls here

Thanks in advance T

 

May 8, 2012 at 5:49 PM

The standard way I think is to first send a note on, then follow it with an equivalent note off of the same note value. Try hardcoding two events with a sleep in between until it works out and continue from there. I wouldn't worry about the delta argument, shouldn't make a difference in most case. If you don't need delta precision just leave it at 0.

May 8, 2012 at 10:59 PM

Hi Yurk

Sorry , just need to make myself a little clearer.  What I've done it terms of note on note off works on lots of synths, what i have a problem with i playing say a chord and releasing they keys in some sort of unordered way.  the VstHost seems to handle this on a particular vst synth where as mine does not.

It particularly problematic on this synth on some sounds

http://www.kvraudio.com/product/phadiz_by_algomusic

Below is a typical scenario that causes the hanging notes

 

---------------------------------------------
Event type :MidiEvent
Detune :0
DElta :0
Len :0
Offset :0
NoteOffVelo :0
byte0 :144
byte1 :72
byte2 :127
byte3 :0
---------------------------------------------
---------------------------------------------
Event type :MidiEvent
Detune :0
DElta :0
Len :0
Offset :0
NoteOffVelo :0
byte0 :144
byte1 :76
byte2 :127
byte3 :0
---------------------------------------------
---------------------------------------------
Event type :MidiEvent
Detune :0
DElta :0
Len :0
Offset :0
NoteOffVelo :0
byte0 :144
byte1 :79
byte2 :127
byte3 :0
---------------------------------------------
---------------------------------------------
Event type :MidiEvent
Detune :0
DElta :0
Len :0
Offset :0
NoteOffVelo :0
byte0 :144
byte1 :81
byte2 :127
byte3 :0
---------------------------------------------
---------------------------------------------
Event type :MidiEvent
Detune :0
DElta :0
Len :0
Offset :0
NoteOffVelo :0
byte0 :128
byte1 :72
byte2 :64
byte3 :0
---------------------------------------------
---------------------------------------------
Event type :MidiEvent
Detune :0
DElta :0
Len :0
Offset :0
NoteOffVelo :0
byte0 :128
byte1 :76
byte2 :64
byte3 :0
---------------------------------------------
---------------------------------------------
Event type :MidiEvent
Detune :0
DElta :0
Len :0
Offset :0
NoteOffVelo :0
byte0 :128
byte1 :81
byte2 :64
byte3 :0
---------------------------------------------
---------------------------------------------
Event type :MidiEvent
Detune :0
DElta :0
Len :0
Offset :0
NoteOffVelo :0
byte0 :128
byte1 :79
byte2 :64
byte3 :0

May 9, 2012 at 8:57 AM
Edited May 9, 2012 at 9:07 AM

If you are using a midi library and keyboard to test I would still try to replicate the issue programmatically just to be sure what is going on.

Something like :

SendMidiData(new byte[4] { 144, 72, 127, 0}, vstKey);
SendMidiData(new byte[4] { 144, 76, 127, 0}, vstKey);
SendMidiData(new byte[4] { 144, 79, 127, 0}, vstKey);
SendMidiData(new byte[4] { 144, 81, 127, 0}, vstKey);
Thread.Sleep(2000);
SendMidiData(new byte[4] { 128, 72, 0, 0}, vstKey);
SendMidiData(new byte[4] { 128, 76, 0, 0}, vstKey);
SendMidiData(new byte[4] { 128, 81, 0, 0}, vstKey);
SendMidiData(new byte[4] { 128, 79, 0, 0}, vstKey);

If this doesn't work and flipping the last 79 and 81 note off message fix it for polyphonic SynthEdit plugins like Phadiz, you'll at least be sure where the problem comes from. From there you can narrow it down to a specific issue with SynthEdit. Maybe the bug occurs only with polyphonic Vst made by SynthEdit and doesn't affect every other native vst plugins programmed in C and compiled with VisualC++.

VstHost by Hermann Seib is notorious for implementing every possible hacks needed by SynthEdit plugins who deviates from the vst standard. The hanging note issue for SynthEdit is certainly a known fact as can be seen from various hosts forums.

For example reaper host has many threads on that issue:

http://forum.cockos.com/project.php?issueid=1296

http://forum.cockos.com/archive/index.php/t-43819.html

 http://forum.cockos.com/archive/index.php/t-44231.html release note: v3.13pre2 - October 9 2009 - + VST MIDI: fixed hanging note bug from synthedit and possibly other plugins

To sum it up, I wouldn't be especially concerned if the bug affects only polyphonic plugins made with SynthEdit. Unfortunately without much research I don't know the specific hack needed to make this work. In the case of Reaper, from the user perspective they made an option called "buggy plugin compatibility" that fixes the hanging note bugs from synth edit plugins.

May 28, 2012 at 2:14 PM

Hi YuryK

I have found the answer to the hanging notes problem.  I had a feeling in my water that it was my code, so I've hunted down the error like a dog!

The point that aroused my suspicions was when i monitored the call made to ProcessEvents by the VstHost.  It constantly calls this method even if there is no midi to send !!.  This got me thinking, VstHost is written in C, which tends not to deal with events, rather it would poll, so I did a stack trace and this revealed that

ProcessEvents was called before  every  ProcessReplacing.

This is due to , (i believe) to the midi data being releveant to the following data populated in  ProcessReplacing.

I suspect my issue was caused by a note off being sent whilst  ProcessReplacing was doing some work. ProcessReplacing runs on its own high priority thread,

since i was sending midi data on another thread without any locking the note off was getting lost. So in short ProcessEvents should be called before ProcessReplacing.  The documentation seems to support this too.

// vst playback using port audio

 private PortAudio.PaStreamCallbackResult AudioCallback(IntPtr audioInputBuffer, IntPtr audioOutputBuffer, uint blockSize, ref PortAudio.PaStreamCallbackTimeInfo timeInfo, PortAudio.PaStreamCallbackFlags statusFlags, IntPtr userData)
            {

                // If host and VstBuffer are initialized
                if (host != null && host.VstOutputBuffers != null)
                {
                    //process any pending midi data to be actioned before getting the sample data
                    host.ProcessEvents();

                    // This function fills vstOutputBuffers with audio processed by a plugin
                    host.ProcessReplacing(blockSize);
}
........
}


// the code for the VstHost

        /// <summary>
        /// A Queue of midi data to be processed keyed by the vsts GUI
        /// </summary>
        private ConcurrentQueue<KeyValuePair<Guid, VstEvent>> midiMessageQueue =
            new ConcurrentQueue<KeyValuePair<Guid, VstEvent>>();

        /// <summary>
        /// to be called by the audio provider to process pending midi data for a vst(s)
        /// </summary>
        public void ProcessEvents()
        {
            // a midi data item keyed by vst guid
            KeyValuePair<Guid, VstEvent> midiDeQItem;

            // holds up a list of pending data for each vst keyed by guid
            Dictionary<Guid, List<VstEvent>> midiOutData = new Dictionary<Guid, List<VstEvent>>();

            //look at all pending midi
            while (midiMessageQueue.TryDequeue(out midiDeQItem))
            {
                //holds midi data for current vst
                List<VstEvent> midiForThisVst;
                if (!midiOutData.TryGetValue(midiDeQItem.Key, out midiForThisVst))
                {
                    midiForThisVst = new List<VstEvent>();
                    midiOutData.Add(midiDeQItem.Key,midiForThisVst);
                }

                // add midi data for this vst
                midiForThisVst.Add(midiDeQItem.Value);
            }

            //get all pending midi data for each vsts
            foreach (var midiWithKey in midiOutData)
            {
                //send the array of pending midi events to the associated vst
                hostCmdStubs[midiWithKey.Key].PluginContext.PluginCommandStub.ProcessEvents(midiWithKey.Value.ToArray());
            }
        }

        /// <summary>
        /// Queue up incoming midi data for actioning by vstis
        /// </summary>
        /// <param name="midiData">the midi data to be actioned</param>
        /// <param name="vstKey">the key of the vst to pay attention to the midi data</param>
        public void SendMidiData(byte[] midiData, Guid vstKey)
        {
            if (vstKey != Guid.Empty)
            {
                //create a midi data event associated with a vst
                var qItem = new KeyValuePair<Guid, VstEvent>(vstKey, new VstMidiEvent((int)Delta, 0, 0, midiData, 0, 0));

                //queue it up for processing
                midiMessageQueue.Enqueue(qItem);
            }
        }


 

Everything now works like a charm no hanging note, I hope others find this usefull

regards T

 

May 28, 2012 at 5:38 PM

> So in short ProcessEvents should be called before ProcessReplacing.  The documentation seems to support this too.

Yeah that's right, and ProcessEvents should be called even if there is no events to process if you want to support every plugin out there...

Also every native calls through the PluginContext should be locked with a critical section, even the most trivial ones like ProcessIdle. Good job on finding this bug, multi-threading glitches are a pain.