This project has moved. For the latest updates, please go here.

Midi In -> Audio Out

Topics: Audio, Getting Started, Midi
Feb 6, 2013 at 3:58 AM
Firstly, thanks for the great library!

I'm currently catching midi-in using MidiProcessor and converting the midi signals to audio streams/samples using soundfont but how exactly do I output the audio within the Midi.Process function?
Coordinator
Feb 6, 2013 at 7:55 AM
Thanx!

The AudioProcessor gets called by the host on its Process method. As a midi instrument you would typically ignore the audio inputs and write your samples to the audio output.

Typically the MidiProcessor.Process method is called just before the AudioProcessor.Process method. I would suggest you pre-process the midi data in the MidiProcessor.Process method in such a way that producing the samples in the AudioProcessor.Process method is as quick and easy as possible - both methods are time critical.

The MidiNoteSampler sample plugin is close to the way you would need your plugin to work.
http://vstnet.codeplex.com/SourceControl/changeset/view/71210#258726

Hope it helps.
Feb 6, 2013 at 10:03 AM
Thanks for the speedy reply. I just grasped the concept, it seems like Audio Process method is called at a frequency(even without any midi event). And if i dont want to output any sound i just don't return anything in the buffer. Now the problem seems to be converting my stream/sample to fit the buffer. Still doing more research on this because I dont get the concept of buffer, its size, the relation to sample rates, and the frequency the audio process method is called. Any idea on this?
Coordinator
Feb 6, 2013 at 1:27 PM
The host usually has (advanced) settings for setting buffer size (also known as block size). Audio is processed in small blocks and each plugin (in a chain) gets a go at manipulating the audio samples. Your instrument plugin will be at the start of that chain - you generate the audio. Possible effect plugins could follow each manipulating the samples in the in-buffers.

The (Audio) Process method receives an array of input buffers and an array of output buffers. A buffer is just a small piece of allocated memory used to transfer sample data between plugin and host. Each position in the array corresponds to an audio channel in the host. So a Stereo effects plugin would receive an array of [2] for both input and output (note that the host could pass you more than the input and output channels you need). You specify the number these audio channels in the constructor of the AudioProcessor base class.

All AudioBuffers (input and output) are all of the same length. You can access the Count property on any one of the AudioBuffer instances to retrieve the length. If all is well it is the same value as was set by the host for the BlockSize property (on the AudioProcessor).

So each time the AudioProcessor.Process method is called you need to fill up the output buffers with the audio samples for a count of 'BlockSize'. You need to keep track on where you stand on copying samples to the output buffers because the block size is very likely to be smaller than a single waveform (or whatever it is that you use) - so it might take multiple calls to the AudioProcessor.Process method to get the complete wave in the output buffers.

Hope it helps.
Feb 6, 2013 at 3:09 PM
Thanks for the explanation, I finally got it working, sort of.

What I do is when I receive a midi-in I convert it to samples and store them in a buffer.

And when Audio Process gets called I output the stored buffer in BlockSize specified and remove the block from the buffer. (4410)

Now the sound become like this http://www.vios.com.au/VH20130206_230224.wav

There seems to be a tiny gaps between each buffer block? I have checked Audio.Process and it took less than 1ms to execute

Any idea why?
Coordinator
Feb 6, 2013 at 3:48 PM
I can hear some sort of delay/repeat but I don't hear gaps...?? Could it be you re-output the same wave data multiple times?

Perhaps the BlockSize is not set correctly? Use the Count property on the AudioBuffer - that is the value that is passed to the Process method by the host.

Also make sure you don't do any memory allocation during either Process method. Pre-allocate the buffers you need to store the samples and use a ring-buffer mechanism to keep track on where you write new data and read out during the (Audio) Process method.

[2c]
Feb 6, 2013 at 4:03 PM
You are right, its some kind of repeat.

Some how if I remove BlockSize*8 from the Buffer everything works perfectly

The first 2 blocks are for stereo left and right I wonder what the remaining 6 blocks contains....weird
Coordinator
Feb 7, 2013 at 6:27 AM
Perhaps post a small code snippet so we can follow along? ;-)
Why are you doing BlockSize*8?
Feb 7, 2013 at 8:57 AM
Edited Feb 7, 2013 at 9:00 AM
I mixed up sample count with bytes hence I needed to mutiply it by bytes per sample(stereo 32 bit floating is 8 bytes per sample.)

I do have a few new problems.
  1. I copied the exact structure for PluginEditor in MidiNoteMapper, and created my own form. I added some random controls onto the form and the editor interacts perfectly in VSTHost. However when the same plugin is loaded in MixCraft 6, if I click on anywhere within the editor, the editor and DAW freezes and a force end process is required. (Note it is just a dummy editor, there are no code whatsoever except a blank ProcessIdle function along with Component Initialization). If I remove all controls and leave a blank form, it wont freeze if i click on the blank form.
  2. If my plugin is loaded (even without any editor UI). When I close the MixCraft 6 it says "MixCraft 6" has stopped working....is it just a bad DAW or do I need to call certain function when the VST is being unloaded?
Thanks for your valuable input...greatly appreciated!
Coordinator
Feb 7, 2013 at 9:28 AM
Attach the debugger and break into the process after you have clicked on the form. Then examine the call stack to see where execution halts. Copy in the stack trace in your next post.
Feb 7, 2013 at 9:48 AM
Edited Feb 7, 2013 at 9:50 AM
I paused the debugger after application freezes but it says no call stack information because no symbol loaded any call stack frame.

In modules I can see my .net dll it says the DLL is not loaded at default load address, manualing loading symbol did nothing...
Coordinator
Feb 7, 2013 at 10:05 AM
Try specifying the host exe as the debug exe (project properties) and press F5?
Make sure the host opens/loads your debug version and the .pdb files are deployed next to the .dll's...
Feb 7, 2013 at 11:11 AM
I tried your solution and I can confirm that the pdb is loaded through the Modules window.

Next to my .net.dll it says Symbol loaded and points to my debug folder.

When I Pause after freezing I get the following:

No Source Available

The current thread is not currently running code or the call stack could not be obtained.
Coordinator
Feb 7, 2013 at 11:21 AM
Any other threads (Threads Window) that do contain call stacks?
Feb 7, 2013 at 11:29 AM
Not Flagged > 5448 1 Main Thread Main Thread Lowest
Not Flagged 3964 0 Worker Thread <No Name> Highest
Not Flagged 10872 4 Worker Thread <No Name>
Not Flagged 6556 5 Worker Thread <No Name>
Not Flagged 2336 6 Worker Thread <No Name>
Not Flagged 5656 7 Worker Thread <No Name>


None of the threads contains any call stack. :(
Feb 7, 2013 at 1:11 PM
Problem Solved. Totally my bad. My form wasnt inheriting Forms.UserControl it was Inheriting Forms.Form.
Coordinator
Feb 7, 2013 at 1:49 PM
Ah, yes, it must be a UserControl. The host provides the 'frame' - something that the Form otherwise does....
The wrapper class just attaches the host provide frame window to the windows form user control.

Have fun.
Feb 7, 2013 at 3:09 PM
Edited Feb 7, 2013 at 3:14 PM
I seem to have endless questions.....Sorry! My head has been literally aching since the day I downloaded this library which is the same day I heard the word VST. On top of that OO programming with VB background is also painful!

How do I change Plugin base class' variable or call a plugin base method upon clicking a button in the UI editor.

My base class Inherits VstPluginWithInterfaceManagerBase

I suppose upon clicking a button i could set a public variable in the usercontrol to some value, and in the plugin editor class's ProcessIdle I will compare the value in the usercontrol to the value in _plugin......this seems to be a mary go around though...
Coordinator
Feb 7, 2013 at 3:37 PM
There are several way's to do that.

If you have values that you also (want to) expose as parameters, you can pass the ParameterManager instances to the UI. This is what the VST.NET Visual Studio Templates do. http://vstnet.codeplex.com/SourceControl/changeset/view/71210#1108764
BTW: do you know about programs and parameters?? See also the Delay sample plugin for an example.

I would advise you to reference the processor classes from the UI and NOT reference the UI class(es) from the processor classes. Note that the UI thread is a different Thread from the Audio thread - the thread that calls the Process methods. So some sort of synchronization is advisable. If you use simple values, the Interlocked class can be all you need.
Feb 7, 2013 at 3:51 PM
Edited Feb 7, 2013 at 6:08 PM
EDIT: nevermind I could just use a module what was i thinking...

To my knowledege Programs is the list of presets available to the Host, and parameter is similar concept.

But I dont want to expose any program or parameter to the Host, all changes are made through my own UI and stored as internal variable.

The problem is how do I reference back to the main plugin object created in stub from the UI form? There is no _plugin reference in the UI form like every other class.
Private _plugin As Plugin
Sub SetPlugin(ByRef plg As Plugin)  ' This will throw error saying I cant expose type outside the project
Coordinator
Feb 8, 2013 at 7:39 AM
I'm not a VB.NET programmer but you should be able to add a property to your User Control and set it...?
Make sure that the property has the same access modifier (public, private, friend - is that what's its called in VB?) as the plugin class. You cannot have a public property that exposes a private or friend type. My guess is, that is what the error is about...

[2c]