This project has moved and is read-only. For the latest updates, please go here.

VST Unit Testing

Feb 21, 2016 at 12:31 AM
I'd like to create some unit tests, but I'm a bit confused as to whether or not it is possible in a VST.

What I'd like to test most of all is saving and loading patches. The involves calling GetChunk, and SetChunk. But, I can't make these methods fire myself. These methods have to be fired from within Ableton. So, I guess what I'm trying to achieve is more like automated testing - as in, I would have to automate the key strokes etc. to make Ableton load an save the project file.

Any ideas about this? Do I need to use an automated testing package?
Feb 23, 2016 at 11:06 AM
Unit Testing is usually done with small units (classes) your plugin is constructed with. This should not need any VST specific calls.

If you want to write Integration Tests you could use the VST.NET Host capabilities to setup a test harness that 'plays host' for the plugin under test.

I do remember seeing a test host that did all sorts of nasty things to the plugin to test how robust the plugin was. Can't remember the name...
Feb 24, 2016 at 12:25 AM
I second that, what you are looking for are integration tests (open/close document). In practice you can wrap your integration test into visual studio unit test framework if you prefer that. You need an automatable host, sending keystrokes is not robust. Vst.Net has a host example you could use if hooking into a commercial host automation package is too difficult.
Feb 24, 2016 at 12:39 AM
obiwanjacobi wrote:
Unit Testing is usually done with small units (classes) your plugin is constructed with. This should not need any VST specific calls.
Normally, this wouldn't be difficult, but in the case of a hosted VST, all of the code relies on the VST host stub being instantiated and working. If I didn't host my VST in Ableton or some other host, this wouldn't be instantiated, and none of my code would work.
If you want to write Integration Tests you could use the VST.NET Host capabilities to setup a test harness that 'plays host' for the plugin under test.
While this sounds like a good idea, wouldn't this be very hard? Would I be essentially recreating Ableton?
I do remember seeing a test host that did all sorts of nasty things to the plugin to test how robust the plugin was. Can't remember the name...
If you have any details it would be good. I guess what would be needed would almost be an app in and of itself. I imagine an app like this would be extremely complex.
Feb 24, 2016 at 12:40 AM
I'm actually considering using some automated testing tool instead of unit testing...
Feb 24, 2016 at 12:42 AM
YuryK wrote:
I second that, what you are looking for are integration tests (open/close document). In practice you can wrap your integration test into visual studio unit test framework if you prefer that. You need an automatable host, sending keystrokes is not robust. Vst.Net has a host example you could use if hooking into a commercial host automation package is too difficult.
Wouldn't I have to host my VST host?
Feb 24, 2016 at 8:39 AM
Nah, the Test Host would be way simpler than a full fledged DAW. Look at the Host sample in VST.NET - that is the level you can start with. Totally doable.

Found the crunch-tool - note that it tests extreme conditions.
http://www.kvraudio.com/product/vst_plugin_unit_test_by_christian_budde
Feb 26, 2016 at 1:49 AM
Edited Feb 26, 2016 at 1:58 AM
If your host also acts as a plugin than you'd have to host your host. If your plugin is a host than I guess you know how to host your host. The host example in Vst.net is very basic but if all you need is getChunk/setChunk it's trivial to add that. I have a WPF host which is a Vst.Net proof of concept host that has simple open plugin and sound playback functionality. It is available here: https://onedrive.live.com/?id=8879321FE91D75FF%21127&cid=8879321FE91D75FF

Image

Another option is to look for a host that is automatable because Ableton Live doesn't seem to have a macro/automation framework. To be fair I don't know if any host has one. I wouldn't suggest an unreliable approach but you can have a go at it. If you properly handle focus and use keyboard shortcuts or have a fixed resolution environment you can try to automate your host with macro software such as AutoHotKey or even MacroMagic. The key with macro is to give plenty of time between inputs and make sure window position and focus is perfect.

Side story, keyboard and mouse macros are notoriously unreliable... but I have very fond memory of combining macro software, ip renewal on each dial-up modem connection and 1$ pay per click pr0n banner advertising. I was earning more when sleeping than my father at work at the time but had to cut it quick before being audited by the ads network.