VST "dualcore pref" and host load distribution.

[gurulogic]

I am a bit confused with how Usine VST CPU load distribution works. I am working with multiple instances (meaning I run more than one true instance of Usine by having copies in my vst folder) and multiple inserts of Usine (the up to 16 VST shells that make up an instance of usine) in my host and am finding all CPU load going to core 1 on my quad core machine on a large project, however if I enable dual core optomization a bit of headroom appears to be freed up but not enough to make any real difference in the project.

If I start a fresh project in Ableton Live and put one true instance on each of four return channels and load each with a heavy cpu load, the load is perfectly distributed across all four cores (if I leave dual core optimization disabled, my machine promptly comes to a grinding halt) . I can then add another four channels and a shell for each instance and cpu load continues to be distributed evenly, killing my previous theory that once more shells are added, Usine vst defaults to core 1.

Then the question is is why does dual core optimiation have any effect when Usine is a vst plugin, why in the one project with two Usine instances and 12 shells for each instance does my cpu load all go to core 1 and which vst Usine shell is master (as in which shell instance on which insert does the host determine is the track that gets processor time? I was assuming this would be the first shell instance inserted but I think this is wrong.

Can anyone help clear this up for me?

[senso]

the dual core optimization option as no effect in th VST version because even if you have several instances of usine VST, in fact in memory there is only one instance.

This allows side chaining in Usine.

[gurulogic]

Thanks for the reply Senso.
I am almost certain enabling dual core in Usine vst has a significant effect when hosting Usine in Live 7.014 as well as the Live 8 beta . I will do more investigation tomorrow as my previous tests where done hastily.
I understand that there is only one instance of Usine and the other instances are of a sidechain type but in the circumstance of a host such as Ableton Live which distributes cpu load by track, how does the host determine which Usine vst insert is the master ? I would assume it should be the first insert made but I am not sure if it even works like that...

[gurulogic]

the dual core optimization option as no effect in th VST version

Ok, this must be a bug of some sort because if I disable dual core optimization in Usine VST, my computer comes to a grinding halt when I load CPU heavy Usine+VST Plugin patches beyond a certain point.

According to task manager and Live's CPU monitor, I am not seeing CPU load become more evenly distributed when I enable dual core optimization but I can continue loading patches and working far beyond where I would be able to with dual core optimization disabled. I would imagine this should be fairly easily reproducable on any quad core intel machine running Windows XP + Ableton Live 7.

As to the Live project in which all CPU load is going to core 1 I suppose I shall have to dig deeper as I just I just made a test project in Live in which I have Mutiple CPU heavy VSTI'sdirectly in Live tracks and Usine inserts also on every track with a very heavy CPU load patch in Usine. My CPU load is nicely balanced after doing this so there must be an issue with how Live is allocating CPU in that particular project.

[senso]

The dual core optimization is still abstract for me (and for most programmers).
In usine you have one thread (or process) for audio and another for the graphic engine.
But if you have 10 usine vst in your host you will have only 2 threads activated because there is only one instance on Usine really running..
Now the repartition of threads on each core is made by Windows not by usine and can change in real time, depending on the CPU load. (calculated by Windows with a complex algorithm)
That's why you can have strange results.

[bsork]

If I understand Gurulogic correctly, he actually runs more than one instance of the VST by using several (probably renamed?) versions of the DLL.

...and that's all I have to add to this thread... I have practically no knowledge of how threading is achieved...

[gurulogic]

If I understand Gurulogic correctly, he actually runs more than one instance of the VST by using several (probably renamed?) versions of the DLL.

That is correct,I am running several instances of Usine VST which I have accomplished by creating duplicates of the dll's and Files folder as well as renaming the duplicate dll's. For each of these instances I am also running multiple "sidechain" or "shell" instances so this is where my explanations might get confusing.

I have already determined that it is most likely the fault of my host that I was not seeing CPU load spread properly across the multiple dll copies of Usine that I had loaded, however the odd behavior I am seeing regarding the "dual core optimization" applies to only one instance of Usine with no further "sidechain" instances loaded but if I at all understand Senso's last post, it would seem that Usine VST is able to have two threads activated, even within a host, which would explain why enabling "dual core optimization" on a single instance of Usine VST has such a profound effect on performance...?

However, this conclusion does not coincide with the statement :

the dual core optimization option as no effect in th VST version because even if you have several instances of usine VST, in fact in memory there is only one instance.

At this point I am mostly wondering if there might be a bug?

[senso]

since the core repartition is done by windows, I can't answer precisely to your question!!

[gurulogic]

Ok, I've filled in some of the blanks for myself...!

First off, the way Ableton Live distributes cpu load between cores is track 1 gets core 1, track 2 gets core 2, track 3 gets core 3 etc...

What happens when I disable dual core optimization in Usine vst is that Usine forces Live to put Usine on core 1 regardless of which track Usine is inserted on. When I enable dual core optimization, Usine then plays within Live's guidlines for cpu distribution and will utilize the correct core accordingly to the track it is inserted on.
There is a serious problem for me.... if I use dual core optimization in Usine in conjuction with multiprocessing in Live, I am unable to use any Usine "sidechain" inserts on other Live tracks without serious audio issues, ie: crackling and bizarre comb filtering effects so it is obvious that Usine vst will not tolerate having multiple instances of itself processed on different cores with two separate muliprocessing switches enabled.

If I disable dual core optimization, with Ableton Live multiprocessing enabled, there is no longer audio issues related to processing so it is clear that Usine vst is not leaving it solely up to the host to distribute cpu.

If I disable dual core optimization, with or without Ableton Live multiprocessing enabled, Usine's performance ability goes way down so it is clear that Usine vst making it's own rules for cpu useage, independantly of the host .


I obviously know a lot less about how these things work than Senso but everything I do know strongly suggests that vst cpu distribution should be handled by the host and that "dual core optimization" really ought to "HAVE NO EFFECT" in the vst version of Usine.

I am sorry if I am making a bunch of noise about nothing, but it seems as though there may be a problem and if so, it is a problem that is seriously effecting my intended usage of Usine. If this is the intended behaviour of Usine vst, then I guess I am going to have to completely rethink my approach...

[gurulogic]

Another fun little factoid...

With dual core optimization disabled, inserting a Usine sidechain instance onto any track in live forces that track and any other track sidechain instances are inserted on to use only the one cpu core.

So in essence, all those beautiful little sidechain inserts are detrimental to performance in a multiprocessing capable host unless you enable dual core optimization and just ignore the crackles and awfull noises that result...

[edited: sorry, misinformation again!]

[ edit: Ok, here's what's really going on. Inserting Usine VST with dual core optimization disabled into a Live set completely disables all host multiprocessing....period! this means that any project that was happily balanced amongst multiple cores absolutely will not run on more than one core with Usine inserted anywhere in the project! (insert curse words here 'cause my parade just got totally rained out)]

If there is any chance of a quick fix for this I would be most appreciative as I have invested alot of time and effort into Usine and would like to actually be able to use some of the cool stuff I have built 1/4 use of my cpu is not going to do it for me though...

[senso]

thanks for the report,
I have a couple of comments about you post

- Usine vst isn't a classic vst but take much more resources than any other vst's .
- you use multiples instances of Usine so core repartition can be unpredictable, the because Usine vst is build to be used as a single vst instance. Your config hasn't been already tested. (Maybe I should!)

I know that the vst isn't perfect at this time, but in my mind the vst functionalities was made to complete leak of most other host, not to be the essential part (ie. one Usine on each track). Usine is too big for that!
The solution should be to think about a new vst design lighter than the actual.

[gurulogic]

Hi Senso, I may have been editing my post to include more information while you made your reply so you should check again the last part where I describe how Usinene forces my multicore aware application to use only one core.This makes any use of Usine vst sort of impossible for me.

I appreciate that there may be limitations, and I think that the way the vst has been implemented is largely genius, however some of these limitations are unfortunately a huge setback for me. Back to the drawing board I guess...

[edit: I guess I am being dramatic. Usine should still be quite usable as a send effect manager in my setup though I sure was excited about being able to insert multiple feeds into one effects rack.]

[senso]

With dual core optimization disabled, inserting a Usine sidechain instance onto any track in live forces that track and any other track sidechain instances are inserted on to use only the one cpu core.

It's a well known problem called 'Bottle neck'. The host choose this option to be sure that the audio synchro between threads is optimal.
I don't want enter in details but the multicore processing under Windows force the host to adapt the latency (with complex buffering), because the threads precision time is around 50ms (more in reality). But if you want side chain you can't have such latency between your audio flows, so the host disables the multicore on concerned tracks.

It's absolutely normal!!
Not a bug neither an Usine problem just a consequence of Windows thread scheduling...
Welcome in the 'real' word: multithread = latency !

I spent a lot of time to find the best way for Usine, maybe my small experience will lead me to consider another solution, but actually the dilemma is between sidechain-low-latency and multicore.

[gurulogic]

Thanks for the good explanation. I think I finally understand. It is unfortunate there is no two of the best worlds option between sidechaining and multicore but at least now I know what I have to work with so I can plan accordingly.

[gurulogic]

Good news! Hours of trawling the internet and trying every dodgy option I could find led me to jsoundbus vst which in initial tests is working like a charm to stream audio from Live tracks to Usine vst with no crackles with only an additional 6ms latency, low cpu useage, multiprocessing intact and automatable bus selection within Usine too!
I am once again happy and ready to torture my poor computer with making too many Usine instances do way too much again

[senso]

thanks for this great tip!

http://www.jaltoh.6x.to/ (click on "here")
http://geocities.com/asenov_z/files/jsoundbus.zip

Start the first vst host and instance one jsoundbus.dll. Set channel 1 to 'send'. Select to which bus you want to put data using the combobox on the left; note that only single Soundbus should get or put to a bus or the sound might get distorted. Then start the second VST host and instance another Soundbus. Set channel 1 to 'receive' and the combobox to the buffer the first Soundbus is outputting to. Now you should hear something in the second app. Click the 'Reset buffers' button to set the bus size to a safe value and inform the host about the receiving latency so it can compensate (if capable).
Note: Only the receiving soundbus can change latency on a specific bus.
Another note: Some hosts tend to click & pop even if the latency is set high enough. (like Synthedit) To resolve this try increasing the host's process base priority in Windows Task Manager.

[23fx23]

that look nice, will have a look, also anyone tried fx teleport with usine, couldn't it drive midi/audio to usine via local network.
from what i remember i had very correct latency from one computer to another, it might be possible on only one computer..