Object order processing silliness

So I have been thinking about this a bit and another thread with the issue of creating stereo patches from mono objects and phasing issues reminded me to post something.

The patcher takes a naive view on how to order the processing of objects, so top to bottom, left to right.

So if we take something simple like this:

The patcher generates the following code:


		/* <object calls> */
		objectinstance_sine__1_i.dsp(0 , ZEROBUFFER, ZEROBUFFER, net0, PExch[PARAM_INDEX_sine__1_pitch].finalvalue);
		objectinstance_mix__1_i.dsp(ZEROBUFFER, net0, net3Latch, net1Latch, net2, PExch[PARAM_INDEX_mix__1_gain1].finalvalue, PExch[PARAM_INDEX_mix__1_gain2].finalvalue, PExch[PARAM_INDEX_mix__1_gain3].finalvalue);
		objectinstance_out__1_i.dsp(net2, net2, displayVector[3], displayVector[4]);
		objectinstance_sine__2_i.dsp(0 , ZEROBUFFER, ZEROBUFFER, net3, PExch[PARAM_INDEX_sine__2_pitch].finalvalue);
		objectinstance_sine__3_i.dsp(0 , ZEROBUFFER, ZEROBUFFER, net1, PExch[PARAM_INDEX_sine__3_pitch].finalvalue);
		/* </object calls> */

		/* <net latch copy> */
		for (i=0; i<BUFSIZE; i++) {
			net1Latch[i] = net1[i];
		}
		for (i=0; i<BUFSIZE; i++) {
			net3Latch[i] = net3[i];
		}

Now that code might look a bit nonsense to the non-programmers here but you can see it is processing in this order:

sine_1
mix_1
out_1
sine_2
sine_3

So that doesn’t really look right, how can it be mixing the three sine waves if only one sine wave has been generated!

It gets around this conundrum using some extra audio buffers, net1Latch and net2Latch. It copies the current buffers to the latch buffers and then uses those next time round

So it is using the current buffer for sine_1, and the previous latched buffers for sine_2 and sine_3 in the mixer, you can see them in the code:

objectinstance_mix__1_i.dsp(ZEROBUFFER, net0, net3Latch, net1Latch, net2, PExch[PARAM_INDEX_mix__1_gain1].finalvalue, PExch[PARAM_INDEX_mix__1_gain2].finalvalue, PExch[PARAM_INDEX_mix__1_gain3].finalvalue);

ZEROBUFFER, net0, net3Latch, net1Latch are the inputs to the mixer, the ZEROBUFFER is just an empty buffer where the bus_in input is not being used in the patch.

So sine_2 and sine_3 are in phase with each other but delayed, sine_3 is not in phase with them as it is not delayed.

So really we want to process things in this order:

sine_1
sine_2
sine_3
mix_1
out_1

Nice and neat, no need for any delayed buffers, everything in phase, no latency being introduced on some of the audio paths.

So I’m thinking of changing the way the patcher generates this code so they would be in that order.

So instead of top to bottom, left to right a “Directed Acyclic Graph (DAG)” would be used instead, for info start here: Directed acyclic graph - Wikipedia

The eagle eyed may see that this is not possible as we can have feedback in the patch, so I could feedback the mixer to the freq inputs of the sine waves:

So what we have now is a “Directed Cyclic Graph (DCG)” which is a bit more of a pain in the backside, but we can convert the DCG to a DAG by identifying feedback cycles and breaking them, we then add output nodes where these cycles happen (latched buffers) and we then add input nodes that use these latched buffers. Like magic it all should work.

So it’s a bit of work this, is it worth it, would people like it to work this way, would it break things, etc, etc?

Yeah… I don’t get any of this :sweat_smile:

it would be better if you have a practical example of how to break the cycle.

are you planning to rework the patcher?

Yeah the patcher would change, but not in a way the user would see.

What would change is the bit that generates the c++ code that is then compiled and run on the Ksoloti.

The cycle breaking is not the same as the issue we currently have, which is objects being processed in the wrong order (I probably got too detailed!)

So the basic idea is that the code generated, orders things more in the way you would expect it to, so do all the sines, then do the mix then send it to the output.

Currently it is not working this way unless you lay out you objects in the patch to force it to do this. So currently if you change positions of objects it can change the sound of the patch, the idea is to stop this happening.

So the idea is that you can lay out the objects in any way and the patcher will work out the ordering so you done;t have to think about it.

so in the picture with the 3 sines and the mixer, how do you position the objects so that there’s no phasing?

With the current patcher:

This obviously gets more complex the more objects you have :slight_smile:

Here you can see the issue, look at the amplitude of the sine wave:

multi-fx.axp (78.9 KB)

so this patch would be very complicated to reposition the objects?

Well I wouldn’t want to try :slight_smile:

So the idea is that by changing the way the patcher generates the code we as users would not have to worry about putting the objects in the correct places, we could lay it out like you have in a way that makes sense to us and is easy to “read”.

I’m guessing this would be good for you.

This is a very interesting topic, because it is known that in axo the sound depends on the positioning of the objects, which does not happen with my clavia G2.

FWIW, in version 1.1.0 each object’s tooltip shows its position in the execution order, for example 1/56, 35/56… So at least we can do a basic check as to how objects stand towards each other by comparing that number in their tooltips.

Definitely a good topic to look into! Isn’t the current sorting a type of (naive indeed) DAG as it is?

It would indeed be preferable to have the execution order go a bit more intuitive IMHO. Is it going to be very complicated? Hoping that it may just be a matter of passing a custom comparator inside the functions in Patch.java:

SortByExecution()

SortParentsByExecution()

I did some work for sorting the object list in the object finder in a bit more refined way, and for that I created the class AxoObjectIdComparator.java. Would be nice if sorting execution order could be implemented in a similar comparator class, using the DAG/DCG topology you mention?

As for detecting feedback, would it be as simple as checking after DCG sorting if an outlet of a higher-sort-number object goes to an inlet of a lower-sort-number object?

Say we don’t have any object feedback in a patch, would the DAG topology allow us to avoid the multiple buffering that is currently going on in the code generator?

What may complicate things further are subpatches though - do we calculate everything inside the subpatch on its own DAG/DCG, then treat the subpatch inlets/outlets on the parent as if it were a regular object, sorted in with the other parent objects? Would this introduce delays or unexpected behaviour inside the subpatches?

Now the ultimate Dream of a New Order would be a graphical overlay showing arrows and numbers between the objects… or might this be overkill…

Finally, another thought on feedback paths - would it make things easier if we simply had a dedicated “feedback creator” object, which users would patch in between objects to create a feedback-able (and feedback-safe) signal? Internally, that object would simply contain a one-buffer (one-sample??) delay? This would make audio feedback a more conscious choice, and might also solve the problem what happens if you patch an object’s output back into its own input (love doing this on my physical modular)?

Yeah it’s the sorting that is the issue, I didn’t make that clear!

For the feedback any cycle just needs to be identified and then broken, an “invisible” outlet buffer object added to the output and an “invisible” inlet buffer object added to the input. Like your “feedback creator” but automatic. This then leads to a new DAG we will use for the code generation which can be sorted by execution order. So only extra buffers will be needed for the feedback objects, no latch buffers should be needed for normal processing. That’s the idea anyway!

Any subpatch would be treated totally separately with its own graph, it is treated just like any other object in the parent. I’m not sure why this would introduce delays or unexpected behaviour? I might well be missing something though :slight_smile:

With the automatic feedback objects or your “feedback creator” object I’m not sure we could get a single sample feedback, the outlets and inlets are buffers?

It’s all a bit pie in the sky, just something I have been thinking about. It would also impact the sound of existing patches I guess which may not be wanted!

I like the idea a lot. Do all the silliness under the hood. Leave the brain power for creating music.

That does not sound like an issue, if a patch really sounds noticeably different users will either have to rework it for the new version, or keep an older Patcher version stashed somewhere for those patches?

1 Like

I have been looking at the spiders web of code to see how it is all done now, gradually getting my head around it.

I’m thinking we may be able to get away with keeping most of the existing code generation code, so:

  1. Detect cycles, add invisible objects to make DAG.
  2. Topological sort to give us our order
  3. patch.objectInstances then in this order

So when the existing code gets this order it will not use any latch buffers as there is no need as everything is in the correct order and there are no cycles (that it knows about).

1 Like

A little work through:

So lets say we have an out of order two sines into mixer into output:


sine1 instance -> mix1 instance

sine1 instance -> sine1 instance (patcher will latch this)

mix1 instance -> out instance

sine2 instance -> mix1 instance (patcher will latch this)

First remove cycles by adding instances:


sine1 feedback output instance (output only, delayed by 1 buffer) -> sine1

sine1 instance -> mix1 instance

sine1 instance -> feedback input instance (inlet only)

mix1 instance -> out instance

sine2 instance -> mix1 instance

Now topological sort to get order to get the order


sine1 feedback output instance (outlet only, delayed by 1 buffer)

sine1 instance

sine1 feedback input instance (inlet only)

sine2 instance

mix1 instance

So I’m guessing at this point as far as the code generator is concerned it has all the inputs it needs at every stage and just doesn’t use any latch buffers at all.

I am looking at the Net stuff in the code and am A little confused.

So a net can have multiple outlets, I’m ok with that.

It seems though a net can have multiple inlets as well, I have no idea about this. On the patcher an inlet can only have one wire attached to it can’t it?

Never dealt with that code, first time I am hearing this…

Correct, I assume - unless a modulation or midi assignment also counts as an invisible “wire attached”?

mmm, good idea. let me look at that…

I think the modulation stuff Is done separately.

I have put a little test in the code generator that breaks if there is more than 1, just running thorough the batch test to see if it is hit…

1 Like

Just a little update for everyone, there is already code in the firmware that does this different sorting, it was just never enabled!

I have set it up so that I can test it for a while to check everything works as planned!

3 Likes

someone should find Johannes and have him make a thorough documentation of the undocumented features
will it be enabled in patcher 1.1.0?