AES 5 - A FREEZE in time

Okay, in the last blog we said it was going to get heavy, in this blog we strap onto a rocket for the moon.

It takes a lot to make a complete development, the AES is proving to be no less than expected. In the previous blogs we have managed to get away with our previous max audio knowledge. In this one we cant!

Effect 4 - time freeze.

Get ready for around 4 hours of content covering everything from the sound theory within through to the javascript to make it.

This entire blog is about the creation and automation off the ability to freeze a 'picture' of sound to repeat for infinity, then allow infinite stacking of sound pictures on top of each other.

In simple terms: capture and loop single samples of audio, without sounding like a robot.

So scope out whether this is something you want to invest your time in, have a quick peak at the video below that explains what we are doing and how we are doing it. The honest answer is there are a lot of 'on the job' style of videos about to be linked so silence, swearing and making massive mistakes is to be expected!

...Im glad you've decided to take this journey with us!

To try and reduce the madness everything is split into parts, skip to what interests, or is of use to you going forward!

Part 1: The Audio effect (MSP processing)

Part 2: Automation (Javascript object making)

part 3: everything else (clean up, testing, integration)

To try and make it easier we've colour coded the headings so you can scroll down and find them quickly!


Part 1: The Audio effect (MSP processing)

As always we will refer to the video above with time links so you can get right into the action and avoid all the 'ems' and 'errs'.

As explained above our goal is to be able to create freezes of time.

We sampe quick heavily from Jean-Francois tutorials for this as we simply didnt have enough sound theory to be able to solo it.

We also relied quick heavily on the cycling 74 tutorials for PFFT~ knowledge.

02:00-12:20 Building the audio effect.

The creation of the audio is what we would like to think as quite straight forward. Transfer the audio into the audio domain, capture a single sample of data when a trigger is received, play that audio back out on repeat thanks to our fft maths.

The real confusing part here is understanding that audio no longer has a length of time, but rather a number of bins per sample.

PFFT allows us to look at a define amount of data (normally 512 bytes) at the same time across the entire audio spectrum, we definitely recommend checking out the links before this chapter to hear someone explain them who understands whats happening better!

17:00 - 22:00 Splitting the buffers

As we pull away from the currently documented information around the creation of PFFT~time freezes we open up the buffer within each fft patcher so that i can be dynamically named.

At this point in the video we have two of the same patcher loaded, but we look at how to give each of them different audio processing/buffers so that we can capture 2 different samples.

There are also some beautiful examples of my angelic singing within this section.

23:00 - 25:00 introduction to mixing matrix~

To top the development of the audio processing off we look at how to properly mix audio signals using the matrix object.

A pretty nice summary of the whole pfft freeze thing!

Part 2: Automation (Javascript object making)

1:00 - 2:30 Little introduction to what is about to hit you

3:00 - 6:00 We remember its been 2 years since we looked at javascript in max

6:30 - 8:30 psuedocode for what we are actually wanting to acheive

This is generally a good life lesson, but when a problem is quite complicated - mapping out the steps you need to achieve it is always a winning solution. You can refine it to as many steps as you need, in as complex a language as you want. Sometimes even writing it out in plain English as in the video can be a game changer for productivity and work flow.

The general idea is that we want to:

1: when user freezes tone, set a flag saying that its buffer is full

2: create a new freeze buffer ready to be used

3: increment counter to use new freeze vs old

4: reset option to delete overflow freeze

10:30 - 30:00 developing the increment system

To handle above we do a bit of research into Javascript within Max between 6:00-10:: so if you want to know the steps we take to understand problems watch us fumble through the reference documentation and tutorials.

We use the max reference for direct help about 15 independent times in this video. THE HELP IS YOUR FRIEND, USE IT!

its quite hard to document the process of the video above as its a fairly simple but long process as we relearn javascript.

By creating a global counter object we are able to track what freeze is currently active, and which do and dont have audio in them. On this principle we name the buffers of each freeze object, create the new ones and route the bang messages correctly. That counter is important!

31:00 - 36:00 creating and resetting the extra pfft objects

Getting back into the swing of things, we get into the meat on bones of the JS object - being able to create, manipulate and delete objects on the fly within the max window.

With a few more dives into the documentation we put together something that can reset the patcher

41:00 - 53:00 Javascript creation, cleanup and connection of new objects

Little deeper again we look at the connection, routing and trigger of objects from within code. Max can start to look like a pretty scary place from the inside of the JS editor. But as far as DE (development environments) go its actually amazingly quick and error catching friendly.

In the past few updates the autowatch and auto compile features also really help!

55:00 - 1:02:00 Routing and banging the correct pfft object in after creation

Finally we have to plug the created pfft objects into our counter system, and make sure that they get pressed when they are meant to.

A fairly simple finale to our pretty heavy session of audio and javascript.

Even more impressive is the fact we have another hour and a half of perfecting the patch and all its elements.

part 3: everything else (clean up, testing, integration)

You can probably stop here if you're adventurous and like a challenge of rounding of the patch yourself.

Otherwise see the time stamps for the headlines of what to expect.

01:00 - 3:30 Altering the wiring slightly for visual feedback

Super simple stuff, just more javascript coding if you're into that. We change the way that the pfft objects are connected to up be able to visually see when one is being triggered or not.

4:30 - 43:20 implementing a mixing matrix

In the past we have simply been using *~ to mix our audio stems together as there have either been:

A: only 2 audio stems

B: no importance to preserving the audio level.

Due to the potential number of outputs from the freeze effect it was important to make sure that we couldn't have 10 outputs all mixed at 100% volume (goodbye speakers/hearing/social life).

To overcome this we use the matrix~ object in non-binary mode. This is vital as it allows us to specify the input to output channel mix as well as specify a gain percentage for each (E.G we can mix 3 inputs into 1 output each at 33% gain - roughly giving us an equal output volume).

When enabled (by having a third value in the matrix object line (matrix~ 2 1 1.)) we can simply specify thee input numbers to correctly pass out data to the outlets.

As we are mixing down to stereo or mono its about correctly taking all inputs and sending to outputs. for example if we had a matrix with three inputs, and two outputs - we have a potential 6 combinations for mixing (input * output). In the conventional matrix~ object we simple say route input 1 to output 1 (pass message [1 0]) in non-binary we also specify the gain of the output [1 0 0.33]. So to correctly route all three of our hypothetical inputs to our outputs we would need to pass the following messages:

In Out Gain

0 0 0.33

1 0 0.33

2 0 0.33

0 1 0.33

1 1 0.33

2 1 0.33

Or as a single split string of:

"0 0 0.33,1 0 0.33,2 0 0.33,0 1 0.33,1 1 0.33,2 1 0.33"

We use 0.33 for our gain as each output is getting three audio stems mixed across it (3*0.33 = 1) so the hope is to maintain a similar input to output volume level.

The problem with our patcher is that we dynamically add objects on the fly...

45:00 - 1:15:00 setting matrix to mix down audio correctly on button press

Back into javascript we go. On reflection at the video I think if we had correctly defined the matrix~ in non-binary mode we would have been able to directly send it messages vs the hacky way we do it via a linked message input.

But if its not broken, and the client doesn't know - it wont matter!

We take the theory set out above and add special javascript commands to update a message object with the correct information then 'bang' it into the matrix~

1:15:00 - 1:20:00 encapsulating the effect

Wrap everything all nicely up in a encapsulated bow we add out inlets and outlets and route everything all over the place.

1:22:00 - 1:30:00 fixing pathing issues within max

A little clean up on the patching pathing issues withing max itself (we ran into the problem of the harddrive seeing multiple copied of the pfft object we were trying to load, it started loading an old revision.

We fix it by adding max paths when an object is saved. Max should be smart enough to go on the most recent file, closest to the main loading patch.


AES part 6 is up next. Adding the effects together and creating the modularity system!

Join the PfP mailing list