Author |
Message |
Acoustic Interloper
Joined: Jul 07, 2007 Posts: 2067 Location: Berks County, PA
Audio files: 89
|
Posted: Tue Aug 21, 2007 7:59 am Post subject:
FFT / IFFT in ChucK Subject description: Does ChucK do frequency domain? |
|
|
Is there FFT and inverse FFT or similar capability in ChucK, for manipulating audio spectra? Can't find it in the on-line docs, and my machine with ChucK is undergoing video card surgery today. Thanks. _________________ When the stream is deep
my wild little dog frolics,
when shallow, she drinks. |
|
Back to top
|
|
|
skwrKing
Joined: Jul 24, 2007 Posts: 16 Location: MI
|
|
Back to top
|
|
|
ge
Joined: Aug 13, 2006 Posts: 108 Location: Palo Alto, CA
|
Posted: Tue Aug 21, 2007 3:29 pm Post subject:
Re: FFT / IFFT in ChucK Subject description: Does ChucK do frequency domain? |
|
|
Acoustic Interloper wrote: | Is there FFT and inverse FFT or similar capability in ChucK, for manipulating audio spectra? |
Good question. FFT/IFFT, along with a few basic spectral feature extractors are due to be in the 1.2.0.9 release - happening this week!! This is part of a new side of the programming - audio analysis that can be combined in real-time with synthesis. Rebecca Fiebrink and I are the authors, and will be presenting it next week at the International Computer Music Conference in Copenhagen.
Stay tuned. If you haven't yet, you might want to join the 'chuck' mailing list to get useful announcements about chuck:
http://chuck.cs.princeton.edu/community/
Thanks!!!
Best,
Ge! |
|
Back to top
|
|
|
Acoustic Interloper
Joined: Jul 07, 2007 Posts: 2067 Location: Berks County, PA
Audio files: 89
|
Posted: Tue Aug 21, 2007 3:55 pm Post subject:
Re: FFT / IFFT in ChucK Subject description: Does ChucK do frequency domain? |
|
|
ge wrote: |
This is part of a new side of the programming - audio analysis that can be combined in real-time with synthesis. Rebecca Fiebrink and I are the authors, and will be presenting it next week at the International Computer Music Conference in Copenhagen. |
Thanks for the heads up, and could you let us know when a copy of the paper becomes available on line. Thanks for that, too.
I'll join the mail list, and I am interested in what you are doing with audio analysis. Being an acoustic semi-purist working with mic'd instruments & voices, I am interested in, among other things, real-time, lossless, reversible signal processing transforms as a primary electronic means of extending the capabilities of acoustic instruments and spoken/sung word, initially stringed instruments in my own case. Sound is an amazing thing, and recombining chunks of real-time performance data (think string rewriting as in GA chromosome manipulation) at one or more phrase-oriented levels of structure (note, bar, chorus, syllable, word, sentence, etc.) is where I'd like to head. No instrument synthesis per se, and only real-time sampling of performing instruments or voices, so I guess this is FX or maybe meta-FX. Still looking at software framework alternatives.
Have fun in Copenhagen. _________________ When the stream is deep
my wild little dog frolics,
when shallow, she drinks. |
|
Back to top
|
|
|
ge
Joined: Aug 13, 2006 Posts: 108 Location: Palo Alto, CA
|
Posted: Tue Aug 21, 2007 4:23 pm Post subject:
Re: FFT / IFFT in ChucK Subject description: Does ChucK do frequency domain? |
|
|
Acoustic Interloper wrote: | Thanks for the heads up, and could you let us know when a copy of the paper becomes available on line. Thanks for that, too. |
You can find our paper here:
http://soundlab.cs.princeton.edu/publications/uana_icmc2007.pdf
The analysis framework is in place, but should be considered to be in the early stages (we are working to add many features).
Quote: | I'll join the mail list, and I am interested in what you are doing with audio analysis. Being an acoustic semi-purist working with mic'd instruments & voices, I am interested in, among other things, real-time, lossless, reversible signal processing transforms as a primary electronic means of extending the capabilities of acoustic instruments and spoken/sung word, initially stringed instruments in my own case. |
Awesome, definitely keep us posted on your work!!
Rock on.
Quote: | Have fun in Copenhagen. |
Thanks!! |
|
Back to top
|
|
|
Kassen
Janitor
Joined: Jul 06, 2004 Posts: 7678 Location: The Hague, NL
G2 patch files: 3
|
Posted: Wed Aug 22, 2007 4:05 am Post subject:
Re: FFT / IFFT in ChucK Subject description: Does ChucK do frequency domain? |
|
|
WOW! That looks insanely great!!!!!!
Excelent work!!! _________________ Kassen |
|
Back to top
|
|
|
ge
Joined: Aug 13, 2006 Posts: 108 Location: Palo Alto, CA
|
|
Back to top
|
|
|
Acoustic Interloper
Joined: Jul 07, 2007 Posts: 2067 Location: Berks County, PA
Audio files: 89
|
Posted: Wed Sep 05, 2007 6:47 pm Post subject:
Re: FFT / IFFT in ChucK Subject description: Does ChucK do frequency domain? |
|
|
ge wrote: | Rebecca Fiebrink and I are the authors, and will be presenting it next week at the International Computer Music Conference in Copenhagen.
Ge! |
How was the conference? Any recommended papers? (In addition to ChucK ) _________________ When the stream is deep
my wild little dog frolics,
when shallow, she drinks. |
|
Back to top
|
|
|
ge
Joined: Aug 13, 2006 Posts: 108 Location: Palo Alto, CA
|
|
Back to top
|
|
|
Kassen
Janitor
Joined: Jul 06, 2004 Posts: 7678 Location: The Hague, NL
G2 patch files: 3
|
Posted: Sat Sep 08, 2007 6:17 am Post subject:
|
|
|
That was great! You should get Guitar Hero controlers with USB converters and perform it live in ChucK! _________________ Kassen |
|
Back to top
|
|
|
Acoustic Interloper
Joined: Jul 07, 2007 Posts: 2067 Location: Berks County, PA
Audio files: 89
|
Posted: Sat Sep 08, 2007 6:55 am Post subject:
Re: FFT / IFFT in ChucK Subject description: Does ChucK do frequency domain? |
|
|
Wow, Copenhagen is nothing like I imagined it! Maybe next year it'll be somewhere nice and laid back, like Princeton _________________ When the stream is deep
my wild little dog frolics,
when shallow, she drinks. |
|
Back to top
|
|
|
Kassen
Janitor
Joined: Jul 06, 2004 Posts: 7678 Location: The Hague, NL
G2 patch files: 3
|
Posted: Sat Sep 08, 2007 7:01 am Post subject:
|
|
|
I have to say; this doesn't look much like the Copenhagen that I visited but admittedly I was only at the airport/station for 3 or so hours before traveling on to Gothenburg. Maybe the demons were being serviced at the time.
I kinda liked it, I speak Dutch, English and some Frisian and German so Danish looked vaguely familiar to me, most of it readable, I considered getting a Danish children's book to try my hand at it but in the end stuck to my sci-fi one. _________________ Kassen |
|
Back to top
|
|
|
Inventor
Stream Operator
Joined: Oct 13, 2007 Posts: 6221 Location: near Austin, Tx, USA
Audio files: 267
|
Posted: Sun Oct 14, 2007 6:57 am Post subject:
I hear voices... Subject description: In my headphones! |
|
|
Here is my second Chuck program that I wrote this morning - the power of Chuck makes it so easy!
It takes an FFT sample of the microphone, copies this to the output spectrum, then copies it again with fractional spacing of the spectral content, then again and again as many times as you want, creating multiple "voices". By setting the multiplier to 1.05, adding 4 voices, and most importantly dividing the amplitude of those voices by their index (the voices get quieter the more you add), it created some kind of voice accentuation effect.
When I sing into the mic with my horrible untrained voice, it has kind of a full, rich, real-singer's quality to it. You can adjust the control variables at the top of the file to create more or fewer voices or to space them out differently in the frequency domain.
I'm curious, is this effect commonly done in music? It seems like it would be a nice effect for a singer.
Inventor
who is having way too much fun with this new music hobby!
Description: |
Makes multiple copies of the mic input and spaces them apart in the frequency domain by even multiples. Creates a full, rich sounding voice - even my cheezy voice sounds like a singer's voice somehow if that is possible... |
|
Download |
Filename: |
voices.ck |
Filesize: |
1.64 KB |
Downloaded: |
355 Time(s) |
|
|
Back to top
|
|
|
Inventor
Stream Operator
Joined: Oct 13, 2007 Posts: 6221 Location: near Austin, Tx, USA
Audio files: 267
|
Posted: Mon Oct 15, 2007 11:59 am Post subject:
|
|
|
Well, i played around for several more hours with the FFT/IFFT code posted previously, and I finally got a good result. The attached file takes my female friend's voice and adds in subharmonics that are compressed in frequency. This has the effect of making it sound like there is another male voice there next to her.
Adding higher frequency content turned out to be very bad, making a tinny, computery, or even squeally sounding train wreck of the test song. So that's why I switched to subharmonics. It puts subharmonics at (k*ni)*f where k is 0.5, ni is 1 to 4, and f is the frequency index. Any subharmonics that fall off the edge of the array are ignored. It also attenuates the subharmonics by Math.pow(af, n-1) where af is 0.5, so they get smaller - I found this attenuation was necessary to prevent computery sounding effects.
Also, this one reads in from a *.wav file and I used the rec-auto.ck file to write out to a *.wav file as well. This way I do not have the confusion of hearing my own voice in my head in addition to the effect, which is not at all realistic, and I also don't have the confusion of hearing the original song plus the effect which I had before. Now it is better quality and reproducible as well.
Description: |
Takes female singer's voice and adds subharmonics that sound like an additional male singer in synchronization with her. |
|
Download |
Filename: |
voices_from_file.ck |
Filesize: |
1.86 KB |
Downloaded: |
304 Time(s) |
|
|
Back to top
|
|
|
Inventor
Stream Operator
Joined: Oct 13, 2007 Posts: 6221 Location: near Austin, Tx, USA
Audio files: 267
|
|
Back to top
|
|
|
Acoustic Interloper
Joined: Jul 07, 2007 Posts: 2067 Location: Berks County, PA
Audio files: 89
|
Posted: Wed Oct 17, 2007 6:44 pm Post subject:
|
|
|
Thanks for posting this. I finally got time to update my ChucK installation and try it out. Great example!
I have been using Ableton Live, but I am not satisfied with the out of the box FX, and some timing issues, so I definitely plan to get to writing some code. But finding time has been tough. I want to try some things out in ChucK and MAX/MSP; Suppercollider was a contender, but it appears not to have the "unit analyzer" abilities of MSP or that direction of ChucK. I am very interested in deriving effects parameters from audio streams, e.g., LFO from finger pattern speeds, which requires some analysis. Your code will help get me back into trying some ChucK. I'll let you know if I do anything interesting.
Have fun! _________________ When the stream is deep
my wild little dog frolics,
when shallow, she drinks. |
|
Back to top
|
|
|
Acoustic Interloper
Joined: Jul 07, 2007 Posts: 2067 Location: Berks County, PA
Audio files: 89
|
Posted: Wed Oct 17, 2007 6:47 pm Post subject:
|
|
|
PS: The piece I have posted at http://www.virb.com/dparson uses some pitch shifting and also some time stretching capabilities of Live. They are OK, although the time shifting in particular is a little warbly. My wife's sister spotted my wife's voice-sound in a pitch shift of my daughter's reading in that piece.
Of course, about 50% of the people calling us on the phone mistake my voice for my son's or vice versa. No picth shift needed. _________________ When the stream is deep
my wild little dog frolics,
when shallow, she drinks. |
|
Back to top
|
|
|
Inventor
Stream Operator
Joined: Oct 13, 2007 Posts: 6221 Location: near Austin, Tx, USA
Audio files: 267
|
Posted: Wed Oct 17, 2007 7:55 pm Post subject:
|
|
|
Acoustic Interloper wrote: | PS: The piece I have posted at http://www.virb.com/dparson uses some pitch shifting and also some time stretching capabilities of Live. They are OK, although the time shifting in particular is a little warbly. |
The FFT/IFFT code I wrote has some side effects also. The copied voice sounds a bit "computery" or as my mom described it, "bionic - like the bionic man on TV". I think the problem can be minimized with interpolation but I'm not sure how to interpolate complex numbers - does anyone know this?
And thanks for the link to your Ordinary Machinery song, I'm listening to it now and enjoying it.
I believe your LFO concept, if I understand it properly, can be quite readily done by post-processing the FFT output. For example, find the peak frequency and count the number of FFT samples between that time and the next time the peak frequency changes by more than a certain amount, then converting this time into an LFO frequency. Something like that might work. Then the LFO frequency would track your picking rate - is that what you want to do? |
|
Back to top
|
|
|
kijjaz
Joined: Sep 20, 2004 Posts: 765 Location: bangkok, thailand
Audio files: 4
|
Posted: Wed Oct 17, 2007 11:33 pm Post subject:
|
|
|
I was doing FFT and IFFT that day..
in a not very complex set up..
I tried doing IFFT transform but with ramdomed timing..
for example.. transform every Std.rand2f(0.3, 0.4) * ifft.size()::samp
it sounds more natural to me. |
|
Back to top
|
|
|
Acoustic Interloper
Joined: Jul 07, 2007 Posts: 2067 Location: Berks County, PA
Audio files: 89
|
Posted: Thu Oct 18, 2007 6:20 am Post subject:
|
|
|
Inventor wrote: | I believe your LFO concept, if I understand it properly, can be quite readily done by post-processing the FFT output. For example, find the peak frequency and count the number of FFT samples between that time and the next time the peak frequency changes by more than a certain amount, then converting this time into an LFO frequency. Something like that might work. Then the LFO frequency would track your picking rate - is that what you want to do? |
Actually, I want to determine the frequency components of the finger picking itself, regardless of the frequency of the notes being played. I think roughly the algorithm will be something like:
1. Find the "plucks," basically the attack points. Basically a search for peaks in envelopes.
2. Conceptually, treat the plucks as peaks in a smooth continuous waveform.
3. Do the FFT of that to get the low frequency components of the finger pattern.
For example, if I am playing 12/8 triplets using thumb-index-middle at a fairly steady speed, let's say 170 ms per pluck = a picking frequency of 5.9 Hz, then 5.9 Hz (or some multiple or submultiple of it) would be the LFO. Picking faster changes the LFO. Moreover, by accenting some strings more than others, or introducing irregularities such as rests into the picking pattern, you get other frequency components.
What you do with the LFO frequency(ies) is then wide open, both in taking (sub)multiples or inversions, and then in applying it(them) to the audio stream itself. The goal is to control everything by playing the acoustic instrument.
We'll see how it goes. Yesterday was the first time I got time to immerse myself in music for a whole day in over a month. Near the end of the day I stumbled into my first new tune on the banjo in months, after which my wife and I did an impromptu banjo duet that sounded really good (even though she plays some guitar but no banjo, and we never tried this before). Unfortunately it's back to the day job grind today. This is why I want a language or tool with production quality signal processing primitives, so I can spend the little time I have focusing on the music. Looks like I should get some time over the next month.
Take care. _________________ When the stream is deep
my wild little dog frolics,
when shallow, she drinks. |
|
Back to top
|
|
|
Inventor
Stream Operator
Joined: Oct 13, 2007 Posts: 6221 Location: near Austin, Tx, USA
Audio files: 267
|
Posted: Thu Oct 18, 2007 7:21 am Post subject:
|
|
|
OK, Acoustic Interloper, I read your post over and over because I'm no musician, but if I recall my Fourier transform theory properly, that 5.9 Hz frequency component is in there at the bottom of your spectrum. It might be hidden by a bunch of noise or sounds from your equipment, but it's there. So it is possible that applying a low-pass filter and a bunch of gain to your sample stream would isolate the signal that you want. You wouldn't want to build such a filter in real life, but on the computer its just numbers.
If the 5Hz component is too small, then we could normalize the signal, apply a threshold, and filter that to get your LFO. In fact, this can probably be done in the time domain completely without using an FFT. Besides, getting 5Hz resolution from an FFT on a sample rate of 44,100 Hz would take a really big FFT, like 32768 samples or so. This might be a compute problem, I don't know. Of course you could lower the sample rate to help a bit but then you lose information and you have to figure out how to change your sample rate which is another hassle.
Actually I have code for finding peaks and normalizing, its easy to rewrite as well. If you could send me a *.wav or *.mp3 example file, I might find the time to cook up an algorithm for you. I have a lot of time and enthusiasm on my hands right now - that is, until it passes and I get either bored or excited about something else. My email address is inventor-66@comcast.net for sending the file.
Now, I take it that you want a floating point number that tells you the frequency of the picking, right? Or do you want a sinusoid at the frequency of the picking? Or both?
I gather you will then use your banjo picking rate to do something else, either in the music stream or out of it. Oh, it would help to know what the maximum and minimum picking rates are. Just let me know if you would like some assistance on this, it sounds interesting to me and I like to collaborate. |
|
Back to top
|
|
|
Acoustic Interloper
Joined: Jul 07, 2007 Posts: 2067 Location: Berks County, PA
Audio files: 89
|
Posted: Thu Oct 18, 2007 9:07 am Post subject:
|
|
|
Oh, thanks. Attached is a short clip of steady banjo picking for you to pick apart! I leave for a trip shortly and have to pack, but this should be a decent starting point.
Inventor wrote: | If the 5Hz component is too small, then we could normalize the signal, apply a threshold, and filter that to get your LFO. |
This is something like what I had in mind.
Quote: | In fact, this can probably be done in the time domain completely without using an FFT. |
Yeah, this question came up from an attendee to a talk I gave on this paper http://home.ptd.net/~dparson/dafx06_dparson_final.pdf , where I am doing much more complicated analysis on a MIDI guitar signal. The attendee asked why I didn't use Fourier, and my answer was basicaly that it is simple enough to do it in time domain. The MIDI pickup on the guitar in conjunction with processing into the Roland GR33 guitar synth is isolating the "plucks" as MIDI noteon events, and my code analyzes those in the time domain.
In the case of the LFO, I don't have MIDI pickups on the banjo (and don't want them), so I want to do in software a subset of what the MIDI pickups-GR33 are doing. If I can locate discrete "plucks" by looking at envelope peaks, I can use the same algorithm to work strictly in time domain. I don't actually need all the data in the paper, although my picking will have more than 1 frequency component. For example, in 4/4 time, one of my finger patterns is (this is the pattern of the attached sample):
ONE and two AND three and FOUR rest
There are accents on the capitalized plucks, i.e., 1, 2.5 and 4, and a rest (skipped pluck) at 4.5. There are 7 plucks in all, then a rest. If the time between plucks is 170 ms., we have that 5 Hz fundamental, but there are other frequency components due to the accents and rests. I'd like to know those frequencies and their relative strengths, but I don't care about phase nor precise timing, which is why an FFT on a reduced data set may still be useful.
Quote: | Now, I take it that you want a floating point number that tells you the frequency of the picking, right? Or do you want a sinusoid at the frequency of the picking? Or both? |
One float is good, although as noted above, a float for the strongest finger frequencies would be good. Not sure how to use those other frequencies from the accents, rests, etc. Definitely a thought experiment up until now.
Quote: | I gather you will then use your banjo picking rate to do something else, either in the music stream or out of it. Oh, it would help to know what the maximum and minimum picking rates are. Just let me know if you would like some assistance on this, it sounds interesting to me and I like to collaborate. |
Let's say 80 ms. minimum between adjacent plucks, and 800 ms. max, although if you consider delays between accents, rests, etc., this could even get into several seconds. Really LFO!
So far I am thinking of modulating the usual stuff, pitch, amplitude, maybe even time. Stretch time of a secondary audio stream as a function of picking speed. Open ended problem.
The attached sample uses the finger pattern from Figure 2 of http://home.ptd.net/~dparson/dafx06_dparson_final.pdf with only G-B-D notes and no slides, chokes, etc., repeated 16 times. I could also do this with a MIDI guitar and capture both audio and MIDI, and give you precise timing from analyzing the MIDI with existing software, when I'm done traveling. I thought it best to start with a banjo.
Have fun! Thanks. This will be interesting. _________________ When the stream is deep
my wild little dog frolics,
when shallow, she drinks. |
|
Back to top
|
|
|
Inventor
Stream Operator
Joined: Oct 13, 2007 Posts: 6221 Location: near Austin, Tx, USA
Audio files: 267
|
Posted: Sat Oct 20, 2007 3:37 am Post subject:
|
|
|
Attached is my first attempt, it did not go very well. As we suspected, the 1 Hz to 10Hz component is buried by noise. This is especially true because of the way I had to get the banjo sample. I couldn't download the file directly, when I control-clicked using my Mac, it tried to download a *.php file for some reason. Also when I tried to dowload from the player it asked me if i wanted to purchase Quicktime! So this is one rare instance where the Mac is inferior to the PC - I cannot download your file!
So I taped it. I set up ChucK as a recorder and played your file over the speakers and it kind of worked but there is room noise. And that room noise, I think, is drowning out the LFO. Anyway, if the technique is so sensitive that it can't stand a little room noise, I think it's better to work with the banjo envelope peaks themselves.
Now I have a question: In ChucK, how do I make a patch that grabs the sample stream and puts it into an array? Is it just adc @=> array_name[], or something like that? I need it to be smart enough to give me only so many samples at a time and then not skip or overlap them. Then I can apply the peak processing and calculate the frequencies. Do you know how to do this?
Description: |
test code, does not work yet, do not download |
|
Download |
Filename: |
banjo1.ck |
Filesize: |
2.65 KB |
Downloaded: |
269 Time(s) |
|
|
Back to top
|
|
|
blue hell
Site Admin
Joined: Apr 03, 2004 Posts: 24075 Location: The Netherlands, Enschede
Audio files: 277
G2 patch files: 320
|
Posted: Sat Oct 20, 2007 5:03 am Post subject:
|
|
|
Inventor wrote: | when I control-clicked using my Mac, it tried to download a *.php file for some reason. |
What you can do is that you just give it a file name with a wav extension instead of the suggested filename and php extension and save it as that - the contents that get downloaded are wav audio, so the file will actually be usable. It's the same on a PC using Firefox BTW, a quirk of the forum software. _________________ Jan
also .. could someone please turn down the thermostat a bit.
|
|
Back to top
|
|
|
Inventor
Stream Operator
Joined: Oct 13, 2007 Posts: 6221 Location: near Austin, Tx, USA
Audio files: 267
|
Posted: Sat Oct 20, 2007 7:23 am Post subject:
|
|
|
Thanks for that tip, Blue Hell, it gave me a clean source file which contained a much more prominent 5 Hz peak. I also did something smarter-than-before: I applied a super-ideal low-pass filter by only examining the lower frequency contents in the various sections of the program: the first peak detector, the normalizer, the multiple peak detector, and the output printing routines. Can't get much better of a low-pass filter than zeroing out everything above a threshold frequency.
Now, when you play the banjo song on iTunes or whatever player you use, and run "chuck banjo1.ck", it will spew out the lowest 32 normalized frequency amplitudes, which is hard to follow so I also make a little ASCII frequency meter and you can watch the little "9" meaning 90% or above move about jittering with the actual banjo pick frequency. At least that's what I think it's doing, there's a clear peak in there from SOMETHING. Also you'll see a little 8 appear or the 9 appear somewhere lower in frequency, that's because it tracks multiple frequencies in an array, in this case anything above 80% of the normalized samples from 0 to 31.
Next thing to do is to actually convert the frequency number from the FFT into a frequency. This will require some intelligent thought as to some constant multiplier such as 1::second / samp * num_samples or something like that. I'll leave that for the next person to work on it as I am eager to get back to my composing of binary sequences. This should be enough code to get you started, Acoustic Interloper, unless I am sadly mistaken about something and it doesn't really work under your test conditions. Either way, I'm happy to make further contributions as I have not gotten bored or distracted yet, haha. Enjoy.
Description: |
This one works, at least in my tests. Takes microphone input and tracks 0-10 Hz frequency peaks. |
|
Download |
Filename: |
banjo1.ck |
Filesize: |
2.68 KB |
Downloaded: |
292 Time(s) |
|
|
Back to top
|
|
|
|