Joined: Apr 26, 2004
Location: Washington DC Metro
G2 patch files: 1
|Posted: Mon Jan 26, 2009 1:33 pm Post subject:
More details on StringPort
Subject description: Keith McMillen starts spilling the beans
|For those of you who don't know who McMillen is, he founded Zeta Music (maker of electric and MIDI violins, celli, etc.). He was involved in work at UC Berkeley's CNMAT which led to the creation of Open Sound Control (OSC).
Here's some of what he had to say about his new product, StringPort, on the midiguitar Yahoo Group:
The Analysis software that comes with StringPort constantly runs multiple FFTs
over each each string's audio signal. We get very rapid period analysis by
applying a sieve transform to the higher harmonics that emerge early from one of
the FFTs. (ie: If you see peaks at 900Hz, 800Hz, 700Hz, you can be pretty sure
the fundamental is 100Hz).
Shorter window size FFTs handle triggering and event analysis. We can detect if
the trigger came from the left neck hand or the right pluck hand. Handy.
I assemble all this into a descriptor frame I have named AIM (for Acoustic
Instrument Message). AIM contains continuous pitch, loudness, brightness (as
centroid), even/odd harmonic ratio, noise amount, and inharmonicity. Taken as a
group and applied to a synthesizer, the resulting sound feels very much under
your control. We had our violinist play a synthetic violin sound at NAMM and
most violinists could not tell that is was a synthesizer. (my version of the
AIM can be transmitted to certain software synths over sysex or CC commands,
over OSC or pretty much any transport including ethernet/UDP. It has to be fast,
though, as we generate 1000's of AIM messages every second.
Yes it all seems pretty demanding and difficult. However, there is great work
going on at secret MIDI gatherings, and support for these types of messages are
being included in plans for future standards.
Of course, subsets of the AIM packet can be used with other forms of synthesis.
We maintain a two channel control paradigm for each string - one the continuous
harmonic sound, the other for noise, pluck or non-harmonic results. These are
suggested implementations and other interpretations of the data are certainly
I chose a spectral descriptor scheme over physical controls like pick distance,
velocity, displacement, etc. Physical descriptors are just not adequate nor
universally understood. We rely on the physics of how we perceive sounds to
Details of frame rates, latency, interpolation, etc will be handled in later
Current and recent work on Soundcloud
Some old stuff on VIRB
Joined: Jul 29, 2004
|Posted: Tue Jan 27, 2009 12:09 am Post subject:
|What's an FFT?
Where there are too many policemen, there is no liberty. Where there are too many soldiers, there is no peace. Where there are too many lawyers, there is no justice.
Lin Yutang (1895-1976)