Multi-Touch Proto 2

[Scott Stites]

Hope this hasn't been posted elsewhere, but I caught this on MatrixSynth a couple of days ago. This thing (IMO) is sheer frikkin' genius!!

Watch the vid and go to his site and download the PDF of this guy's Masters Thesis - good reading (I know, because I only understand half of it ):

http://matrixsynth.blogspot.com/2008/12/multitouch-prototype-2.html

[neandrewthal]

OMG. Did he really just do the old tighten the skin of the drum while you bang on it trick....on a CONTROLLER? That's amazing!

[Scott Stites]

Yeah - that was the part that really got my attention!

[wetterberg]

[slabman]

It's a brilliant piece of lateral thinking. As I understand it, there are 16 'pickup' outputs. Each output contains a signal made up of any or all of 16 'carrier' sinewaves. The interface pipes the pickup signals to Max MSP which decodes the amplitude envelopes of the carriers and deduces the position and force of the percussive impacts.

This provokes two trains of thought for me:

1. Is the interface and decoding achievable with a DSP chip of some sort?

2. Could the matrix of sensors be physically rearranged (eg a 'marimba' layout)?

[wetterberg]

[Tasmanian Alkaloid]

That's incredible! I'd like something like this for a modular synth.

[slabman]

Not sure if this is of any value whatsoever but I'll float it anyway.

It's a slight diversion from the concept of the prototype. Imagine n carrier strips under one pickup strip. Multi-touch pressure on that pickup strip is going to give you an output signal composed of a mix of the carrier signals at various amplitudes. Choose the carrier frequencies to match the filter frequencies of an n-channel vocoder. Feed the pickup signal into the modulator input of the vocoder. Voila - touch-controlled filter bank.

Maybe...

[slabman]

I guess you could also use this technique to implement a touch sensitive waveform mixer as seen on the Electronic Sackbut

[urbanscallywag]

Wow!

I am surprised at the size of the copper strips. I would have imagined that you would need finer granularity for usable results, but apparently not.

[ian-s]

[Randy Jones]

Hi there. I'm glad people enjoy my project! If anyone is crazy enough to make one of these, I'd be happy to offer advice and software. I'd be interested to hear if other audio interfaces work OK. One could start with a single-point touch sensor very easily...

I am also looking into refining this into a product and seeing if people will buy some. Encouragement and ideas are welcome.

[urbanscallywag]

Nice work Randy.

Would you make more and narrower strips in a future version? Was the number of audio I/O the limiting factor in the original design?

[Randy Jones]

[blue hell]

[urbanscallywag]

[skrasms]

That's clever, multiplexing in the frequency domain to get 64 pieces of data over 8 transmission lines. Most of the commercial capacitive sensor chips use a single excitation plane (only one frequency) and then put a grid of sensors on top of that. They would take 16 channels to do the same thing.

Have you looked into higher frequencies (though you wouldn't be able to use a regular sound card anymore) and the dissipation factors of different types of rubber or polymers? Are you doing any compensation for environmental factors?

What is the overall system latency?

I'm really curious about this. You could take it much further with a dedicated DSP.

[urbanscallywag]

Hello Randy,

Congratulations on your master's thesis! I just made it all the way through. I really appreciate application oriented thesis work rather than purely academic/theoretical research.

How long would you say it would take to build the sensor box? I am interested in giving a hardware-only version a try. Do you have any ideas on what dielectric might be better than rubber? Would something less dense like foam give a "faster" response time?

I think I may be able to help with the DSP algorithms as well as bring everything into a cheaper self contained unit. Are you actively working on your project at all?


Nicholas

[Randy Jones]

Thanks Nicholas.

It's hard for me to answer how long it might take you to make a device. You have read what goes into it. It could really be a one-day project, but making it work well is likely to take longer. I have thought of making a how-to guide that would be more practically oriented than the thesis, with antenna layouts, and step-by-step construction details. On the other hand, I think everyone weird enough to make this thing in something like its present form will want to customize the design, probably a lot, so the general use of such a guide might be small. So I'm here to answer questions in the meantime, and happy to share software.

You want a very resilient material if the dielectric is providing the restoring force for the surface. The problem with less dense foams is that they take a second or so to recover. There would be no problem with using air for the dielectric and using some other kind of spring, though.

I think of the DSP as more or less done, but would be interested to hear what ideas you have. I am working on making a self-contained version of the device to sell. I think that openness about the design and helping people make their own will be an important part of this work. Monome has been inspirational in this regard.

-Randy

[urbanscallywag]

OK I should have clarified how long to build the actual device. Making it work (well) is a whole 'nother project.

I think the DSP could be done much more efficiently as well as bandwidth used more effectively (allowing use of cheaper ADC/DAC and processor). I am also interested in using your design without a computer.

For example you are sampling at 44.1 ksps but your lowest frequency of interest is 6 kHz, more than a quarter of the bandwidth isn't used. I also suspect the FFT is not the best way to do what you're trying to do. Individual filters may offer significant computational savings as well as better responsiveness. Your signals have effectively 0 bandwidth which lets you play some neat tricks in filter design.

Like I said I just finished reading your paper today and I found it quite inspiring. I have not studied physical modeling at all so that was an eye opener, and I think your processing of the actual multi-touch data is very nice. I think that some improvements could be made into acquiring and processing the signals to create that data.

I will try and do some simulations on the filters I was thinking of using instead of the FFT and get back to you. Knowing whether that will work will help me decide on what kind of converters to use.

[Randy Jones]

Making the antennas is probably the hardest part of the project. It can probably be completed in a day though. I'll add a few details that are not in the thesis:

My antennas are 1/2" 3M copper tape with approx. 1/16" spacing in between. laid down on 3 mil polyethylene film they are removable for experiments, yet do not seem to creep across the surface even after weeks of playing.

I arrange them like

[urbanscallywag]

Thanks for the construction tips. I don't think I'm mechanically fit enough to take this on. I wonder if the flexi-PCB stuff would be durable enough for this project.

Bandwidth is the highest frequency of interest minus the lowest, fH - fL. If you are looking for a single tone, fH - fL = 0, there is no bandwidth. When you have less bandwidth you can lower your sample rate, when you lower your sample rate you save computations and so on.

1/2 of your FFT bins are empty, right? But you're computing them still, so those computations are basically wasted.

I already did some groundwork on the FIR filters I had in mind, I think they meet or exceed the performance of a 32 point FFT per operation. I am not really into computer science but general purpose CPU/DSP aren't particularly great at FFT, right? Broken pipelines wasting clock cycles and such? While a FIR filter lends itself to pipelining very well? So what I'm thinking is for a given number of operations, say 100, an FFT will put a greater load on the CPU than a FIR. Or am I way off? Perhaps the increase in efficiency will mean more when the target is a hardware processor rather than a computer CPU.

I could be wrong about all this, its just my initial impression that an FFT might not be the best way to do this, or certainly not the only. Either way its something interesting for me to think about.

So to test my filters I can generate 8 samples at 16 bit and 44.1 kHz. Each sample will have a random amplitude of a sinusoid at [5 6 7 8 9 10 11 12]/32*44100 Hz. Anything else to note?

Also there are ways to recover that bandwidth between DC and 6 kHz.

I am much too interested in your project over doing my own stuff.

[urbanscallywag]

This link has what I was getting into about the FFT inefficiency:

http://cnx.org/content/m12021/latest/

[Randy Jones]

The test signal you want to look at is a sum of eight sine waves of the frequencies you mention. The amplitudes of the sines can vary smoothly, but if you make them random, you have to lowpass filter the randomness. If one carrier varies too fast, its frequency will spread into adjacent bands. So I would probably use another sine as a test signal to start.

This would indeed be a great application for FPGA.

[urbanscallywag]

OK got it, not random amplitudes per sample. I actually just left each tone at maximum amplitude for my test.

I believe I have a neat solution. I will verify it and write something up tonight or tomorrow.