http://xyinteraction.free.fr/photos/LilyPad.jpg

Software R&D

Here we propose a set of programs usable with the XY interface. Those pieces of software are to be considered as a modular base to be adapted to a particular application. The code used for this library is mainly Pure Data, a multi-platform open-source visual programming. This library is inspired by the ENIGMES-Lib originally designed for Max Msp by Roland Cahen for the ENIGMES project. "ENIGMES-Lib" is a set of programs made for playing sound through interactive graphical metaphors.

• Data acquisition (Pure Data) : Read XY textile datas.
• Data acquisition (Processing) Read XY textile datas.
• Curves following : realize the mapping between a graphic curve and sound datas. Your finger is acting as a tape reader. Vincent Rioux
• Zone detection : describe zones, with complex shapes and possibly overlapping, and associates them with sound datas.
• Angular detection : determine distance and angular value of one vector made by the finger and a reference point.
• Munger player : a real-time multichannel granulator, a.k.a. the swiss-army-knife of real-time granular synthesis. By Dan Trueman.
• Gesture recognition : learn curves and then try to match with current input. This program uses Python and NumPy (Numerical Python) within Pure Data. Coded by Vincent Rioux.

\\\

http://xyinteraction.free.fr/photos/icon_xyi.jpg %width=183px%http://xyinteraction.free.fr/photos/plan_textile.jpg %width=183px%http://xyinteraction.free.fr/photos/weaving_machin.jpg

The XY interactive textile is a large tactile interface for playing electronic music. The performer plays it simply by the movement of his/her hand on it's surface. This textile interface allows users to compose and interpret electronic music by choreographic movements. By its size, its texture, its flexibility and its transparency, this textile interface involves the whole body in the musical interpretation. Like a music score, the sound elements are graphically represented on the fabric, giving to the composer the ability to locate and play (with) them. Thus, for each piece of music, a graphic partition is carried out. The idea behind this project is also to develop an opposite approach to a classical process (from graphic towards the sound). Created in collaboration with graphic designers and composers, these hybrid textiles offer spaces to be explored by look, touch and gesture. Today, this multi- functional project inspires new usages in numerous domains. For example, we're currently working with a choreographer to integrate dance with the musical fabric. We have already created an installation in which allowed two people to communicate by drawings on the fabric over an internet connection. We also plan to use this programmable surface for gesture recognition. The idea is to associate symbolic gestures with a particular function: for example, to make an ambient light glow, you need to draw a circle on the sensitive fabric placed on a room wall.

The XY interactve textile is currently used by Marco Marini a French Electro-Acoustic composer. Within the research project "ENIGMES" on the topic of navigable score, Marco Marini was taken part in creation of a dedicated XY textile score. Currently finished, through performances Marco Marini uses all the gestual and acoustic actions patterns required to interprete this interactive score: musical notations, sound lines, trajectory... The movement of his hands on the textile surface triggers musical lietmotives. Inspired by echography (graphic sounds patterns), each graphic pattern uses simultaneously technical and symbolic codes. These patterns allow the musician to localise the sounds, their shape and their duration. The role of the performer is transformed, he is musican and dancer which makes for a unique mis-en-scene of the musical score. Facing the public the performer (both musician and dancer) shows how the movement of his hands bring to life the music as he traces the patterns on the surface of the translucent fabric. His fingers in contact with the fabric playlike the record head on a record : distorting the duration of the sound or simply triggering sound events. For the time being the sound environment that we are exploring are highly experimental but new, more "pop" partitions are in perspective.

Stakes

• Leave the traditional interaction screen, keyboard, mouse, too generic and getting old.
• Give a physical dimension to the numerical interface.
• Rehabilitate the gestures into the interaction.
• Make evolve range of the interactive devices.
• Bring a sensible dimension to the numerical world.
• Propose an interaction who give a direct contact to the hand (haptic nteraction).
• Create new tactile objects.
• Produce accessible objects.

Material originality

• Simplicity
• Flexibility
• Transparency
• Lightness
• Texture touch
• Large surface interaction
• Low cost per m2
• High input resolution
• Modeling around the subject

-----

On this smart fabrics, the hand's contact position is transmitted to a computer via a 10 bytes resolution Lilypad card. This allows an input resolution of 1024 by 1024. Two conductive fabrics are attached to a frame, each one weaved with conductive threads in different directions. When the performer presses any point of this textile instrument, the two fabrics are connected and the current electrical value is sent to the computer. The technology of the textile is based simply on a voltage divider resistive principle! I developed this project at the National School of Industrial Creation (ENSCI) Paris, France, as a part of my senior thesis project in December 2005. Since then I have become specialized in the creation of innovative electronic textiles and elaborate the design of this tactile fabric.

http://xyinteraction.free.fr/photos/principe.gif \\\

Technology.

On this smart fabrics, the hand's contact position is transmitted to a computer via a 10 bytes resolution Lilypad card. This allows an input resolution of 1024 by 1024. Two conductive fabrics are attached to a frame, each one weaved with conductive threads in different directions. When the performer presses any point of this textile instrument, the two fabrics are connected and the current electrical value is sent to the computer. The technology of the textile is based simply on a voltage divider resistive principle! I developed this project at the National School of Industrial Creation (ENSCI) Paris, France, as a part of my senior thesis project in December 2005. Since then I have become specialized in the creation of innovative electronic textiles and elaborate the design of this tactile fabric.
http://xyinteraction.free.fr/photos/principe.gif

XY firmware.

http://upload.wikimedia.org/wikipedia/commons/thumb/b/b5/Potentiometer.jpg/512px-Potentiometer.jpg

To obtain the XY contact value we do a two time sensing. Eatch sensing give us a resistor value who is proportional to X or Y finger position. This principle can be illustrated by two potentiometers which middles points pins are connected together and the others pins connected to the LilyPad board. Below, the two-time program to obtain significant values of XY finger position. http://xyinteraction.free.fr/photos/arduino_xyi.gif

1- Calcul de la valeur des résistances (ou potentiomètres) X1-X2 et Y1-Y2 :

Pour que le circuit consomme peu et fonctionne bien. (Plus nous diminuerons la valeur de la résistance plus les valeurs captées seront stables mais plus le circuit consommera de courant).

A faire :

Schéma de connections des toiles à la carte Lilypad

http://xyinteraction.free.fr/photos/LilyPad.jpg

Arduino sketch with optimized communication

// SERIAL COMMUNICATION PROTOCOL for X and Y values (10 bits each)
// To optimize the communication for each value, we only need 2 bytes.
// On each of the byte we use the 2 leading bits (on the left) that act as IDs.
// Then, decoding this protocol will first analyse these 2 bits and reassemble the values accordingly.
//
// Each 10bits value { xxxxxxxxxx }
//                     ||||||||||
//                     9876543210  ...correspondng bit position
//                     /       \
//              { ab009876 , ab543210 } ...is broken into 2 bytes
//                    ||||     ||||||
// MSB(Most Significant Bits)  LSB(Least Significant Bits)
//
// abxxxxxxx   format explanation for each byte transmitted :
// ||
// ||---------> a=0 => MSB, a=1 => LSB
// |----------> b=0 => this is X, b=1 => this is Y
//
// { 00000xxxx , 01xxxxxx } ...will be transmitted for X
// { 10000xxxx , 11xxxxxx } ...will be transmitted for Y

int Y1=10;                             // select the analog input pin 10
int Y2=9;                              // select the analog input pin 9
int X1=11;                             // select the analog input pin 11
int X2=12;                             // select the analog input pin 12
int Val;                               // Val is use alternatively for X and Y Valus
int Xmsb, Xlsb, Ymsb, Ylsb;

void setup() {
        Serial.begin(19200);
}

void loop() {

        // FIRST TIME : GET X VALUE
        pinMode(X1, INPUT);                  // put pin in high impedance (+infinite)
        pinMode(X2, INPUT);                  // put pin in high impedance (+infinite)
        pinMode(Y1, OUTPUT);                 // declare the Pin as an OUTPUT
        pinMode(Y2, OUTPUT);                 // declare the Pin as an OUTPUT
        digitalWrite(Y1, LOW);               // turn off the Pin 10
        digitalWrite(Y2, HIGH);              // turn on the Pin 9
        delay(3);
        Val=analogRead(1);                   // Read Analog Input 1

        // octet 1 for X ; MSB for X (More Signifiant Bits)
        // 00000xxx  Value (after being shifted 5 times)
        // ||
        // ||--------> MSB
        // |---------> X
        Xmsb = Val >> 5 ;                    // Division by 32, save the 5 Msb bits
        Serial.print(Xmsb, BYTE);            // Send X MSB

        // octet 0 for X ; LSB for X (Less Signifiant Bits)
        // xxxxxxxx       value
        // 00011111     & mask (31)
        // 01000000     + flag
        // 010xxxxx     = result for
        // 01000000     + 64     for LSB flag
        // first bit is 0 for X flag
        Xlsb = Val & 31 ;                    // save the 5 lsb bits
        Xlsb = Xlsb + 64 ;                   // set second bit to 1 for lsb marker
        Serial.print(Xlsb, BYTE);            // Send X LSB

        // SECOND TIME : GET Y VALUE
        pinMode(X1, OUTPUT);                 // declare the Pin as an OUTPUT
        pinMode(X2, OUTPUT);                 // declare the Pin as an OUTPUT
        pinMode(Y1, INPUT);                  // put pin in high impedance (+infinite)
        pinMode(Y2, INPUT);                  // put pin in high impedance (+infinite;)
        digitalWrite(X1, HIGH);              // turn on the Pin 11
        digitalWrite(X2, LOW);               // turn off the Pin 12
        delay(3);
        Val=analogRead(0);                   // Read Analog Input 0

        // octet 1 for Y ; MSB for Y (More Significant Bits)
        // 10000xxx  Value (after being shifted 5 times)
        // ||
        // ||--------> MSB
        // |---------> 1=>Y
        Ymsb = Val >> 5 ;                    // Division by 32, save the 5 Msb bits
        Ymsb = Ymsb + 128 ;                  // add a bit for Y
        Serial.print(Ymsb, BYTE);            // Send Y MSB

        // octet 0 for Y ; LSB for Y (Less Significant Bits)
        // xxxxxxxx       value
        // 00011111     & mask
        // 000xxxxx     = result
        // 01000000     + 64     for LSB flag
        // 10000000     + 128    for Y flag
        Ylsb = Val & 31 ;                    // save the 5 lsb bits
        Ylsb = Ylsb + 64 ;                   // set second bit to 1 for lsb marker
        Ylsb = Ylsb + 128 ;                  // add a bit for Y
        Serial.print(Ylsb, BYTE);            // Send Y LSB
}

Arduino sketch : fonction digitalMedian[5];

http://xyinteraction.free.fr/photos/median_5_Image.png
Pour supprimer les abhérations des données captées, nous utiliserons un filtre médian. Ce filtre permet de classer les valeurs du flux dans un ordre croisant et de sélectionner la valeur médiane. Sur la base du l'objet médian5 de Pure Data, Nous implémenterons ce filtre pour la carte Arduino

//fonction digitalMedian[5];

#define filterSamples 5
#define SensorPin1  0
#define SensorPin2  1

int SmoothArrayX [filterSamples];        // array for holding raw sensor values for sensor1
int SmoothArrayY [filterSamples];        // array for holding raw sensor values for sensor2

int rawDataX, smoothDataX;                // variables for sensor1 data
int rawDataY, smoothDataY;                // variables for sensor1 data

void setup() {
        Serial.begin(19200);                    // initialize serial communication with computer
}

void loop() {
        rawDataX = analogRead(SensorPin1);
        smoothDataX = digitalMedian(rawDataX, SmoothArrayX);
        rawDataY = analogRead(SensorPin2);
        smoothDataY = digitalMedian(rawDataY, SmoothArrayY);

        Serial.print(smoothDataX);                          // send it to the computer (as ASCII digits)
        Serial.print("   ");
        Serial.println(smoothDataY);
}

int digitalMedian(int rawIn, int *SmoothArray){     // "int *sensSmoothArray" passes an array to the function - the asterisk indicates the array name is a pointer
        static int index;
        int a, b, c, d, e, f, g, h, i, j, k;
        int median;

        if (index = filterSamples)                        // if we're at the end of the array...
        index = 0;                                               // ...wrap around to the beginnin
        SmoothArray[index] = rawIn;                  // read sensor value
        a = max(SmoothArray[3], SmoothArray[4]);
        b = min(SmoothArray[3], SmoothArray[4]);
        c = max(SmoothArray[2], b);
        d = max(a, c);
        e = min(a, c);
        f = min(b, SmoothArray[2]);
        g = max(f, SmoothArray[1]);
        h = max(e, g);
        i = min(e, g);
        j = min(d, h);
        k = max(i, SmoothArray[0]);
        median = min(j, k);
        index++;                                                   // advance to the next index
        return median;
}

Pure Data acquisition (X,Y) --> Librairie de programmes

• Adjustable Buffer a potentiometer can be added to the circuit to adjust the sensitivity range.