PhD Thesis

Ph.D. ATIAM (Acoustics, Signal Processing and Computer Science applied to Music), Aix-Marseille II University, CNRS-LMA (Laboratoire de Mécanique et d’Acoustique de Marseille), Computer Music team, France.

Title:

"Advanced use of graphical interfaces for the gestural control of sound processes"

• Study and experiments of the use of graphical interfaces in digital musical instruments.
• Use of Computer-Human Interaction techniques into the computer music area.
• Design of specific graphical interfaces and sound algorithms controlled with different gestural controllers.
• Development of a touchscreen-like device with multiple pointing.
• Design of computer programs available on Internet.
• Design of digital musical instruments, used in live performances.

Awarded December 7th, 2005, Marseille.

Download thesis (pdf, French)

Abstract:

First part of thesis deals with various aspects of gesture control of sound processes in digital musical instruments, especially the link between gesture and sound, called mapping.

Second part is about graphical interfaces. A work has been carried out to apply the Instrumental Interaction model to the control of sound processes. After having shown how graphical interfaces appear in the mapping, a typology that classifies the graphical elements is proposed. Practical aspects for the creation of graphical interfaces are then introduced.

Last part introduces the musical instruments created during the thesis; those instruments use graphical interfaces, sometimes with several pointers. The instruments take advantages of the graphical interfaces; some of them have been used to perform live musical pieces. The instruments that have been created strengthen the interest of using suited graphical interfaces in digital musical instruments.

Appendix video files

The videos files introduce the devices developped during the PhD Thesis. They refer to particular chapters. List of videos (format mpeg1):

• Voicer.mpg (chap. 2.5.2): Extract of "D'ici et d'Ailleurs", Loïc Kessous, with the Voicer (WebBar concert, Marseille, april 2002)
• Photoso.mpg (chap. 2.5.3): Extract of "Route 729", Daniel Arfib, with the Photosonic Emulator (WebBar concert, Marseille, april 2002)
• Scanned.mpg (chap. 2.5.4): Extract of "Vibrations Feuilletées", Jean-Michel Couturier, with the Scanned Synthesis instrument (WebBar concert, Marseille, april 2002)
• ScanCintiq.mpg (chap. 4.2.1): Scanned Synthesis controlled with the Cintiq interactive pen display
• ScanPF.mpg (chap. 4.2.1): Scanned Synthesis controlled with the Pointing Fingers
• ReveFun.mpg (chap. 4.2.2): "Le Rêve du Funambule", with the Filtering String (Café Julien concert, Marseille, october 2003)
• PhotosoPF.mpg (chap. 4.2.3): Photosonic Emulator with graphical interface and Pointing Fingers
• VoicerTac.mpg (chap. 4.2.4): Voicer with multipoint interpolation
• EtirVib.mpg (chap. 4.2.5): Time-stretch and vibrato control interface
• WavTerrain.mpg (chap. 4.2.6): Control of Wave-Terrain Synthesis
• Etheraction.mpg (chap. 4.2.7): Demonstration of the devices used in the piece "Etheraction" and short live extract (Ziem museum, Martigues, France, april 2004)

Table of contents:

CHAPTER 1 INTRODUCTION

CHAPTER 2 GESTURAL CONTROL OF SOUND PROCESSES
2.1 From synthesis beginnings to nowadays
2.1.1 From acoustic musical instruments to analog and digital musical instruments
2.1.2 A new area to explore
2.2 Generation and transformation of son : the sound processes
2.2.1 Sound synthesis
2.2.2 Digital audio effects
2.3 Action and perception
2.3.1 The instrumental gesture
2.3.2 Instrumental gesture and gestural controllers
2.3.3 Feedbacks
2.4 Mapping
2.4.1 Mapping attributes
2.4.2 Perceptual spaces in the mapping
2.4.3 Related-to-sound-perception parameters
2.4.4 Related-to-gesture-perception parameters
2.4.5 Mapping between perceptual spaces
2.5 Instrument examples designed at LMA
2.5.1 Designing instruments with Max/MSP
2.5.2 The Voicer
2.5.3 The Photosonic Emulator
2.5.4 The Scanned Synthesis instrument

CHAPTER 3 GRAPHICAL INTERFACES FOR THE CONTROL OF SOUND PROCESSES
3.1 Human-Computer Interaction and graphical interfaces
3.1.1 Evolution of Human-Computer Interaction
3.1.2 Historical overview of interaction styles
3.1.3 What is a graphical interface?
3.1.4 Instrumental interaction
3.2 Graphical interfaces and instrumental control
3.2.1 Instrumental control specificities
3.2.2 How to use graphical interfaces for instrumental control?
3.3 Instrumental Interaction applied to digital musical instruments
3.3.1 Domain objects
3.3.2 Interaction instruments: controllers and tools
3.3.3 Relationships between physical and logical parts
3.3.4 Réification of actions on sound
3.3.5 Properties of controllers and tools
3.4 Typology of graphical elements
3.4.1 Typology extended to music creation
3.4.2 Typology specific to instrumental control
3.5 Graphical interfaces in the mapping
3.5.1 Introducing graphical interfaces in the mapping
3.5.2 Controllers, tools and mapping
3.5.3 Modularity in the mapping
3.6 Designing graphical interfaces
3.6.1 Key-factors in graphical interface design
3.6.2 Some specific design rules
3.6.3 graphical interface design in Max/MSP-Jitter
3.6.4 Multi-pointers graphical interfaces

CHAPTER 4 DESIGNING DIGITAL MUSICAL INSTRUMENTS WITH GRAPHICAL INTERFACES
4.1 Gestural controllers
4.1.1 Existing gestural controllers
4.1.2 Design of the Pointing Fingers interface
4.1.3 Comparisons between the gestural controllers
4.2 Designed instruments
4.2.1 Scanned Synthesis
4.2.2 Filtering String
4.2.3 Photosonic Emulator with graphical interface
4.2.4 Voicer with multipoint interpolation
4.2.5 Control interface of time stretch and vibrato
4.2.6 Control of wave terrain synthesis
4.2.7 Etheraction: Interpretation of an electroacoustic piece

CHAPTER 5 CONCLUSION

GLOSSARY

APPENDIX

BIBLIOGRAPHY