## Synesthesic ## Musical Instruments Adding a new dimension on how the environment is perceived by us

## Goals * Add a new dimension on how environmental data is perceived by us: **hearing** * Bring environmental awareness to a new sense * Build new tools for data analysis and artistic expression

How? #### Converting data to sound! 

Data related to temperature, luminosity, wind can be used as input on electronic musical instruments for: * #### Creating sound samples * #### Controlling effects * #### Controlling MIDI devices

## Why synesthesic? [Synesthesia](https://en.wikipedia.org/wiki/Synesthesia) is a phenomenon in which stimulation of one sensory or cognitive pathway leads to experiences in a second sensory or cognitive pathway.

With synesthesic musical instruments, sounds related to environmental observables that usually can be felt, seen or smelled will be heard.

Let's call these the ### _synesthesic musical instruments_

## Background * [International Community for Auditory Display (ICAD)](http://www.icad.org/index.html) * [The Sonification Handbook](http://sonification.de/handbook/), Edited by Thomas Hermann, Andy Hunt, John G. Neuhoff Special thanks for Jan Luc Tavares for finding [these valuable resources](http://lief.if.ufrgs.br/~janluc/tdef/sinestesia/2017/08/09/Apresentacao-TDEF1.html).

## Prototype Development Rafael Pezzi @ Hybrid Forms Lab / Vrije Universiteit Amsterdam August 14th to 25th, 2017

## SMI Design 

## First prototype Temperature data for envelope and fine structure  Source: https://research.meei.harvard.edu/Chimera/motivation.html

## First prototype in action 24 h Temperature data collected at home (Viamão/Brazil)  

## Fine structure and Envelope  

Final waveform

## First prototype ### Current status * Loads time series from file and select audification range * Tunning: possibility to select resulting frequency * Linear harmonizer * Create basic waveform and envelope from temperature data * Output waveform as .wav file

## Project documentation * [Git Repository](https://git.cta.if.ufrgs.br/emm/SMI) * [Audiolab Jupyter Notebook Prototype (html renderization)](/~pezzi/synesthesic_musical_instruments/include/audiolab_prototype_August_2017.html)

## What else happened? Collaboration with [Mark Buitenwerf](http://www.markbuitenwerf.com/) (left), who is making the instrument playable with [pure data](http://puredata.info/). 

### Special Thanks * [Centro de Tecnologia Acadêmica IF/UFRGS](http://cta.if.ufrgs.br) * [Synergetica Lab](http://synergeticalab.com) * Hybrid Forms Lab / Vrije Universiteit Amsterdam

### Possibilities for further development * Prototypes for different physical quantities for fine structure and envelope formation * Develop new methods for harmonization and range selection * Search of audible identification of known phenomena * Perform musical composition and soundscape construction * Find open-data sources

### Georeferenced data driving ### synesthesic musical instruments will let us listen to ### Synesthesic Soundscapes

### Soundscape? [Soundscape](https://en.wikipedia.org/wiki/Soundscape) is the component of the acoustic environment that can be perceived by humans. Sounds produced by physical features of the environment such as running water contribute to natural soundscapes

## Want to join us? * Get in touch: pezzi _at_ if.ufrgs.br