Sounds Like Kandinsky: synaesthesia in the emulator from Google on the basis of works by Kandinsky

Wassily Kandinsky is an artist who could “see sounds”. Together with the Pompidou Center, Google Arts & Culture specialists have developed a virtual exhibition Sound Like Kandinsky, which, using machine learning algorithms, tries to emulate the effect of synesthesia – and allows viewers to also “see sounds.”

The work of Wassily Kandinsky is one of the most famous manifestations of synesthesia. “Colors and shapes were transformed into sounds, harmonies and vibrations, gathering into patterns,” explained Serge Lasvignes, president of the Pompidou Center. For example, red sounded like a violin and yellow sounded like a trumpet.

Experimental musicians Antoine Bertin and NSDOS were invited for the virtual exhibition Google Arts & Culture – they were the ones who tried to dub Kandinsky’s paintings. They analyzed the artist’s notes about how he perceived the world, what effects of synesthesia he felt, and transformed the painting “Yellow, Red, Blue” into a melody.

The neural network analyzed the music of the time when Kandinsky was working. And this system, based on the analysis, has created new soundtracks for the pictures. In a virtual exhibition, you can bring different elements of the picture closer – and each of them will sound the way, quite possibly, the artist perceived it. You can assemble your own mix from various pieces of the picture and share it on social networks.