During the initial stage of this project, multiple visualization techniques were studied to determine a suitable development process. Based on the research, the following process was undertaken to convert the audio data into a digital visualization experience.
Data bending is the process of manipulating a media file of a certain format, using software meant to edit files of another format (Word Finder, n.d.). Thus, through the Audacity 2.3.2 software, the MP3 files were converted into the RAW formats that would generate the deepest available description (signed 32-bit) and the widest dynamic range (A-law encoded).
The Glitch Effect is created when there is an error or a flaw in an image file (Leigh, 2019). In this research however, the Glitch Effect is caused intentionally through the data bending process. Then an image is created from the RAW file when photoshop depicts the binary code as pixels and reads the colors into the image – 0 being Black and 225 representing white (File Formats, n.d).
Animation & Immersion
Once the images were generated from the RAW File, they were brought over to Blender to create a data visualization experience:
3D Textured Room
The Glitch images are transformed into textures to create a 3D environment. Note that, prior to the pandemic, the textures would have been showcased using projection mapping and artificial reality to allow for immersion.
Animations were created by compiling the glitch images to create a video. This video was then used to inform the displacement of objects in Blender. In this exercise, the Glitch image texture’s RGB values were used to displace the mesh of the object in the XYZ direction.
In this animation, the glitch image video was used to inform the direction of the wave. while the soundwaves were used to inform the strength of the waves.