Journey Into Sound Design by Charffy Wang
Before joining Moment Discovery, I was a computing science student who was interested in the production of video games and films. I love to immerse myself in a fictitious world, where the infinite potential and incredible expressiveness deeply fascinate me. My love for immersive experiences drove me to take a music course in university, because in my opinion, music is an indispensable factor in creating atmosphere. I used what I learned about music to create sound effects for my video games, as well as dubbing and recording for a spoof video I made about Transformer.
Music editing usually takes me half a day and can be very tedious. Although I feel very exhausted, unexpectedly pleasing results always wash away my fatigue. This reminds me of learning the flute as a child. From successfully blowing the first sound to being a level 10 professional, I have experienced frustration and anger; but it also brought me a lot of joy and a sense of accomplishment. I might have imagined that I could step into the field of music as a flute player one day, but as a digital music producer, never; it sounds like a career that is far away from me. However, maybe it is because I have always been attracted to the splendor of the virtual world. I was meant to be a part of it, whether in software programming, visual art design or digital music production.
When I joined Moment Discovery as a programmer, I thought my job would simply be coding and debugging. But surprisingly, Moment Discovery gives me lots of opportunities to collaborate with artists. Throughout these technologically-enabled creative projects, I have broadened my horizons in both the musical and visual spheres. In my collaboration with youth artist Dylan Rominik, I built a sound library for all the pixelated paintings we created in Draw. As a result, when a painting is displayed on DotWall, you experience not only the visually dynamic media art, but also the background music corresponding to the theme. While embedding sounds into our interactive media contributes to immersive experience, we are constantly exploring other methods to add more flavour to our audio system.
When I was told that my future work would involve acoustic research with Will, a feeling of excitement suddenly welled up inside me. My main task is to use Ableton to receive MIDI commands from Spatio and then perform corresponding digital sound processing. At that time, I was a complete novice user of Ableton, so the sudden mission left me at a loss. Fortunately, my familiarity with Logic Pro enabled me to quickly understand how to manage things in Ableton, and soon I got the chance to put what I had learned into practice. In Emelia Symington-Fedy’s Book Reading, I was assigned the mission to do live spatial audio control. In Ableton, I received Emelia’s voice from the microphone. After editing reverb and frequency equalization, I routed the result to our speakers that surround the space. In this way, we intended to reproduce the scenes described in the book, to achieve emotional resonance by leading the audience through different circumstances, allowing them to experience the quiet night and the noisy party.
In addition to recording and editing, I also needed to learn how to program in MAX, an interactive graphic environment for music, audio processing and multimedia. After reading massive amounts of documentation and tutorials, I gradually began to appreciate the vastness of MAX, and was impressed by its unlimited capacities. Through constant learning and practicing, I started to get the hang of utilizing MAX to achieve the various effects we wanted, which mostly were put into practice in our recent experiments with Jesse Thomas, Associate Professor of Fine Arts at University of Alberta.
During the experiments, we utilized all of our software and hardware to build an interactive environment for Jesse’s paintings. I worked in Spatio and Ableton simultaneously, to create 3D maps and soundscapes for each painting based on the visual elements it contains, the time of day it depicts and the underlying emotion. In addition to triggering different sound effects when a player enters different zones, I also created MAX for Live devices in Ableton to control the state, timing and movement of the sound. One of the devices collaborates with AutoPanner, controlling the sound source’s movement in X and Y directions. I calculated different trajectory shapes and assigned the output to our 8 speakers anchored on the trusses that surround the tracking area. Once a specific sound is triggered, it automatically starts moving on its designated path, while the player hears the sound bouncing back and forth around him. This feature also integrates with Spatio’s body tracking system taking in the player’s X and Y coordinates as MIDI CC values, which are then fed to the Panner. In this way, we achieved a sound effect that follows the player.
In the process of sound production, I really enjoy discussing how to achieve different musical effects with Will. He always has brilliant ideas and gives me inspiration when my mind comes to a dead end. When we were editing his guitar recordings, we experimented with a lot of effects to match Jesse’s paintings making them sound broken and erratic in order to create a strange, outer-space atmosphere. Before entering the realm of digital sound synthesis, I believed that there must be obscure principles and calculations behind turning every dial to control a parameter; so I felt very nervous and helpless for a while. However, once I got my hands dirty, fear gradually gave way to familiarity. I became more confident and started thinking about how to make it work instead of worrying if it would work. I truly appreciate this sound production journey at Moment Discovery, it adds so much creativity and colour to my work and research!
在加入 Moment Discovery 之前，我是一名对电子游戏和电影制作很感兴趣的计算机系学生。我喜欢让自己沉浸在一个虚构的世界里， 它无限的潜能和不可思议的表现力深深吸引着我。对沉浸式体验的喜爱驱使着我在大学期间进修了一门音乐课程，因为音乐于我而言是氛围营造不可缺失的一个因素。我于是利用我所学到的音乐知识为我的电子游戏制作声效，以及为我制作的关于《变形金刚》的恶搞视频配音和录音。
音乐制作非常乏味且通常会花费我半天的时间。虽然我很疲惫，但出乎意料惊喜的结果通常会洗刷我所有的疲劳。这让我想起了小时候学竹笛的情景。从成功吹响第一声到成为一个十级专业演奏者，我经历过沮丧和愤怒，但它同时也给我带来了很多快乐和成就感。我也许想象过有一天可以作为一名笛子演奏家踏入音乐领域，但作为一名电子音乐制作人，我却从没想过，因为这听起来像是一个离我很遥远的职业。不过，也许是因为我总是被虚拟世界的绚烂所吸引，我注定会成为他其中的一员，不管是在软件编程，视觉艺术设计还是电子音乐制作。
当我作为程序员加入 Moment Discovery 时，我以为我的工作会只局限于编程和调试。但令人惊讶的是，Moment Discovery 给了我很多与艺术家合作的机会。这些创意科技项目大大拓宽了我在听觉和视觉上的视野。在与青年艺术家 Dylan Rominik 的合作中，我为我们在 Draw 中创建的所有像素画建立了一个声音库。因此，当一幅画在 Dotwall 上展示时，你体验到的不仅是视觉上的动态艺术，还有与主题相对应的背景音乐。虽然将声音嵌入到我们的交互媒体有助于带来身临其境的体验，但我们仍在不断探索其他方法来为我们的音频系统添加更多色彩。
当我得知我未来的工作将与威尔一起进行声学研究时，我内心突然涌起一股兴奋的感觉。我的主要任务是使用 Ableton 接收来自 Spatio 的 MIDI 命令，然后进行相应的数字声音处理。当时我还是一个十足的 Ableton 新手，突如其来的任务让我不知所措。幸运的是，我对 Logic Pro 的熟悉使我能够快速了解如何在 Ableton 中处理事物，很快我就有机会将所学到的知识付诸实践。在 Emelia Symington-Fedy 的读书会中，我接手了现场空间音频的控制任务。利用 Ableton，我从麦克风中收取 Emelia 的声音，进行编辑混响和频率均衡，然后将结果发送到环绕整个空间的扬声器中。这样我们就可以重现书中所描述的场景，带领观众穿越不同的情境来达到情感上的共鸣，让他们体验静谧的夜晚和喧闹的聚会。
除了录音和编辑之外，我还需要学习如何在 MAX 中编程，MAX 是一个用于音乐、音频处理和多媒体的交互式环境。在阅读了大量的文档和教程后，我逐渐开始体会到 MAX 的浩瀚，被它无限的能力所折服。通过不断的学习和实践，我开始掌握如何用 MAX 来实现我们想要的各种效果，这些效果大部分是在我们最近与阿尔伯塔大学美术副教授 Jesse Thomas 进行的实验中付诸实践的。
在实验过程中，我们利用了所有的软件和硬件为 Jesse 的画作构建了一个交互环境。我在 Spatio 和 Ableton 同时工作，根据每幅画包含的视觉元素、描绘的时间和潜在的情感为每幅画创建了 3D 地图和音景。除了当玩家进入不同区域时所触发的不同音效之外，我还在 Ableton 中创建了 MAX for Live 设备来控制声音的状态、时机和移动。其中一款设备与 AutoPanner 协作，控制声源在 X 和 Y 方向上的移动。我计算了不同的轨迹形状，并将输出分配给固定在追走区域周围桁架上的八个扬声器。一旦特定的声音被触发，它就会自动开始沿着指定的路径移动，玩家同时也会听到声音在他周围来回弹跳。此功能还与 Spatio 的身体追踪系统协作，将玩家的 X 和 Y 坐标作为 MIDI CC 值，然后将其发送到声像器，于是我们就实现了跟随玩家的声音效果。
在声音制作的过程中，我非常喜欢和 Will 讨论如何实现不同的音乐效果。他总是有绝妙的想法，且每当我的思维走入死胡同时，他总是能给我灵感。当我们编辑他的吉他录音时，我们尝试了很多效果来配合 Jesse 的画作，使它们听起来破碎且飘忽不定，以营造出一种奇怪的外太空氛围。在进入数字声音合成领域之前，我相信在每一个转动旋钮来控制参数的背后一定有者晦涩的原理和计算，所以我有一段时间感到非常紧张和无助。然而，一旦我亲手去做，恐惧就逐渐被熟悉所取代。我变得更加自信，开始思考如何让它发挥作用，而不是担心它是否有效果。我真的很感激在 Moment Discovery 的这段声音制作之旅，它为我的工作和研究增添了如此多的创造力和色彩！