--------------- EDITOR'S NOTE ---------------

HTML Newsletter and E-Mail List

I wanted to start this week by highlighting that the Consortium is in the process of switching the C3 Weekly Update from this text e-mail into an HTML format. As part of this process, we are creating mailing lists for the newsletter. During the switchover, each subscriber will receive an e-mail verifying your subscription to the Weekly Update. You just need to click through and verify your interest in receiving the newsletter. If you have any questions during this transition process, please don't hesitate to contact me directly.

MIT Futures of Entertainment

More information on the second annual MIT Futures of Entertainment conference should be coming out this week. The conference will be held Nov. 16-17. We are hoping that a variety of the consulting researchers, C3 alum, and members of our partner companies are able to come to the conference. As mentioned last week, planned panels will focus on advertising, mobile media, metrics, fan labor, and cult media properties. We hope to have significant details on the lineup of speakers in next week's C3 Weekly Update. More information is available here.

This Week's Newsletter

This week, our Opening Note features the conclusion of C3 Consulting Researcher Stefan Werning's two-part series making a comparison between digital music in games and silent film stores. The Closing Note features the work of Doris Rusch, who has accepted a postdoctoral position here at the Singapore-MIT GAMBIT Game Lab in the Program in Comparative Media Studies here at MIT. Doris provides an in-depth game review of Lord of the Rings online and how this piece relates to the Tolkien universe as a whole.

As usual, the newsletter this week features all the entries published during the week on the Convergence Culture Consortium Weblog.

If you have any questions or comments or would like to request prior issues of the update, direct them to Sam Ford, Editor of the Weekly Update, at samford@mit.edu.

--------------- OPENING NOTE ---------------

From Silent Film Scoring to Digital Game Music:
Discourse Continuities and Congruences
Part II of II

By: Stefan Werning

In last week's Closing Note, Werning wrote about the history of non-linear music to the 17th and 18th century and began the comparison between silent film scores and video game music by examining talk about music for silent film in the early 20th Century. This piece looks at a recent case study, draws to some conclusions, and points to further resources.

As a second, more recent case study, the Wwise industry-level "audio pipeline" demonstrates the epistemological implications of increasingly converging production tools which tangibly informs the conceptualization of interactive, non-linear music in similar ways like silent film scoring. (cf. develop 72, 2007: 44) Wwise usage is described as essentially a workflow optimization process, i.e. an attempt to avoid the constant input of a programmer in the implementation of game music and making best use of the usually limited amount of music (in the given example 45 minutes) set down in the composer's contract.

According to the Wwise workflow overview, the audio engine requires the composer/audio designer to specify "how many game states there are and what their names are," i.e. the underlying concept of music appears quite congruent with silent film accompaniment compilations organized by type of scene and mood or [Affekt].

A "transition matrix" is used to shift between music segments both horizontally and vertically, i.e. essentially a layered rule system in which very basal transition rules may be formulated first and later complemented with more differentiated lower-level rule sets. Thus, a production semiotics look illustrates how the repeated use of particular tools can influence the producer's collective imaginary and formal musical repertoire.

Most notably, transitions are becoming the key segments of musical composition, not unlike minimal music which also featured algorithmic composition principles and, later, utilized and playfully applied programmable sound technologies. Second, the idea of layering and prioritizing transitional rules, both indebted to the general properties of program code and to silent film scoring techniques, influences this thinking about musical transitions. As seen in the case of repurposing classical music, this even shapes the re-reading of music composed with linear performance in mind.

A related production report from the same source is the audio design for the upcoming Halo3; (cf. develop 72, 2007: 18) the ambitions of the project are already indicated by the unusual 'Audio Mix Complete' milestone dates used in the game development process.

A revealing insights from the article is the fact that the actual composition process occurs only late in the audio development; first a "unique script" for each whole level is prepared to determine where and how to play, mute or change the music. This approach not only requires the composer to adopt programming logic but also assumes a pseudo-nonlinear 'script' as role model bearing on the compositional imaginary. Moreover, developer Bungie uses a proprietary adaptive/interactive music system for 'fillers'.

As an interesting sidenote, the habitual overuse and gratuitous deployment of background music is termed "wallpapering games with music;" this rhetoric suggests conceptual and terminological overlaps between music production and the sphere of graphics programming, increasingly leading to the exchange of concepts on the level of tools. Moreover, the production process is being increasingly 'unified' on the level of program code, e.g. by tying visual and auditive cues to the same conditional variables.

One example from the short production diary to illustrate this point is the innovative audio LOD (level of detail) system designed for Halo3. This technique of using separate audio files used depending on the distance to the listener's position in a 3D environment works similar to identically termed LOD algorithms in 3D graphics rendering, i.e. interpolating different bitmap or texture files accordingly.

Another hypothetical concept in interactive audio design could parallel the Photoshop technique of 'smudging' the boundaries between areas of an image; auditive 'smudging' would involve blending 'colors' at edges, i.e. in this case musical or 'tone colors' [Klangfarben], for instance keeping the instrumentation of the previous track and gradually overlapping it with percussive elements from the new track. This playful exploration of semantic transfer between previously unrelated areas like graphics and music can provide a wealth of inspiration for production; AI programming is another such field which would lend itself extremely well to tapping into for tools design, similar e.g. to the use of AI algorithms for animation tools like Naturalmotion's Endorphin.

Coming back to the initial example of repurposing classical music, a historically comparative juxtaposition of digital game music with silent film scoring or film music in general can be visibly productive in enriching digital game production.

For instance, a game equivalent to 2001: A Space Odyssey, i.e. a game which would only use copyright-free existing (classical) music for its soundtrack might have powerful effects in the proper context. Another speculative but feasible approach, derived from the AI proposition above, would be to ‚model' the silent film performing situation in digital games by re-creating the musician as an AI entity and providing tools like the Sam Fox Moving Picture Music book as digital libraries of archived and metadata-tagged musical phrases similar to how Endorphin virtually models a claymation animator as an AI instance. First projects inquiring into AI use for musical compositions are already underway; (cf. e.g. Casella, P, Paiva, A. Mediating Action and Background Music) while these are technologically well-founded, comparative insights into areas like silent film scoring might notably enrich and help to direct the development process.

This confounding of explanatory approaches of course also applies to other areas of non-linear media production like narrative. For instance, the Wwise "transition matrix" technique with post-exit and pre-exit segments, maintaining "reverberation tails", decays and similar effects applied to the source segment to ensure smooth transitions might be a fruitful source of inspiration for thinking about algorithmically-driven narrative generation as in digital games or professional simulations. A non-fictional application using such a generative plot algorithm is e.g. the DEFACTO project training firefighter by generating emergent threat scenarios. (cf. Piquepaille, Roland, Using AI to Train Firefighters, ZDNet Blogs: Emerging Technology Trends 6 Jun., 2007)

To sum up, both silent film scoring and digital game music have been comparably marginalized forms of music, although the discursive vertices which produced this marginalization are quite different. While this niche status has precluded intensive scholarly attention, precisely this neglect and lack of theorization in both cases explains the high level of innovation and, paradoxically, the profound impact both genres had and have on more mainstream notions of musical style. Thus, with regard to current philological undertakings focusing on silent film music (cf. e.g. Mungen, Anno Music, Theater, and Image in the 19th Century: Towards an "Archaeology" of Film Music for the Opera Studies Group, Music Department, Stanford University, Stanford (California), USA, 8th April, 2002), an archeology of early digital game music could be a very worthwhile project for various reasons.

Some references for further reading:

Collections of scholarly resources on the topic of digital game music can be found here and here.

Harland, K., Composing for Interactive Music Gamasutra 17 Feb., 2007.

Bessell, D., "What's that funny noise? An examination of the role of music in Cool Boarders 2, Alien Trilogy and Medievil 2." G. King & T. Krzywinska (Eds.) Screenplay: Cinema/Videogames/Interfaces London: Wallflower Press, 2002: 136-144.

Stefan Werning works in the product analysis department of Nintendo of Europe, where he started in March 2007. He joined the Convergence Culture Consortium as a consulting researcher during his semester as a visitor of the Comparative Media Studies Program at MIT when he was a doctoral candidate and associate lecturer at the Asian Studies Center in Bonn, Germany. He writes on topics including e-learning solutions based on digital games and modeling terrorism in recent military policies to interactive media analysis.

---------- NEWS FROM THE C3 BLOG -----------

Recent Study Focuses on Swedish Viewing Behaviors. Sweden's MMS finds interesting gender differences in video viewing and a culture of downloading surrounding video series not made available in the U.S.
http://www.convergenceculture.org/weblog/2007/09/recent_study_focuses_on_swedis.php

The Disney Channel: Educating Children for a Transmediated World. Thinking of the Disney experience as an educational for media literacy in a transmediated world creates a new lens to view the P.R. machine of the company's cable channel through.
http://www.convergenceculture.org/weblog/2007/09/the_disney_channel_educating_c.php

C3's Balance between Industry and the Academy: The Consortium in the Press. A profile on Henry Jenkins looks at the Consortium and its relationship to both academia and the media industries.
http://www.convergenceculture.org/weblog/2007/09/c3s_balance_between_industry_a.php

SIGGART: Trying to Emphasize the Importance of Nimble UGC Campaigns. A recent project from The Gold Group provides a small case study of an initiative building a brand through user-generated content, providing the industry with a concise look into this type of campaign.
http://www.convergenceculture.org/weblog/2007/09/siggart_trying_to_emphasize_th.php

Gamekillers and Branded Entertainment. The recent development of an Axe-sponsored television show on MTV leaves the industry questioning the values and challenges with branded content, product integration and placement, and the balance between advertisers and audiences.
http://www.convergenceculture.org/weblog/2007/09/gamekillers_and_branded_entert.php

Quarterlife and the Rise of the Online Video Series. The new online video series generating significant discussion is based on a pilot never picked up by ABC from the creators of My So-Called Life and thirtysomething, the latest in a growing number of online projects from established names in the television industry.
http://www.convergenceculture.org/weblog/2007/09/quarterlife_and_the_rise_of_th.php

Jonathan's Story: Guiding Light's New Transmedia Project. The Procter & Gamble soap opera launches a book that fills in the gaps of a character's story while he is away from the actual show.
http://www.convergenceculture.org/weblog/2007/09/jonathans_story_guiding_lights.php

Catching Up: Net Neutrality, Online Video Ads, and Nielsen. The Justice Department weighs in against the concept of net neutrality, while discussions continue about the nuance of pre-roll ads for online video and Nielsen has unrolled a new product to help buyers trace the placement and performance of their 30-second spots.
http://www.convergenceculture.org/weblog/2007/09/catching_up_net_neutrality_onl.php

C3 Community: Jason Mittell on Canon and Tenure, Edery on Violence, Kozinets on Britney. Mittell examines the recent Time list of the 100 greatest American TV series of all time and looks at the worth of online publication for those within the academy. Meanwhile, David Edery weighs in on the educational value of media violence, while Rob Kozinets focuses on the Britney Spears media maelstrom stemming from the VMAs.
http://www.convergenceculture.org/weblog/2007/09/c3_community_jason_mittell_on.php

How We Make Media Theory at MIT... Henry Jenkins cross-posts a recent piece from his blog focusing on the history of media theory at MIT and the roots of the Program in Comparative Media Studies.
http://www.convergenceculture.org/weblog/2007/09/how_we_make_media_theory_at_mi.php

An Interview with the Organizers of Fandom Rocks (4 of 4). The final part of this four-part interview with the heads of this fan-driven fundraising initiative looks at the impact of social networks on the organization's activities, future plans, and the legacy they hope Fandom Rocks will create.
http://www.convergenceculture.org/weblog/2007/09/an_interview_with_the_organize_3.php

An Interview with the Organizers of Fandom Rocks (3 of 4). The third part of this interview with the organizers of a grassroots charity fundraising initiative among fans of CW's Supernatural examines the group's campaigns, the selection process for charities, and the meaning of these pro-social activities for the fan community.
http://www.convergenceculture.org/weblog/2007/09/an_interview_with_the_organize_2.php

--------------- FOLLOW THE BLOG ---------------

Don't forget - you can always post, read, and carry out online conversations with the C3 team at our blog: http://www.convergenceculture.org/weblog/.

--------------- CLOSING NOTE ---------------

No More "Narratively Inconvenienced?"
Story-Pleasure Through Complementary Media Use in Lord of the Rings online

By: Doris C. Rusch

A new door has opened to the famous narrative universe created by J.R.R. Tolkien. With the MMPORG Lord of the Rings online (LoR) it has become possible to dig oneself even deeper into the world of Frodo Baggins and his fellows and to add an interactive experience to the pleasures of reading the books and watching the films.

Admittedly, I was skeptical at first. Yes, I liked the books well enough and I had watched the films several times, twice all three of them in a row (extended version!). But still, I could not imagine how a story as complex and dense as Tolkien's Lord of the Rings could be satisfyingly conveyed into a game. After all, games are still a little "narratively inconvenienced". The attempts at storytelling made so far mostly by single player games often suffered from poor hypothesis building opportunities for the players. What would happen next? You only knew when you had tried and failed once. "Learning by dying," as Gunter Hager, game designer at Games that Matter likes to call it. But when surprise always gets the better of suspense, the narrative experience lacks an essential quality.

With LoR, things are different. LoR is a story-centered MMPORG that manages to significantly regulate player interest through the story. The story is absolutely central to the playing experience and a good reason to stay put just another hour longer. But the reason for this interactive miracle cannot be found within the medium itself, but is due to the complementary use of various platforms. Basing the game on Tolkien's world and its famous events facilitates in-game hypothesis building by providing a well-known out-of-game reference system that strengthens the game's fictional components. Like in the early days of filmmaking, when filmmakers relied on the familiarity of the plots they used rather than on their medium specific storytelling expertise to cognitively and emotionally engage their viewers, LoR uses the familiarity of the story to keep the player glued to the keyboard. However, it is important to note that it does so by offering a different angle to the known events, not by regurgitating them.

The game is structured in books and chapters, which proceed in a linear, pre-defined fashion. Epic quests that follow a fixed sequence and can only be finished in a group (or a "fellowship" as groups are called in LoR) move the storyline forward towards the liberation of middle earth. Between epic quests there are numerous normal single and group quests that give the player some space to level up so he / she will be up to the next challenge in the epic quest series, explore the world in a more liberal way and to flesh out his / her character.

Naturally, the epic quests have the strongest connection to the events known from the books and films. But since one does not play Frodo or any of his immediate fellows, but a more distant cooperator, complex hypothesis building processes are initiated. The way the game ties up to the known events without simply reproducing them keeps the player guessing and interested. Two main categories of player interest regulating interrelations between books / films and game can be identified:

The first category deals with visible gaps in the background story left by the books and films. I like to think of this category as the "I've always wondered..." category. E.g. I have always wondered why Gimly and Legolas are wary of each other. What is the story behind the racial conflict between elves and dwarfs? The game offers many answers to this sort of question. By providing different information depending on which race -- hobbit, human, elve or dwarf -- you choose to play, it further stimulates exchange between players (or motivates the player to play different races to get the full picture).

The second category of interrelations between game and previous media that regulate player interest deals with the tension that arises from recognizing a familiar element and wondering how it will be instrumentalized in the game. When you enter the Old Forest you know that it will be dangerous and that somewhere along the line, you will meet Tom Bombadill. But you do not know what your dealings with him will be. Or take the quests revolving around the Weathertop. Without the previous knowledge from the books / films, the Weathertop is just another location -- a platform overlooking plain lowlands. But once you know that this is a place of drama, that Frodo was severly injured there by the witch-king, you look up at the little hill with a sense of foreboding and anticipation. What has this place in store for you? It is clear that something dramatic has to happen there, something that echoes the meaning this location had in the book or the film. And you will hang on until you know what role you are playing in the context of the already familiar story.

LoR is a true enhancement of Tolkien's narrative universe and it consists of countless cross-references to the original plot. It is an intellectual pleasure to recognize elements from the book, to realize when the game fills another gap, to hypothesize about when and where one will cross paths with Gandalf and Strider again and in which way one is going to influence the further course of events.

Games as media might still have a problem to initiate narrative hypothesis building processes. In many games, the story will still be a reward at best (if not a nuisance), not the main reason to keep playing. But until we have learnt to couple rules and fiction in a way that allows the fiction to stay in the foreground of player interest during game-play, making good use of multi-media storytelling seems to be a very effective way to overcome the challenges games face in this regard.

Doris C. Rusch is an consulting researcher with the Convergence Culture Consortium. She has done postdoctoral work at the Institute for Design and Assessment of Technology at the Vienna University of Technology in Austria and worked as community manager for the online game Papermint through Austrian game company Avaloop. She is preparing to take a postdoctoral position with the Singapore-MIT GAMBIT Game Lab in the Program at Comparative Media Studies at MIT.

****************
Compiled and Edited by Sam Ford (samford@mit.edu)
http://www.convergenceculture.org

You are receiving this update as a member of MIT C3.

To subscribe or unsubscribe, send a request to samford@mit.edu.