Reading Lev Manovich’ “The Language of New Media”
A book review
It is not an easy challenge Lev Manovich sets in his book “The Language of New Media“. In the early days of cinema hardly anyone could foresee the enormous cultural impact this new medium would have on our society. A new artistic language, cinematography, was born; but no one recorded its first steps systematically. This wasn’t only because people didn’t see the importance of cinema. Documenting an emerging language is a form of historiography; merely impossible without the advantage of hindsight. It simply takes time, before it becomes clear what the right or interesting historical questions are. So, I never believed Manovich, could succeed in “providing a potential map of what the field could be” (p11) back in 2001. But, I do feel his book deserves a close reading. The first steps in building a theory are usually the hardest and, despite the difficulty of the task, Manovich did cover interesting ground.
Manovich tries to understand new media with the eyes of an artist. Two ingredients are necessary for every art piece. First there is the influence of existing media. A key concept in Manovich’s theory of new media is the cultural interface (p 69). Much in the way the human computer interface, structures the interaction between humans and the computer, cultural interfaces give structure to the users’ interaction with culture (or “cultural data”). The interfaces of CD-ROMs, web pages, games and apps are all cultural interfaces. Often new media reuse ideas, forms and conventions from older media. Early cinema build on theater, rock music on blues. So the forms of new cultural interfaces also stem from older, already familiar forms such as: magazines, newspapers, photography or -indeed- cinema. Second, the new media have offer new technological possibilities and affordances to the artist (Manovich speaks of operations). As media makers experiment with these new operations, some older conventions will fade and new conventions emerge. So to understand the language of new media we need to look carefully at its ancestors and at the possibilities of computers.
Manovich believes three older cultural forms are most important to describe the language of new media: print, cinema and the human computer interface (in practice Manovich specifically refers to the graphical user interface, the GUI). Print, the oldest form, was adopted first. The page is a cultural convention of print that persisted into the digital age; although the worldwide web also revived the ancient form of scrolls. Hyperlinks were the most radical and disruptive innovation to texts. Hyperlinks challenge old ways of organizing information. The structure of the web isn’t like a library (with an index) or a book (structured through rhetoric narrative) (p 77), but more like a walk on the beach where you may find random objects one after the other (p 78). Cinema, is the second cultural form that influences new media; in particular games. The moving camera is a convention borrowed from cinema, apt for navigating in virtual worlds. Also many games borrow story forms from cinema. But games moved beyond cinematic conventions too. They break the rules of (natural) perspective and story, in search of forms that are more suitable for interaction. The graphical user interface (GUI) brings controls, menu structures, and the desktop metaphor to new media. These seem to fit in, but there is friction as well. The controls offered by GUI’s need to stick to the underlying metaphors (for usability) and simultaneously blend in with the story world of the cultural interface. While new media often show mixtures of print, cinema and graphical user interfaces these mixtures may be rough. Often, the underlying ideas of what the screen represents differs so much (flat surface with information, window into an immersive environment, control center), that these forms cannot be wedded easily.
So print, cinema and GUI’s, are to new media what photography and theater were for cinema. They provide the “raw material” of cultural conventions that are available to new media makers. But they do not yet describe the other major influence on the language of new media: what new media creators tend to do with this material. In other words, Manovich needs to turn to the affordances of new media creation technology, or as he calls it: the operations. Cinematography, for example, makes creative use of different types of shots and editing techniques. Do these operations have new media equivalents? Manovich believes these are selection, teleaction, and compositing. Teleaction, the ability to see and act at a distance, allows the camera to be everywhere and users to cooperate across the world. In massive online multiplayer games for instance or in massive online open courses. Computers allow easy access and reuse of older material and both selection and compositing capitalize on this possibility. The rise of the DJ, cleverly choosing and combining existing materials to create new music, is an example of the power of selection and compositing as means to create new forms. The growing importance of special effects in movies, another.
New media creation technology also allows to create new types of illusions. Computer generated images for example. Mimesis: to mimic nature, has been an important goal of cinema and this remains so in computer simulation. For computer vision scientist, for example, cinema is an important market and source of inspiration: “High quality, means virtually indistinguishable from live action motion picture photography” (p191). In practice, creating a realistic immersive experience involves more than just photo-realism. It involves many forms of mimesis like: touch, interaction (with virtual characters), moving about in virtual space and 3D graphics. But mimesis isn’t at all one-dimensional. Rather, in cinema, the quest for realism progressed through a succession of ‘codes’ in which only some parts of the experience mimic real life and the viewer filled in the gaps. In computer graphics ‘realism’ was first achieved by ‘deep perspective’ and later by ‘correct lightening and shading’. Considering the broad playing field for new mimetic codes in new media (for touch, interaction, movement and so on), we may expect a long period in which media makers try to set new mimetic frontiers.
After describing the conventions and affordances of the new technology, the cultural interfaces, operations and illusions, Manovichs turns to the emerging genres or forms. He focusses on two of those new forms: the database and navigable space. Databases are special because they have no beginning or end. Much more than any old form, new media objects allow the user random access to items in the piece. The user experiences new media through hyperlinks, browsing and searching rather than through the guided tours that traditional narrative forms offer. The second form is navigable space, which is the dominant form in many games and interactive stories. It is the first time space is a medium. Space can now be stored, retrieved, transferred and it can be used to tell a story. Today, architecture touches media and new media designers need to learn how to tell a story with space.
The Language of New Media, is a rich book which offers a comprehensive theory of new media. It is an interesting idea to use cinema as a model for new media and with this approach Manovich could get to some marked insights in the language of new media. But I do not feel the book can live up to its goal to be a map of the field. Manovich tries to bring clarity to the field by thinking of new media in layers: cultural interfaces, operations, illusions and forms. But the distinction between these layers is not always clear and he fails to show how these layers influence each other. How did, for example, spatial navigation as a form emerge through the operations selection and compositing on cultural interfaces? Manovich raised the question, but he gives no answer. There were many other questions like this, that haunted me when I was reading the book. At the end of the day Manovich book is more a structured description of the status quo of new media at his time, rather than a theory through which we can understand it’s development. And then there is the scope of the book…
When you compare (early) cinema and new media, you start inspecting the parts that are most like cinema. This way, Manovich misses a lot. As an interaction designer I can hardly agree with Manovich treatment of the human computer interface (and its history). Interaction design in games draws much more from board games and free play, than from desktop tools. So board games are important cultural interfaces that feed new media. Also, in contrast to film, new media has multiple context of use, and is paid with a bigger diversity of business models. This must have an impact on the developing language of new media, and studying the history of cinema cannot tell you what it is.
Shortly after the language of new media came out, web 2.0 unfolded. This changed what most people see as the dominant forms of new media. As early as 2005, four years after the language of new media was first published, Mark Deutze, considered “participation”, “remediation”, and “bricolage”, to be the principal ingredients of digital culture. Deutzes description of – in a word – participatory remix culture, would be part of any new book on the language of new media, but Manovich could not have foreseen this development when he wrote his book. Manovich knew that this might happen, as it happened with cinema before. He writes: “It is tempting to extend this parallel a little further and speculate wether this new language is already drawing closer to acquiring it’s final and stable form, just as film language acquired its classical form during the 1910’s. Or It may be that the 1990s are more like the 1890ies, in the sense that the computer-media language of the future will be entirely different from the one used today”.
So did Manovich write his book 10 years too early? Probably not, rather I think his scope was too wide for his analytical means. Cinema has never been the ‘meta-medium’ which the computer is today. Cinema, has always’s been expensive to produce and it has always offered a special and exclusive experience to the audience. So the technology and the economics (business models) of cinema quickly converged, in turn allowing for stable cinematic conventions to emerge. The meta-medium of the computer is cheaper, more diverse in the types of experiences it can offer, the contexts of consumption and the diversity of business models available to support their production. There is not one language of new media, there are many. And unlike with cinema, they are not likely to revolve around a common core any time soon. Manovich was right and observant when it came to games and digital cinema, but he may have underestimated the power of (linear) narrative, overestimated virtual reality and he may have missed forms like augmented reality or social media. Many new media forms, like immersive games and websites have stable language for years. For these forms, Manovich book provides at the least a descriptive framework. But other forms, such as augmented reality, social media and embedded media have their proverbial 1890ies still to come.
In my post Cognitive Bias in the Global Information Subway , I discuss the language of search and its impact. In Collateral Damage of the Robots Race (on the Web) and Social News Needs a Nuanced ‘Like’, Quickly, I discuss the impact of artificial intelligence on the way web experiences are structured.
Here is a good Dutch summary of “The Language of New Media”.
Filed under: (re)thinking media, review | 5 Comments
Tags: Book Review, Business Model, Cultural Interfaces, Lev Manovich, Semiotics, The Language of New Media, User Experience