Particle or Wave? Linear and Non-linear Storytelling in Museums
Juan Sanabria, GuideOne, USA, Jason Reinier, Earprint Immersive, Inc, USA, Peter Samis, SFMOMA, USA
In this session we will discuss recent mobile based projects that use a mixture of audio interpretation and interactivity to help visitors explore and dive into contents. Our aim is to outline methods, techniques and models of user experiences and shed light on why some interactive experiences are more successful than others. We’ll compare examples where non-linear storytelling worked well and examples where interactivity diluted the experience.
Part 1: A brief history/Setting up the problem
How do we learn: through systems or through individuality? Through relationships between objects constellated in a common room or through complete and utter absorption in a nameless moment, when time and space collapse/fuse within us and we stand transfixed? We daresay both systems apply, though the learning that issues from each circumstance are of a different character.
Beholding works of modern art, we are surprised by the feeling of obligatory silence that they bring on, rather than by what they compel us to say. (Bosch, 1998) [i]
That’s on a good day—a very good day. But we all know that the archetypal transfixed visitor—and our own experiences as that visitor—are the exception rather than the rule. For many visitors, the verbal inadequacy before the picture plane comes not so much from their experience exceeding the bounds of speech as from simply not knowing what to say. “Give me a handle here,” they think, as they reach for a wall label—and the artist’s name, nationality and dates may be all they get, implying that recognition and classification are the most important activities in mastering museum-going. Alternatively, they might get a few sentences—sometimes even a mini-treatise—that attempts to situate this object in an unseen cosmos, a world of circumstance. With a little bit of luck, it might even refer to the visible characteristics of the object, as they take it in from an oblique angle at their label-reading distance.
It is into this world that multimedia interpretation penetrates—hopefully, as Kafka’s metaphor goes, empowering the art to act as an axe in the frozen sea within us. More often, no doubt, offering us a few interesting factoids or a story about the particular works on view.
This question: how to optimize the art encounter, is a real one, still. In today’s parlance, “It’s a thing.” Each artwork comes with its personal InfoCloud, and each person comes with his or her own proclivities, and we’re in a constant quest to mesh the two, to create the spark that bridges the cloud and illuminates the Subject.
Traditional wisdom was that the linear audio tour, with its narrative voice-over intoning pieties in the visitor’s ear, was the proper way to deliver the informative, even reverential, experience that viewers desired. Herds followed, and in blockbuster after blockbuster, crowds gathered around selected artworks, while other works lay fallow, unattended. But always, the line, or narrative thread, was observed. We’re going way back here: early days, like the 1980s. Think tape.
Come the Nineties, with the advent of digital technologies, random access tours enabled museums to go non-linear for the first time, and in so doing to treat their permanent collection highlights in a scattershot way, free of any over-arching narrative.[ii] And with digital non-linearity came sub-levels. “Press 1 to hear more about apples, press 2 to hear about oranges.” We were still a ways from visual menus on iPod Touches and smart phones, but message length was shortening and info options multiplying.
And so it has gone since the 2000s. Now, no self-respecting museum—at this conference, at least—is without its random access multimedia tour or app offering a list of options for the objects on view, each one a flash going off in the latent InfoCloud surrounding each work. (“No flash in the galleries,” I hear you say. Don’t worry: these are InfoFlashes.)
What have we gained and lost in this transition? On the one hand, visitors have autonomy: they can be pulled and pushed by their visual attractions in a beelike Brownian motion of gallery-going, secure in the knowledge that wherever they alight, a honeyed info-nugget awaits. On the other hand, our drive for story, a greater over-arching narrative, may be thwarted. The ideal solution is to offer both: the object-specific puncta that allow people to dip in and out as they wish, as well as the web of story that knits these objects in relation. The pitfall here is logistical: in permanent collection galleries, artworks are subject to being swapped out in ways most unconducive to preserving the filament threads of a narrative web, and most museums are still at a stage where once they’ve put an audio tour to bed, they don’t have the resources to re-craft their story stops to reflect the new—and perhaps equally transient—reality of a changed gallery hang. Calling for solutions to this problem!
For these two tendencies—the punctum and the linear, the isolated instance and the river—are both important to us, and essential. They are as essential to each other as particle and wave, the two ways of describing the way light behaves. Both are correct, neither is exhaustive. Neither one depletes the subject. We expand in the moment and we move in story.
Having said that, there is a limit to our attention spans—and to the stamina of our legs and feet. So a stop that’s too long, or menu after menu with too many options (even if they’re tantalizing) can lead to exhaustion, and leave the fruits of a lot of creative content production on the table. The Wikipedia article on “Attention economy” describes Herbert A. Simon’s invention of the concept of Information overload:
He noted that many designers of information systems incorrectly represented their design problem as information scarcity rather than attention scarcity, and as a result they built systems that excelled at providing more and more information to people, when what was really needed were systems that excelled at filtering out unimportant or irrelevant information.[iii]
After all, not everyone visits our galleries the way Alfred Barr and William Rubin installed them: in a wheelchair! (Figure 1)
https://mw2014.museumsandtheweb.com/wp-content/uploads/2014/02/barr-300x231.jpg 300w, https://mw2014.museumsandtheweb.com/wp-content/uploads/2014/02/barr-388x300.jpg 388w" sizes="(max-width: 900px) 100vw, 900px" />
Figure 1: “Alfred Barr, Dorothy Miller and the Demoiselles.” Detail from the exhibit “Sites of Modernity” by the Museum of American Art, Berlin. Part of Time Machines – Reloaded at the Van Abbe Museum, Eindhoven, The Netherlands, 2010–2011.
Part 2: Letting go: Story, authority and visitor experience
As the technology available to deliver museum interpretation has evolved, storytelling has evolved along with it. This cart before the horse development has meant content developers – and museums – have had to continually relinquish control to keep story and medium in synch. As the table below shows, in the beginning there was the cassette tape, which gave content shapers maximum authority over both story – crafting a linear narrative arc of specific length – and visitor experience: telling the visitor where to look, when and in what direction to move, which objects to stop at, and exactly how long to spend in the exhibition.
TABLE 1: Levels of authority
|Linear||Non – Linear||User/Community generated|
|Maximum authority over both story and controlling visitor experience||Less authority over visitor experience, but still control over story||Least authority over story and visitor experience|
When keypad-equipped audio tour players replaced the Walkman, museums could still control parts of the story, but had to let go of the linear narrative arc. The visitor’s behavior took on a greater role in shaping their experience. We could suggest visitors follow a numbered route through a temporary exhibition, but the choices they made about which objects to view, which content to access, and in what order to do that were, in fact, up to them. No longer could we build a single connected story with a beginning, middle and end – unless it was contained in a single “chunk” of content – and expect the visitor to hear it in any particular order. Visitors became freer to listen to as much – or as little – as they liked.
Building on this transition from the museum’s linear narrative to the visitor’s, software such as Halsey Burgund’s Roundware, when paired with a mobile device and some sort of location service, offers a platform on which to develop user-generated narratives—if the museum is willing to give up even more control over both the story and the visitor experience. In this case, the job of content development becomes a collaboration with the museum’s community – a process of shaping, prompting and editing. Location technology lets us deliver content to specific locations – so instead of moving the visitor around in a game of “Simon Says,” we let them wander where they will, and deliver the content to them whenever they happen to arrive.
Roundware encourages visitors to become content creators by adding their comments to a “soundtrack”: a mix of visitor comments and an evolving musical score – in response to museum-curated prompts. The museum’s sphere of control is now in the prompts we create, the GPS points we choose to “seed” with comments (which both encourages more comments at that location and “models’ the level of quality and engagement we hope the comments will have), and our selection and/or editing of the recorded content the community creates. Visitors create their own experience – and influence the experience of other visitors. This experience is enriched by more visitors adding more – and more interesting – comments to the soundtrack, and is augmented by the visitor recording comments at new locations, thus adding new GPS points to the recorded mix and encouraging overlapping, ever-expanding content “clouds.” When embarking on this kind of open-ended, user-generated experience, all parties involved must make a firm commitment to the nurturing and management of the content. With the proper pruning it can grow like a well-cultivated garden; the risk is that it can also grow out of control and create a thicket of untended comments that may confuse, bore or offend the listener, as opposed to providing insight and learning.
As content developers working together with museums, we use all three of the approaches to content presented in this table. Finding ways to develop compelling narratives using non-linear technology presents unique challenges, but also inspires creative solutions. Crafting relevant, topical, focused “chunks” of content – and using tagging, labeling and metadata to help the visitor quickly find what’s of interest to them – allows us to aggregate, cross-reference, and relate disparate objects in the collection – and chunks of those objects’ content “clouds” – to one another, introducing new opportunities for comparison, contrast, and, we hope, insight across the sea of sometimes disparate objects that is a museum. Non-linear storytelling harnesses the power of the visitor to mix, match, and make connections between objects, historical periods, artists, and cultures – offering the possibility of brand-new, individualized, visitor-created narratives constructed from seemingly disparate content chunks. Finally, curating and creating focused opportunities for visitor-generated content begins to get at what most museums say they are after: getting visitors to engage with the museum and its collection in meaningful ways; to become active participants in the museum’s community; and to enjoy and remember their visit, and want to come back for more.
Part 3: Back to reality: Examples and potential of linear vs non-linear interfaces
In the process of describing the visitor experience in the context of linear vs. non-linear engagements with audio tours and interactives, we’ve touched on threads of theoretical experience and the history and future of audio guide technology and how this has impacted the concept of linear storytelling. Beyond merely philosophical discussions about the topic, we were also hoping to shed some light on a few methods of presenting information in ways that are appropriate for the subject and most advantageous to the user or visitor. We’ve chosen a few examples of interactives that present compelling audio and video in a way that does not overwhelm the visitor and that is contextually appropriate.
A. The Diego Rivera Murals at the Detroit Institute of Arts
In this iPad tablet interactive, The Diego Rivera Murals at the Detroit Institute of Arts, a user stands in a large room surrounded by the murals and nothing else, except perhaps for a few fellow visitors. The purpose of this multimedia guide app is to allow users to visually browse the walls and then click on various elements in the interactive to pull up information about details of the mural.
This non-linear approach to the content, allowing the visitor to consume short segments of audio and video, has worked well for the museum, and engagement in the content is significant. This type of simple interface combined with high quality content encourages users to follow their noses and consume short 2 minute segments in whatever order they choose. There are also tours but the strongest part of this piece has always been the open-ended experience because it allows users to mimic the natural exploration of a museum within a structured app.
https://mw2014.museumsandtheweb.com/wp-content/uploads/2014/02/rivera-222x300.jpg 222w, https://mw2014.museumsandtheweb.com/wp-content/uploads/2014/02/rivera.jpg 900w" sizes="(max-width: 584px) 100vw, 584px" />
Figure 2: Rivera Court iPad app. Native iOS app. Full screen mockup of the Rivera Court allows a user to click on mural details. Everything is presented equally and stops are not prioritized over each other.
B. Clouds Over Cuba
Clouds Over Cuba is a web-based interactive that immerses a user in the 13-day-long Cuban Missile Crisis. This media-rich experience is centered around a timeline navigation that has numerous jumping off points and non-linear ways to explore the full story. The interactive interface borders on over-design and information overload, yet the content is extremely compelling and engaging even if one decides to resist clicking on a detail and diverging from the main narrative.
As a time-based real world event it’s entirely logical that the creators chose to follow a linear path while providing numerous access points along the way where users can jump off and explore details to get additional texture and context. Essentially, the narrative just would not be the same without the linear timeline – it wouldn’t have the same emotional build. The non-linear details are extremely interesting and when experienced with the linear flow, it results in a great overall story that feels like a high quality documentary production.
https://mw2014.museumsandtheweb.com/wp-content/uploads/2014/02/Screen-Shot-2014-02-03-at-3.31.16-PM-copy-300x192.jpg 300w, https://mw2014.museumsandtheweb.com/wp-content/uploads/2014/02/Screen-Shot-2014-02-03-at-3.31.16-PM-copy-467x300.jpg 467w, https://mw2014.museumsandtheweb.com/wp-content/uploads/2014/02/Screen-Shot-2014-02-03-at-3.31.16-PM-copy.jpg 1193w" sizes="(max-width: 584px) 100vw, 584px" />
Figure 3: Clouds Over Cuba. HTML 5 web application. Immersive web documentary with dynamic timeline. Commemorates 50th anniversary of Cuban missile crisis. Explore 15 related events in greater depth. Explore in detail from multiple perspective. Information is presented chronologically.
C. Soundcloud.com Audio Player
Linear vs non-linear experiences are not just about a one way process of content design and consumption. Social experiences have opened up new models of content design and a third interface worth mentioning is the player on the Soundcloud.com website. It’s a central part of the browsing and listening experience and it’s much more than just a standard linear listening tool. For example, users can insert comments at various points in the audio track and these comments enable both social interaction and serve as a utility that can provide more information about details of the audio. They allow listeners to take part in an annotated discussion around music, thus enriching the experience however non-linear the experience may become.
https://mw2014.museumsandtheweb.com/wp-content/uploads/2014/02/sound-cloud-300x86.jpg 300w, https://mw2014.museumsandtheweb.com/wp-content/uploads/2014/02/sound-cloud-500x144.jpg 500w" sizes="(max-width: 749px) 100vw, 749px" />
Figure 4: Sound Cloud Audio Player. Allows users to annotate audio creating a social and non-linear experience
Getting back to the context of museums we have to ask—as above with Roundware—what would an experience be like if a user could engage in an audio or multimedia experience where they could discover and engage with media footnotes made by other users?[iv]
We’re quite curious about how other non-linear experiences would play out. Consider the possibility of an audio tour where the curator had the opportunity to add informal side footnotes into the player, thereby creating an additional level of interpretation – a way to have side discussions that would almost give the impression that they were happening in real time. What could this add? Would anything be taken away from the museum experience?
In pointing out the above interfaces we’re simply reinforcing the idea that linear or non-linear should not necessarily be a consequence of evolving technology, styles or technology, rather it’s a design choice and one that should be made intentionally by the content creators. How the visitor learns best depends on a host of factors but being aware of and considering the impact of our content design will enable us to create higher quality and more contextually appropriate experiences.
In exploring these topics it has been tempting to diverge into the many issues including: What do visitors want versus how do they actually behave? What role should audio interpretation play and what are some approaches to writing and audio design, and what role should technology play in the experience?
We’re looking forward to covering these topics and discussing real world examples in our upcoming session where we’ll have a chance to offer insights on audio design and interface design. We’ll also conduct a live experiment with audio, and lead a discussion on issues that are sure to continue long into the future, as the idea of what is interpretation and how do visitors learn best continues to evolve.
Thank you to Catherine Girardeau from Earprint Productions for detailed assistance with this paper.
[i] Eulalia Bosch. The Pleasure of Beholding: The Visitor’s Museum. Barcelona: ACTAR, 1998.
[ii] Of course, galleries always had the potential to be non-linear: to be approached as spatial wholes. Not everybody turns right and proceeds along the wall at a crawl, reading it like a book (and a Hebrew one at that!). Visitors have always had the option to take in the room in its breadth, and then sense which work pulls them in, following their nose and whims where they led. In fact, one could argue that early audio tours enforced a linearity on what had previously been an essentially non-linear experience. But I don’t have the early tracking and timing data to make that comparison, and my suspicion is that there have always been different kinds of visitors, with different approaches. While Hungry Minds cling to the walls, reading every label, Social Animals follow their friends, and Facilitators look for the moments when their kids’ eyes light up.
[iv] Inviting comments into an interactive designed for museums also brings with it a host of issues including the possible need moderation or filtering to promote a quality experience.
. "Particle or Wave? Linear and Non-linear Storytelling in Museums." MW2014: Museums and the Web 2014. Published February 4, 2014. Consulted .