CART Transcript for Is it working? Analytics and evaluation for digital tools and projects

Thursday, April 3, 2014 10:33 a.m. – 12:02 p.m.

Museums and the Web 2014 Conference: Is it Working? Analytics and Evaluation for Digital Tools and Projects.

Held at:

Renaissance Baltimore Harborplace Hotel

Ballroom A

202 East Pratt Street

Baltimore, MD

Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.

>> Jane Finnis: Hi, everybody. Hello? Hello and welcome. My name is Jane Finnis from the UK and I’m sharing the session on “Is it Working? Analytics and Evaluation for Digital Tools and Projects.” So the fact you’re all here means it maybe isn’t working so well as we would like it to work. I think that’s what the ‑‑ we’re hoping, what we’re looking at. How different people are using analytics and data to tell stories and create change within an organization. Take risks, try things out and try and be better at what we do.

So up first is Jane Alexander from the Cleveland Museum of Art and she’s here with her colleague, Keeli Shaw. I asked everybody speaking to tell me something about themselves that you wouldn’t know. You all know that she works with the museum, I’m sure you’re aware of the gallery project. So Keeli is a runner. And has run over ‑‑ sorry, Jane is a runner.
>> Keeli Shaw: I run too.
[Laughter]
>> Jane Finnis: How many marathons have you run? So Jane’s run ten marathons, which is impressive. It’s not working on the running front for Jane at all at the moment. Keeli is ‑‑ she can do it. So if anyone has a bat, you can find one during the session. Challenge, a volunteer. You never know. There’s probably a gift shop downstairs. Will you give us a demo?
>> Keeli Shaw: Sure.
>> Jane Finnis: You guys are up first.
>> Jane Alexander: Well, I’m Jane Alexander, the CIO of the Museum of Art, and this is going to be the first time I do a Gallery One presentation without showing our famous video. So this is for people who have already seen the video.
So Gallery One is part of our building project that just was completed. It was a $350 million renovation, and part of that was to create a place where art design, technology would be in one space and that it would be a place for — to engage with art in different ways. The big question has been is Gallery One working.
So just to give you ‑‑ during the couple years that the building was going through the project, there was a lot of evaluation being done. A lot of visitors thought art museums were boring and dusty. Even if they had a PhD in physics, if they didn’t study before they came, what were they going to do. Families thought their kids would be bored.
In putting together Gallery One, we decided to give people the tool set to be able to look closer at art, to engage in new and better ways.
So attendance increased, yes, 39% this year. We also opened up this brand new fabulous atrium and restaurant and store. It’s hard to say that 37% of that is because of Gallery One. I’ll say that — no.
[Laughter]
It’s really because of that. We also have a 25% increase in family visitorship. So that’s been huge in just one year of opening.
So Gallery One is located when you first come in, and when we put together the goals of Gallery One, even though the project was restarted in the middle of 2010, we really kept the same goals that had begun in 2005.
Through this, we really want to ‑‑ there was a big thing about the visitors having fun. And so I just like to repeat what our goals are because a lot of people ask different questions. Does it do this, does it do that. We’re learning that it does. But what we wanted people to do was to have fun with art. We wanted them to use interactive games and interpretation to spark their understanding and social experiences with art and find transformative moments of study that make them relevant for today.
Another thing we had during the first year, the opening, is a lot of community partners were really involved and interested in Gallery One. One example is our museum, our Cuyahoga public library had a new opening branch. And they have a tech walk. If you’re in Cleveland on the west side, the east side, they don’t speak to each other.
This library said wouldn’t it be great if we got people to the art museum which is on the east side. So they had taken one of our lenses, from the 1930s, which has shape and also our ArtLens to engage visitors to come to the museum.
We have a solstice party every year. And last year was the first time everything — the atrium, everything was complete. We took our collection walk and at eight times the size put it on the 1916 building and there was over 5,000 people from northeastern Ohio that came to this. These were people who didn’t even know Gallery One existed inside.
The other big thing about Gallery One, it was the first test bed of our digital strategy. One reason we sort of changed gears in the middle of 2010 was because we wanted to make sure every single thing in, on the wall, in the ArtLens, pulled from our back end systems. We didn’t know what the hardware was going to be or the space. But we knew it was going to change. If we put the work into the back end, whatever device is new and latest we’ll be able to support.
So in our ArtLens, our online collection and our wall, we actually have created a CMS that pulls from our digital asset management system. And that system pulls all the metadata from our collection, meaning if something is in session, goes on loan, goes to conservation, it’s reflected on the wall. It’s reflected dynamically in our lens.
With that in place and Gallery One opening last year, people are like it’s been probably a relaxed year. No, we revved it up.
[Laughter]
Because you have that, you’re maintaining that. We’re doing new devices. But we also looked at the strategy. This is our simple little ‑‑ all our databases at the Museum of Art, and our goal is for everything to talk to everything in some way. That we set up a system, and anytime we look at a new system or renovating or modernizing an old system, we look at the whole strategy. This is our current system setup today.
We also look at our technology infrastructure. Without this in place with the building project, we wouldn’t have been able to do half the things we did with Gallery One, but also how to increase and improve it.
These two teams are working side by side nonstop, and they all told me do not add another project.
[Laughter]
And I said, well, I can’t promise that, but this is what our saying in IMTS is. No one‑offs. There is no project that someone comes up that has no correlation to anything else, unless it’s going to bring in a million dollars to the museum, we’re not going to consider those. We’re going to look at it from the bad end, hardware, sustainability and the hardware of the systems to support it.
We have an application team, and they were super busy this year. They had a couple projects they were doing. One thing was the wall. Objects on the wall, when you touch it, it shows what collections involve it. You can also switch to its medium, to its time and to its gallery. Except for the gallery, everything else had it sort of rolled up, unless we had departments that had 150 words on the wall. We set this up but didn’t have a way to automatically pull in new objects that had dates that didn’t exist before, or new mediums and things like that. So we figured out a way, we worked with Piction to make this dynamic. Our team is so proud, they will go against any museum in the country, we have been the cleanest ‑‑ we have the cleanest data now because our data is projected every day all over the place.
[Laughter]
So when we see ink on silk, someone takes a picture and the tech in the Gallery One puts in a picture and sends it to application and they immediately get to see is this some sort of weird back end system we changed or was this a data problem, or is this an application thing. We did a lot of updates to Piction and flattening the data.
And this is our holy grail, oh, the central channel because we have patrons and we have forward ticketing. We have a different system for our store, for our restaurant. We have a different store for our donors. We all know there’s no system that does great even one thing. What we decided to do was create a central table that we could pull information from all those different databases so that we can begin to get a complete picture on our visitors.
So we’re in the middle of that project and that’s been pretty exciting for the business side of the museum.
The other big thing we did that was painful to our CTO was upgrading our wayfinding. Every single gallery has not only Wi‑Fi, but wayfinding. We started the project. We thought we would add just a couple AP ‑‑ sorry, I’m nervous. Thinking about that year of that project.
[Laughter]
And when we started the project, we realized we couldn’t do that because we were opening up galleries. When we looked at how they had been placed there was quite a bit of straight lines, and the head of design said no, you’re not cutting a hole in our brand new gallery. Come up with a new idea. We did an assessment of the entire building and we came up with this initially posted solution. Another thing done this year. As you can see, the light tracking was really tracking to accommodate right away, but it is an operating cost we want to get rid of quickly.
So we also had with the new upgrades and the new IOSs it was constant with the old and the new building. And gallery has walls that don’t go to the ceiling. We set up a network just for iPads alone. We took that, we put it back together. We’ve done everything. And actually if you haven’t been there in the last four months you’re going to ‑‑ come test it out because we’re pretty proud. Although there’s still no perfect answer there.
The last thing we’ve also been working on is looking at all the places we had to add new access points because none of them have walls that go to the ceiling.
But what about the iBeacon? Why don’t you just do that? So actually our CTO said, yeah, let’s do that because I don’t want to do this anymore. We called our developer and said let’s do iBeacon. We didn’t really do that. What we discovered, it’s not going to give the accuracy we currently have. It’s not really there. People do have an expectation of sort of GPS quality, and that’s not really what’s there with wayfinding right now. We do think that iBeacon would be great to replace our RIFD.
I don’t have my phone, but any device that we can do a RFID. It’s a sticker and every time you come to Gallery One, you can see objects that recognize your device and you can create your own tour to go into the galleries. That would be a way to get rid of the RFID piece and it works on Android as well. So we might be looking at that.
We’ve done a lot of preventative maintenance the tech team put in place. One thing we had come up with a better way of doing something was how to assess a problem.
Because we have applications going that talk to each other, we have devices with new operating systems. We have hardware that’s new, doing different things that it’s never done before. We have back end systems in-house, so we’re upgrading this year, and when something went wrong it was a mess of e‑mails from every department saying it’s not us. And then going in a circle.
[Laughter]
We came up with a message of how to assess a problem. And that actually we’re pretty proud of that. That could be a presentation in itself. Yes.
>> I was thinking like whose fault is it?
[Laughter]
It’s an annoying game to play.
>> Jane Alexander: It was always funny when one team realized it was on their side and then it circled around again. We also learned operating the space. We originally had a person dedicated to Gallery One on the tech side and we also have people from education, mostly in studio play, but in the main galleries that are volunteers. And the higher person will be dedicated to the family space.
The tech people have a lot to do. They have to make sure the systems are working every day. They have to change something out. Our goal is nothing can ever be down. We have screens up, it still looks like ‑‑ you can look at the prettiness of the wall. We have them helping visitors who have never used technologies, visitors that use it all the time. It’s been a space where we have lots of visitors who just ask questions about the process.
We realize this is a burnout position. So what we did was take the people from help desk and AV and even one person from visitor services who is super tech savvy. We have a team now, and they have worked together now, and they actually look forward to their days in Gallery One, and out of Gallery One.
[Laughter]
It’s sort of the first time that really the help desk is in the front of the house. And this is our ‑‑ actually, sorry. That’s our developer and our project manager learning how to use Gallery One. And that’s using our central table. They’re scanning it and going right to scan our work.
We also content development, the whole process of how we ‑‑ should we do it in big bulk, at different segments of time. So we set up back end systems and we’re working with interpretation and they’re even working with how they produce content, and after looking at some of the analytics I think a couple things will change in the future. That’s another presentation in itself.
One big thing is that all interactive ‑‑ all installations don’t work in interactive spaces. This is our famous ‑‑ that’s the sculpture lens where people are posing and ten feet away from the lens. People were walking into it and falling into it. So I said, well, before we have a death, you know ‑‑
[Laughter]
We all discussed ‑‑ but the problem was what can we do. We had this great acquisition that came in and we replaced it with the Wilson’s chandelier. It’s been a big success and people enjoy it. Although people do miss the Richard Long. It will find a new home in the museum. Also as you sign off, working with your vendor, they can’t wait for the day you sign off. You can’t do anything out of scope. We sign off and 6 months later you might find a problem. It might not be a bug. It might be something you can’t see until time has been used by people’s interaction. This is an example of someone would touch ‑‑ when you touch the screen you get Dutch paintings and then you have 12 different objects. Then you switch to a medium, you would have paintings. So on. 1865.
I would come down and people would visit in the morning. The contemporary artwork was always close first. And then the second work we also noticed. Well, those are the most favorited objects in the museum. But then I noticed the same sword was coming up. How could everybody be favoriting the same sword?
We called both projects and we said there’s a problem with the wall. No, it’s perfect. It’s not our fault.
[Laughter]
So we kind of forgot about it because a lot was going on. Every time we got to the wall it got more and more alike. We didn’t really notice it because we have 18 different departments. But the application team we looked at it one day, oh, it’s not randomly picking 12 objects. It’s putting up relevancy. So something had been favorited, it’s pulling it in first. Well, then more people are going to favorite it first. Guess what, the same will be on the wall at the same time. We all came up with a happy solution and thus it works perfectly again.
>> Keeli Shaw: We fixed it.
>> Jane Alexander: Everyone wants to know about our research. We were doing in‑house research. It’s not something going to be done until summer 2014. But Elizabeth Bolander is presenting. If you’re going to be there, she’ll talk more in depth. This is sort of our objectives, and we have had people ‑‑ it was from July, I think, through August we had observations.
We also have been looking at the Google analytics. I looked a lot at the Google analytics and local projects with the Google analytics and saw certain things as we were making changes to the interactives too.
All these initial findings are in the paper, and we looked at ‑‑ it’s depth of lenses we looked at.
But one of the things when we came to it, we realized the games that had a lot of interaction, people thought it was really cool but didn’t look at the objects as much. And then the lenses like globalism, people looked at all the objects because the games were about looking at the objects.
It didn’t mean that people liked them ‑‑ we saw the way they made the game was how they looked at it. There’s logic, and we will post this presentation so you can look at it closer. One thing we did notice. The lens was looking through and seeing art in front of them. We had people, not a lot, but enough that would say this is a cool object, where is it in the museum. And it’s like right there.
[Laughter]
So that makes us think, OK, we have to look at this. I’m going to quickly turn this over to Keeli who is going to talk about our phone and the analytics there. Thank you so much.
[Applause]
>> Keeli Shaw: I’m sure you might have some questions after this. But what we did, the projects with CMA is embark on a second phase to incorporate iPhone and Android after doing the iPad application. So how do we start this process?
Well, we decided we wanted to look at some of the analytics that came out of the iPad application first.
Knowing that we had a very tight schedule, and a limited budget, what changes could we make to accommodate some of the data that came out of that.
Our initial findings were people were being really drawn towards the object pages and the scanning. So try to privilege functionality that people were using the most, but it could to look and feel the same.
>> Jane Alexander: We added the search function and we added a ‑‑ how people use the phone ‑‑ you didn’t have to just use the Mac.
>> Keeli Shaw: Make sure you have analytics in there when you open and launch, and we didn’t in the first couple weeks so we missed out on very important data.
We assumed that the users that would be engaged with this would be onsite users, and we were right. Very, very, very right. So the average time that people spend onsite is about almost 20 minutes. Versus off site, 2 minutes, 30 seconds.
A lot of people scan. That’s one of the most used functionalities within the entire location. And no one uses social media.
[Laughter]
Nope. At all. This is total. Total times Facebook has been used is 615. Total.
So ‑‑
>> Jane Alexander: The iPad ‑‑
>> Keeli Shaw: Oh. So there are some ways to accommodate that. But at the same time it’s only 615 people at all.
>> What’s the total sample size?
>> Keeli Shaw: We had in terms of 14,000.
>> Do you track if people take the photos?
>> Keeli Shaw: They can share an object through an embedded share. So it has to go through the application to track it. So people could be even taking screen caps or photos of the artwork and then sharing it independently. But we couldn’t track it that way.
>> Jane Alexander: 9500 people have shared from the sculpture gate. People are sharing from the interactive games.
>> Keeli Shaw: OK. People are sharing. But the one thing to note is that with IOS7 you’ve got embedded functionality for share. So what we’re looking at now is does that help with social media. Just having it embedded in the IOS functionality. It might help, but at the end of the day, there is other modes that ‑‑ spend your money elsewhere. You know, focus on other ways of getting people engaged in sharing because they’re just going to take picture and share them on their own. But placing a premium on design and real estate for embedding social media, it just wasn’t worth it in the end.
Also, biggest question is how do you keep up with this constant versions and new device releases. The advice is don’t customize so much, you will create bugs that are harder to fix. Not to say you shouldn’t customize design to a degree that your brand isn’t represented, but don’t try to make exceptions to the rule constantly.
What we’re finding as new devices are coming out and new versions are released, the custom components of the ArtLens application are coming up with errors and bugs. Almost consistently the same ones every single time. So ‑‑ I can custom code that, but it will be harder to do, if they tell you that, maybe rethink that functionality and how important it is because you may have to fix it again and again and again over the next year and a half.
>> Jane Alexander: Gallery One is on schedule for summer of 2016 to coincide with our centennial. And these are some of the things we are thinking about.
>> Keeli Shaw: So let me stop.
[Laughter]
>> How much money does this cost?
>> Jane Finnis: This is a seriously expensive project, isn’t it?
>> Jane Alexander: So we were given a gift of $10 million for the project, but it’s the interactives and the hardware, we’re nowhere near that cost, but it was for the whole project and the space and other things that go through the building project.
But I don’t know if we share ‑‑ this is what I always say. I can do the number, but it’s not ‑‑
>> Jane Alexander: You’ve given it to us.
[Laughter]
>> Jane Alexander: I can say that we spent on integration, $1 million, but you are not going to be able to do all the interactives and everything we did for $1 million.
We had a set, a design to build contract. It was one of those projects that the vendors got sucked in and were excited about. We didn’t have any change orders and they just moved with it and kind of used something that was there. I give that number.
How much did it cost to make the wall. Well, the back end of the wall ‑‑ there’s a huge part of the wall that is all in‑house back end. So there’s no way to give a real number. It won’t give you any information.
On the technology integration content side, $1 million ‑‑ for an iPad.
That’s the painful hit. And this is ‑‑ in December, our two‑year warranty is up with our hardware. And they realized oh, no, we have to charge a lot more because it was a lot harder to do than what we thought.
>> Jane Finnis: Very quick. Two questions.
>> How much user testing did you do as you were developing these tools, and then I want to know if you had an outside contractor for the analytics element.
>> Jane Alexander: There was paper prototyping before we really got into what we were building in the previous group. But from 2010 on we did prototyping, but we were on two years from new concept, January 2011, to 12/12/12 was the soft opening. So it wasn’t really saying to say do visitors like that, more how does this work, or is there a button in the wrong place, does it work with the back end. And also people were building, so people could see what we were doing and get more involved and where it was going.
>> Keeli Shaw: There was limited user testing for the mobile application, because the schedule was so tight. It was let’s make sure this works and then we can go from there. It was a piecemeal user type.
>> Jane Alexander: The thing about not going too fast, we had to go with intuition and just keep moving. That was sort of a blessing. We also had the next gen with the iPhone to look at it better. Internally we’re doing an evaluation.
But I am going to suggest that I also think we should, with a project this big and being a test bed of so many things, it would be nice to get an outside view. Everybody in this project brings biases and their own stuff to it. And heartache.
[Laughter]
So.
>> Jane Finnis: Any questions? No. Thank you so much.
[Applause]
So while we do the laptop changeover, there are some more seats down the front, for those of you in the back. If you want to come and squish in, it might be more comfortable.
We’re welcoming now Robert Stein. What you don’t know about Bob, as a boy he was a child soprano. And performed Handel’s Messiah.
>> Robert Stein: Don’t worry, I won’t be singing.
[Laughter]
>> Robert Stein: Come on.
>> Jane Finnis: Things not working, we might need that. What you don’t know about Bruce, quite a few things you probably don’t know about Bruce. He told me a couple so I’m going to share a few with you. He had three offers of marriage from weddings in Kenya in 1997. Actually from chiefs trying to marry them into their tribes.
>> And it still happens.
>> Jane Finnis: He also was an emergency veterinary technician for six years. So if any of you have any pets with problems. But off you go guys.
>> Robert Stein: Thanks, Jane. Good afternoon, I guess it is.
>> I’m Rob, I’m from Dallas where I’m the deputy director, Bruce is principal, at USD Design and Mock Consulting. And Bruce and our team at the Dallas Museum of Art work closely together to create a project called Friends. So some of you may have heard us give a talk on it last year. So we’re trying not to rehash all of that, but we will give you at least a little bit of coverage.
I do want to set the plate a bit first to make sure we’re all on the same page. How many of you in the room are working on projects with measurement and metrics?
Raise your hands. OK. So the question we started with was this one. How do you measure things that you can’t see. And for many of us, in museums, we’re critically worried about the impacts that’s our museums are having on our audiences. For us, in art museums, many times that impact is purely internal. It’s a very personal experience. Sometimes it’s all inside your head.
Sometimes there are impacts that happen months or years later, and you may or may not ever know about them. So how do you, as a museum, figure out if you’re doing a good job. That’s sort of the critical question and when we started.
I come from a technology background and Bruce from many, technology among them. But we think about systems. You think about the museum as a system and in the museum, if you take the whole thing, the people, the building, the collections, the tech, is it possible to build a platform where you could get better at engaging your users over time. And that would help your museum actually know that it was happening and improve it.
Could we start ‑‑ and I think this is a lot about what we talk about here at this conference. Could data be used by the whole of the staff, not only to prove that these things were happening, but also to get better at it over time. Because in fact we know that we are engaging users because they tell us stories about how they have connected with our museum every day. I’m certain that every one of you in the room has a story that you could tell me about how somebody has really benefited from visiting your museum. There’s all sorts of things we do that don’t have that impact. So how do we know what they are. How do we know the ones that are really succeeding and then how do we pick apart what was it about that successful program that actually made the difference so we could apply that learning to something that’s not as successful.
That’s what we’re going after. So let me tell you a little bit about the ‑‑ seizing. That was a joke folks.
>> Bruce Wyman: Laugh with us.
>> Robert Stein: Let me tell you a bit about the Dallas Museum of Art. So Dallas and North Texas in general are really blessed with an awful lot of great museums. The DMA is among them. It stands as the largest art museum in the region. The only cyclopedic museum in the region. It’s really in the center of downtown and it’s 110 years old.
The building is owned by the city. We’re very much the city’s museum. We receive a lot of benefit from that, not just from tax incentives, but from private places and from name recognition, Dallas Museum of Art. That’s our name.
Dallas, if you’re not aware, is not a tourist destination. Unless you’re a Cowboys or a Rangers fan.
[Laughter]
But we are not drawing lots and lots of cultural tourism. Our primary audience is in fact people who live within driving distance. In Dallas, that totals about 6.5 million people.
Last year the Census Bureau told us that Dallas‑Fort Worth metroplex was the fastest growing metro in all of the United States. It’s growing by somewhere between 1,000 and 1200 people every day.
Our ten‑year average attendance in the museum was 485,000 people. And remained flat for ten years.
We had tremendous investments in some very large exhibitions that would spike us up for one year and the very next year we would be back to 485,000. So while the city is booming, the museum is flat. 8%. We have no more, we are reaching no more than 8% of our possible audience.
And Dallas is growing and we’re staying flat. We’re shrinking and we’re losing this battle. We decided to try and get under the hood. I want to issue a caveat here. The things we’re going to talk to you about today, we are still learning.
This is not the end of the story and we designed the entire system, the entire approach was a system for learning ourselves. So we are learning about what the right metics are for us. They may or may not be right for you, but what are the right metics for us. And how good are we doing, how can we capture them, all of these things.
So we decided on four to start with. One, we decided that repeat visitors are more important than lot attendance. It’s really easy to get somebody in the door once and relatively more difficult to get them in the door twice. More difficult than that to get them in the door three times.
In fact, if you really want to change this number, what you need is a set of passionate people who see your museum as their home. So we think repeat visits is really important. We are measuring ourself against the diversity of participation. So it’s not enough that somebody only shows up for the party we throw on Fridays, but we also like them to visit the permanent collection. We want to see that they go to the education spaces. We want to know that they sometimes attend lectures. So that visit diversity is something that we’re looking at. But think about that for a second. That means that you have to know a person as a person. As an individual, not as a group. Not as a cluster, not as a segment.
If we’re doing a good job, we ought to increase the affinity that our visitors feel for us over time. They ought to like us better and they ought to like art more, if we’re doing a good job. We think that’s pretty important. We ought to be able to issue sort of calls and responses. If we have a platform that works, we ought to be able to say, hey, let’s design it to do this thing and see that it actually does that thing.
So that’s what I mean, ability to motivate action. I’m going to hand‑off to Bruce and Bruce is going to talk a little bit about what we did.
>> Bruce Wyman: I have to admit there’s a bit of personal experimentation I want to do for a moment. Is see how quickly I can talk and the prompter can keep up. I’m fascinated watching the words on the screen.
Let’s talk about a year ago. The picture is relevant. That’s the Titanic. Everybody was hopeful at the start that it was a good project. Clearly didn’t end in a good way. We think we’re in a better spot than that. So that’s our baseline of standard, we’re feeling pretty good about where we are.
The core of it was we needed a bunch of different things to actually come into play throughout the museum. Part of it was changing the overall visitor experience, what people begin to encounter in the museum. But at the core of it.
In addition to process and changing the way the museum works there is some technology that everybody interfaces with and you interact with when you come into the museum. There’s a series of iPads. You sign up and you get acclimated to the system and learn about the day and the general program and what we have to offer. Off-the-shelf hardware. Probably the best mobile platform in existence right now. We’ve chosen it as an interface because it’s a self-contained, all-in-one experience. There’s a camera inside, Wi‑Fi capability. In terms of trying to develop other infrastructure you can’t get cheaper and much more reliable than this.
So once you start to join Dynamic Friends, there’s a basic dashboard. The notion behind DMA Friends is transactional relationships with the museum. By that, we look at the different people who come into the museum and what their motivations were to participate and engage with the museum. Two different categories, those motivated by giving and those motivated by I want a membership because I want a discount or get additional people in.
So a lot is geared towards that basic transactional motivation. We break it down to give you activities that you do with the museum. As you engage with the museum you can check in with DMA Friends and you get credit for doing that. It’s essentially a basic loyalty system.
You want to make sure you incentivize the right things. It’s not we want to give them a lot of points for doing stuff but they engage with the museum.
The fatal flaw with the airline industry loyalty 30 years ago is you get points for flying a lot of miles. Which is not what they want. They want you to spend money. Now when you spend more money with the airlines, that’s how you earn more. So they don’t want you to fly lots of miles, they actually want to buy the participation you’re going to do.
When you log in there’s a basic set of background information, the things you can do at the museum and the things you can check in at. There’s a wide variety. We customize them fairly extensively and the bigger part of that is working with the organization to figure out what are the kind of experiences that we can incentivize and recognize that people are going to want to do.
The badges that you may receive as a part of it are more a biproduct of what you’ve done. It’s a tricky balance there but important because you want the motivation to be on the experience, not the direct benefit you get from that.
Any of these badges can be done in a number of different ways. Multiple stops. We can provide things in different quantities and we can look at multiple steps. And we tried to evolve a system that meets all the potential scenarios we could ever dream of and for the most part we seem to have done well. After the first year there were some basic modifications we need to do. But this has been sustainable and as we think about figuring out to make it grow it seems to work out well.
We deliberately made it a cell phone integration. You can use a smartphone and use a mobile interface, but it ended up that texting ‑‑ you realize there’s a large population that have old phones. Those people will all die and we’ll have a new generation.
[Laughter]
But in the meantime we still want to accommodate. Texting codes to a server actually works very well and it’s very reliable and sustains well.
A basic architecture and we used to have a much more complex diagram. At the core of it DMA Friends is a ‑‑ we’ve worked to develop a BadgeOS which sits on top of it. So essentially DMA Friends runs through BadgeOS. Which is a huge. They integrate with the mobile devices you have there.
All the experience is on a responsively designed site. Once we develop it for an iPad or a mobile device we can apply it across a variety of different things. We had originally this complex scenario trying to get multiple systems to talk to each other is brutally different and hard. We went back to an older technology where we printed coupons. It’s easy for somebody to say, oh, you get a discount and I can punch it into the register. But the key bit of integration there, when you’re at a DMA Friends kiosk there’s a printer that will print out coupons.
One of the benefits ‑‑ you could customize the things you want, choose the benefits.
It ends up being relatively simple and straightforward. As with most presentations MSQL back end. It’s stored in a large container and that’s allowed us to do a lot of custom analytics. We’ve been chart ‑‑ there’s a nominal fee that takes complex data set.
The bonus of having a good MSQL architecture, we can do different analysis tools.
All of these things end up being plug‑in components that we can flex back and forth. So launch day. A little over a year.
>> They missed that as the Titanic launch.
>> Robert Stein: We’ve completed our main voyage as it were. We’ve been running for a little bit over a year. We launched in January 21 of 2013.
I’m going to speed up a bit and throw a lot of data at you. I apologize for that in advance. The data is all in the paper, which is online. It’s also on our website and you can download it and play with it if you want to.
So we’re going to get into this. So this is a look at the back end chart interface. This is what is up at the front desk and every visitor, services staff member keeps track of this almost every five minutes. They’re watching this. Part of what we’re doing is training the staff how to integrate data and decision‑making in their own time. As of this morning 58,407 friends. These are people who would physically visit and be greeted by a person at the museum and agree to sign up and participate in this. In a little more than a year 58,000 folks.
That’s an annual attendance of about 550,000 people and our prior membership program was 22,000 people. We can track all sorts of things about friends participation, when are they coming, how are they coming back. We count friends visit, which is completely optional. You can come to the museum, you either choose or don’t choose to tell us that you’re here, and the very choice of telling us that you’re here is a factor of engagement.
If we’re doing a good enough job offering you value and make your life easy, then you’ll tell us that you’re here. We’ve had about 84,000 of those visits since launching, 40,000 since October 1. The graph below, you can see the blue lines are signups of new friends, the yellow lines are all repeat visits of friends who are coming back to do things.
So the repeat visits are smaller than the new friends signups. But in an aggregate basis for all friends across the nation 10‑11% repeat their visit. On a daily basis, that number is between 35‑50%. The difference there is some people are visiting an awful lot. You can see that we track the kinds of rewards we give away. Free parking is our most popular reward.
[Laughter]
That still works out good for us. There’s no tangible cost for us to give you the ability to park in our garage. There’s an opportunity cost, perhaps. We are tracking each and every activity that an individual does. So who does it, where were they when they did it, what time was it, who were they with. Again, this is all completely opt‑in. The most popular activities of course, visiting the museum.
We teach people how to read accession numbers on our object labels and they can text us the accession number to reference works of art they love.
Bringing other people with them. We’re trying to incentivize that behavior, so bringing other people with them is a really frequent behavior, as are, a bit to our surprise, visits to the permanent galleries. By designing badges that have creative ways of packaging those experiences together, like taking a trip around the world, we’re incentivizing people to travel to galleries they wouldn’t go to. They also understand. There’s no trick. People get it immediately.
We have a ‑‑ a lot of people are pitching this as a big data problem. And it is big data for museums. But not really big data in the corporate sense of big data. For us, we’re generating about a million rows of user activity on 60,000 users. User profile fields have somewhere in the range of 20 or so fields there.
And then we have a system log that somewhat overlaps that activity stream of another 1 million rows. So it’s not large in the sense of tetabytes. But the series of queries you want to do at this scale gets complicated.
So what are we looking at. The URL at the top here. This is where you can play with these graphs, and you can get into some of the underlying data. These are the friend recruiting trends. The blue lines are the weekly numbers of recruits, which have hovered right around 1,000 people a week.
Then the green line is our running total. You can see the slope on that green light is remarkably steady. It’s not tailing off and it didn’t start much faster and then slow down, so that slope of the line is really important to us.
This is the same data looked at a little differently. The black bars are a 12 week rolling average. So you can see in the springtime, right in here, we started to change the way we were pitching the program to visors to the front door. This is when our front‑house staff and our gallery attendance really came into the ownership of the project as a way to engage audiences. They really began to own it.
You can actually see the result in the increase of that rolling average. So they figured ‑‑ we started here. And that average dipped a little bit. We kind of fiddled around to figure out what the right way to work was and then they figured it out. So there’s a lot of increase. Over, I guess this was fall, they kept working their system and then just recently we’ve experimented with a few different kinds of pairings.
Because we really wanted to try to drive repeat visits. So we started to watch for friends coming back as repeats. Funny enough, this also actually impacted our new recruits, which was unanticipated. But you can see it from the data.
This is the multi‑colored green bar charts.
[Laughter]
And the only thing to say here is that we’re tracking each and every activity that happens on each and every day. What you might see is some big lumps that show up in here. This is when we finally figured out how to start promoting the level work of art thing. I saw you cringe when we told people how to read accession numbers on object labels. It was hard to figure out how that works. You can see from the data we got it right in the fall. That’s when we saw an explosion of people texting about works of art.
This is a graph of reward redemption. This is Bruce’s favorite one here. Notice the blue bars. This blue bar, we give you free exhibition tickets if you earn a certain number of points. So you can come in and see the show for free, which would normally cost you between $8 and $16. What we did in the winter, in December of this year is we were having some exhibition attendance that was below what we wanted it to be. So we called out to these 60,000 DMA Friends and said, hey, friends, guess what? If you’ve got 1,000 points you can come see this exhibition for free. And they did.
This was an example of that call and response. We created an occasion for them to do something. We rewarded the people who were participating with us the most and we created a bounded time with them. So it was a three‑week only opportunity. You can see in three weeks that call and response happened in a dramatic way.
>> Bruce Wyman: The reason I’m in love with that actually is we never knew it was going to be watch ‑‑ we knew it was going to be aware of what people did in a way people never did before. It’s not enough to watch. You have to influence behavior. We’re able to see so directly that we can do some simple and subtle things and that people respond en masse and enthusiastically and willingly. This is a great tool to do things with visitors. I love this graph.
>> Robert Stein: This is a look at every single friend who has visited more than once since the launch of the program. What you can’t see, that the access on this end tops out at 200 visits. The bottom here, this is 20 visits. Can you imagine enabling a new member of your audience who will come back 20 times in one year? That’s a lot, right? Five times is a lot. Let alone these “superfriends” who are up here at the 150‑200 range. Those are incredibly passionate users and you ought to know them individually, if not by name.
One way we’re using this data. We asked you optionally to give us your ZIP code. In addition to place, we get great joins against census data for a rough guess of demographic splits and something as simple as commute time. Commute time is a significant factor to consider why or why not people may be visiting your museum. So this is the population of friends against the population of ZIP code. In this graph, gray is good. Gray is on even split on those things. A zone that’s green tips the scale where we have more friends than the population says we should and red says we have proportionately less friends than the population says we should.
Fort Worth is in the west. These red zones here are all red. You might guess this is because Dallas and Fort Worth don’t like each other. I like Fort Worth. There are three or four art museums in Fort Worth that are closer than the Dallas Museum of Art. So that’s going to impact that graph. We’re using it to look at zones like this one. This is close to where I live, but it’s an area called Frisco. It’s a booming area. We’re not doing as good a job drawing people from Frisco to come to the art museum. We don’t know why, but we’re looking into it.
This is Bobby Coffman, we thought Bobby was cheating us. And we were wrong. Bobby quickly sailed to the top of the list. He was earning zillions of points. He was our first really high reward earner. So instead of accusing him of cheating, which would have been a bad idea.
[Laughter]
We invited him over for lunch and said, hey, can you come by, we’ll buy you lunch, and we want to know how you’re using friends. He’s a high school English teacher and the museum was on his commute home. So instead of sitting in traffic, he would pop a quarter into the meter and zip into the galleries for 30‑45 minutes. He was using this as what he called his daily dose of art. That’s exactly what we want to have happen and that’s the way we want people to think about using the museum.
And that’s Bob.
[Laughter]
No joke. That’s Bobby and this is what he does when he comes to our museum. That same chart. So we can see there’s some variation there. He’s not actually gaming us, he just has things he likes to do.
Another great example. We try to do what we call boutique rewards. And this is actually a boutique. This woman’s husband was an avid friend and used his points to buy his wife a present where she could do the art beauty shop rewards. We have this painting behind them which is a 1930s beauty shop. So the reward, come with five of your friends and have your hair and makeup done in the style of the painting. And then have your portrait painted by our photographer. That didn’t cost us very much, right?
It only, maybe, a couple hundred dollars. But a super‑great opportunity, not just for these people but all the other people who talked about this. And now I’m telling all of you about it.
So that was well worth the $200.
>> Bruce Wyman: We’re looking at the forward facing element of what are we going to do. We have an interesting problem as we go through the data set and collected real stuff. It’s not actually a big data problem. We have some of the problems that exist there. Like any concern, we have scaling issues. At this point we’re looking at how do we make WordPress faster. How can we actually streamline that to make it quicker and better with the amount of data we know we’re amassing because we see it on a daily basis. You see this low grind of things slowing down in a subtle way and we need to be proactive to change that. We’re going through an overhaul of the back end mechanics of how the system is architected.
I’m going to keep going. That’s good. So the question really becomes how do we make this museum more effective at producing our engaging and invested audiences and we need to figure out what it means to do that in Dallas, but also elsewhere. We know the perimeters of Dallas. We know how to leverage that. The question is what’s that look like in other parts of the county. Everyone is its own special unique flower that blooms in its own magical way.
[Laughter]
Everyone has their own individual scenarios that people want to do a membership in the way they want to continue to do. Some have different demographics that need to respond in different ways. As we fill this out, we have to ask ourselves, what’s it do in other parts of the country. Is it a model that is sustainable and repeatable and makes sense everywhere else. We have partnered with our other museums to build pilot studies with very different needs and requirements. There’s the Grace Museum in Texas, Minneapolis Institute of Art in Minneapolis, and one in Los Angeles, and then the Denver Art Museum, which is DAM versus DMA, so it’s easy to confuse the two in conversation.
But over the next year, and we’ll be developing pilot projects, they’ll launch in the fall. We’re going through the process of having identified different use cases. It’s interesting because nobody wants to do the thing that we did except for the Grace. At the core of it because we understand how the system works we’re looking at things how do you make the museum a digital concierge. How can you make recommendations to people based on what they’re looking at.
People participate enthusiastically and they give you that data. We have given you good incentive to give us that data.
The most important outcome, we alluded to this earlier, is when we need a strong technology platform and the only reason that DMA Friends is successful is because of staff participation and the fundamental changes that change the way the organization will work and the way they thought about visitors. But to that end.
>> Robert Stein: I’m going to shorten this for sake of time. If everything else breaks and goes away, if we throw away all the software and the tech, at the end of the day we get a staff who understand how to build a process where they can learn to do their job better, then that was worth however much efforts and time we put into it.
So I want you to take away, this particular approach and this particular software implementation isn’t the important part. Building a structured museum where your staff can learn, that’s way more important.
So if you’re interested, the code, you can have it, and you can do this yourself. We would like you to do it. If you do, we’d like to invite you to aggregate data together with us so that we could get smarter from each other. I want to give you my data, the things that are working in Dallas may work for you. You may discover something I’ve never thought of and I really want to use that. If you do this, please tell us about it. So thanks a lot.
[Applause]
>> Jane Finnis: Thank you guys. Any quick questions? Go for it.
>> The question was did the genesis of this program coincide with our free admission switch and it did. So we were making a move to open the museum up, we dropped our admission charge to the permanent galleries and we said we want this to be ‑‑ if you want to be here, we want you. And then we said if you want to belong here, please join us as a member for free.
>> But you don’t have to?
>> Robert Stein: You’re not required to. There’s no date. You can walk in anytime you want. People won’t hassle you. And participation is entirely optional.
>> Jane Finnis: In the middle there.
>> Robert Stein: Yes, right there.
>> How many admission ‑‑ anyway admin ‑‑ any admission underwriting?
>> Robert Stein: We did not have underwriting to start. We decided with our board it was important to do. It sacrificed $1.2 million of earned revenue, and we decided it was worth foregoing that money to do this. Since that time we’ve been awarded about probably $10.5 million in grants and donations specifically connected to either free admission or this friends program. So you can look at it that way, it’s generating funding revenue.
To be respectful to our other presenters, I’ll make myself available outside of this room afterwards if anybody wants to keep talking. Thanks.
>> Jane Finnis: Next up we’ve got Lindsey Green who is here from Frankly, Green & Webb. And Lindsey used to serve pies in the [Inaudible] Club in the UK.
[Applause]
And actually dropped on the back of the goal keeper’s head and burned his head. For the match. She’s here with her colleague Laura Mann, who spent the night at Alcatraz. You’re going to have to find out why by asking her. Lindsey, over to you.
>> Lindsey Green: Thank you very much. This is a different topic to the presentations that have come before, but that kind of theme of decision‑making is very cool. Yes, Laura and I from Frankly, Green & Webb. We work for organizations to help design and improve their digital experiences. We particularly specialize in mobile content. And this is the type of paper. “Listening to Visitors: Research Findings on Mobile Content.”
First a little bit of a confession. This session isn’t about mobile. It’s audio guides. We use the title because I think we started to brand audio guides as mobile land.
This is very much a talk about audio guides. And the reason we kind of have to start reframing audio guides to mobile is because I think audio guides kind of started to be seen as the problem child. They’re a bit of a pain and they can be difficult to manage and tend to have a reputation to be quite expensive.
And that’s kind of how I was seeing it when I was putting this presentation together. Our colleague Alyson who is not here today, no, no, no. They’re not a problem child, they’re an embarrassing parent.
[Laughter]
This is my dad.
[Laughter]
And this is my wedding where he stood up in front of all my friends and I disappeared behind the table and shared to her what he was going to tell them. He’s wearing fake eyebrows because they thought they made him look like George Clooney. So I’m going to come back to him very shortly.
The other thing you should know is I’m going to be talking about audio guide content. Because I’m talking about content, this is the ubiquitous content slide. That term is starting to grind me down.
I’ve heard it so many times really over the past few years that it no longer means anything. As I was kind of putting this talk together, I started to think about what content is supposed to be about. It’s supposed to be about respect content, making content work hard. And hardworking content does mean valuable content. What we’ve turned to look at making valuable content is low cost and distributing it across multiple platforms.
Here’s kind of the problem child, or the embarrassing parent, has a few issues to deal with. Because it’s quite difficult to transfer audio guide content to mobile platforms. That’s kind of a provocative statement to make, but I truly believe it and the research we’ve been doing is showing this more and more.
Here’s why I think there’s a problem transferring audio guide content onto multiple platforms. It’s really about if you think about it in terms of a movie, if you watch a film the audience is this yellow point here. The weird physics symbol, that’s about all your story behind the screen is being told ‑‑ there’s objects around, moving the content. You can change its size, the location of the screen, and the shape of it.
That content is not going to lose much meaning by putting it somewhere else. And here’s the same kind of thought in terms of audio guides. Audio guides, the experience is by putting ‑‑ moving them around the story. Moving them around ‑‑ it’s not really that easily transferable because the content has to be designed to go intimately with the physical space.
It’s an incredibly unique experience. It doesn’t really happen in other places apart from the museum now. That could change. But I think it’s something to remember that that’s actually quite unique, valuable thing that we do.
So that kind of embarrassing parent, it turns out they’re not that great at being low cost and distributing on multiple platforms has been a real issue too. But what if we look at having value in content, impact. What about if we’re actually talking about big impact, making big impact on engagement and understanding.
So this kind of brings us to where we were, which is working with the National Gallery in 2012 and 2013, working with Charlotte Sexton, the director of digital media, she wanted to understand whether or not audio guides were delivering on impact and whether or not they had to refresh that 40 hours of content they had access to.
40 hours of content, refreshing it? We’re not like wiping its face and kind of shoving it out the door. That’s a massive effort you have to put into doing it.
So a bit of a back story about 40 hours of content. It kind of came from one of those instructions of a new technology where the sky was the limit on the amount of content you could put on a CD rom. And that was said to be, well, you’ll be able to do as much as you wanted.
Some of the content was produced quick to get it out the door. They used any narrator they could get ahold of really. They kept it low cost. They didn’t have interpretations and they didn’t have testing galleries.
There was also a big bunch of new content that kind of had an editorial team attached to it and it actually sat about what they wanted the experiments to be.
We wondered if they could really spot the difference. Charlotte wanted to understand the value of the guide to this experience. She needed to bring people on board to help her deliver it and she wanted to be able to say whether or not it was working, whether or not to put money into the value of this actual experience. Was this embarrassing parent really embarrassing?
We started evaluating the content, but we knew that Charlotte would need to have content design principles because she was going to have to articulate to people about what made great quality visitor experiences through their content. We thought there was definitely an opportunity to extend the do audio guides work, but we wanted to understand the principles of what makes that audio content work.
We had to recruit visitors before they were visitors because we were going to ask for an hour and a half of their time. Hijacking a tourist saying we’re going to take over your experience for two hours, that wasn’t an option. We chose specific content that related to the styles and techniques we wanted to test and what was important.
We developed a series of strategic questions. So they had a little sheet that they went around and they could fill in boxes to capture what their gut instinct was. And finally we interviewed them about the overall experience.
We recruited them by Facebook. One side of that is we ended up with really passionate National Gallery fans, but also with a huge group of people who had never actually taken an audio guide. Some of them were taking a few, but nothing in the National Gallery.
So the question was do audio guides work. As part of the study we actually just asked them what they felt their impact was. We asked them what the guide had led them to do. Was it less or the same.
It was recorded as having led to as having enjoyed their visit more. Three quarters of participants reported that they learned more and they discovered about art more. That’s kind of core learning objects for the National Gallery.
These are kind of some of the questions ‑‑ everything you see in speech bubbles is from quotes from the actual audience. We heard from visitors that the experience of listening to audio helped them apply ‑‑ it increased their skills, it increased their amount of looking. They actually knew that was happening and they increased their confidence to look in between different objects.
What’s interesting is they were able to apply that knowledge to other works of art. They felt that they owned that. What was most interesting for me was this kind of visitor really enjoyed listening to audio in front of objects.
People who meditate from audio guides are like this is a brilliant experience! Why didn’t you tell me this was available? It’s only in the front desk as you walk through the door.
One of the big realizations with me, just like I realized with my embarrassing dad, as he stood up, he told an amazing story and he did an amazing speech. And I hadn’t realized he was fantastic, even though he was wearing fake eyebrows. So one of those kind of, as you become that embarrassing parent, you learn the value of this skill set that this person has and qualities nobody else has, that’s something that we can use those audio guides for, looking and using audio.
So what are the principles that make great content for visitors? There was a huge body of literature on audio guides available in lots of different locations, but it’s this surprising small. Small amounts of data when it comes to content design.
The overall insight was that visitors appreciated quality. But this is just ‑‑ when they talk about content quality they could spot stuff that didn’t work. Stuff that could be created quickly, cheaply, hadn’t been thought about in that experience, they could smell it.
It was really, really apparent to them. But they could also see qualities in the elements of the old stuff that had been done, but also, there was stuff that they loved.
I don’t think anybody sets off to go I’m going to create some major quality content or some really bad content, that’s my objective today.
[Laughter]
But it was interesting, a voiceover content. It turns out that visitors in terms of the quality experience, the design that they’re looking for.
So this is where it gets ‑‑ I realized by talking about audio guide content sometimes we all have different visions in our head of what that means. I want to give you a quick sample of the content that kind of followed the quality principles from the visitors.
Somebody talking about testing. I actually haven’t tested if this works yet. But I’m going to give it a go. So this is the Mars/Venus. And this is the stop from the National Gallery, grand tour.
If you’re close to the speaker you can really hear it. This is what good quality sounds like from a microphone.
>> With kind of serene.
>> She’s ‑‑ it represents the victory of love over war. The fact that he’s being left — to the extent that somebody could even blow a shell horn into his ear and he doesn’t wake up. Half human represents a sexual side. The play with his enormous pipe and fiddle around with his armor. It was probably commissioned to decorate someone’s bedroom. One of the reasons was because she’s very beautiful and so is he. And one of the ideas that people had was if you looked at something that was really beautiful ‑‑ what makes this picture remarkable, brilliant, is making a picture which was very, very serious and deeply profound moral content, but at the same time a way that people could laugh.
>> Lindsey Green: So from that clip, I want you to keep that in mind when I’m describing what I’m talking about in looking at how those principles have been applied in some of those areas. These are some content design principles from the visitor. Number 1, information is important, but meaning is more important.
This is not a median that’s about information. Totally, it’s kind of more about the idea of understanding what is actually ‑‑ what we’re actually looking at. Getting that context, finding out opinions, provoking and challenging. When we think about delivering information, one of the challenges we face is that we see audio guides as a mechanism for bringing information, we totally frame what that experience is meant to be about for that visitor. And they were quite clear about where that kind of meaning came from and where they saw value in it.
The second principle is about choosing a style of content that matches your resources and skills. Visitors don’t have a preference for a particular style.
We kind of expected them to say, I love interviews, or they would say I really didn’t like interviews because it was too much opinion. That’s actually not what we saw. We saw they didn’t have a preference for particular styles. They liked interviews, they liked debated conversation. In fact what they did say was we like the mixed approach that we take by calling this research together. Whatever the style of production, if the production is done badly what they recognize was that if the narrator phoned in, the interview felt a bit forced, the production becomes a real distraction and it sits in front of the actual content.
Principle number 3 is that respect is a really personal intimate experience. Audio is a personal intimate experience by definition. But you expect some level of social norms.
Things like introducing the person speaking, why should I listen to you, why are you important. Addressing their needs first so we can get some ‑‑ if it’s a narrative‑based painting, they want to understand the story behind the painting. That was the only question they had. If you tried to give them a huge beautiful soundscape, if you’re thinking about what the painting actually shows me. You couldn’t get past that.
This doesn’t sound like someone who would work with part of the National Gallery. Actually we focused very much on what our visual branding looks like. Very few organizations have an idea of what their voice would sound like.
Principle number 4 is make time in front of the object count. Visitors make perception of time to how engaging and entertaining content was. They could listen to a 1-and-a-half minute stop that was entertaining and they would be left wanting more. They could also listen to a stop of 1-and-a-half minutes of boring content and all they wanted it to do was end.
So something that was actually too short was totally complement. What it doesn’t actually mean is ‑‑ give you a little bit of —
connect the experience to the space. One of the things that is about the idea of you creating an element of experience within an intimate ‑‑ relates to the design of the space, create something that connects with what they’re feeling.
There are certain differences in how you direct somebody when you stood in front of them versus when you’re writing stuff down at the desk. So actually one of the ways to look at it is how can the processes reflect that experience. Look at drafting content actually inside the gallery through the curator through the audio. What would the person say to you if they actually stood in front of that work.
These are the audio guide design principles that drove the research. I don’t think we should look at them as a set of commandments. Just like any principles, they should be broken, but intelligently knowing what compromises that you’re making, knowing what the impact on your overall experience would be.
Because if we can look at those principles actually you can learn to answer some of these questions on how hardworking content ‑‑ is it delivering big impact.
But more importantly, we can start to answer questions about, OK, we need to make this content more low cost and give it to a larger audience and we can start thinking about what those compromises are that we’re taking. One of the statements that we see across the board, and I have to say, the next bit of research we do, I’m definitely going to try to nail whatever this means. A lot of people say I want to go at my own pace. I’m not entirely sure about what that actually means. What we’ve attempted to do in the past, turn what a linear experience was into an access tour. They want to access the content in their own order, but actually when you do that, you compromise a principal that is actually quite important to them, which is make time in front of this object count.
So what you tend to do when you make things random access, you don’t know which order they’re listed in. You have to tell them a bit of background information where they go. Background information has not been kind of the core of what they’re wanting. It’s not the meat.
So what if we look differently and said, actually what they’re saying is I don’t want to be stuck in front of an object for what seems like a long time.
That’s that principle which is make time with the object count. So actually, then it becomes really about selling this engagement because that’s the most important thing for them. You’re going to enjoy deeper engagement.
Or what about if we take a technical issue. So a lot of people are looking at bring your own devices for very good reason. But actually what we see is a lot of visitors don’t have their headphones with them so they don’t access audio content. So we’ve seen some choices made which is about delivering content via text. That’s certainly a reduction in the amount of looking at the very core of what is an audio guide experience, what makes it important in terms of meaning has been lost.
I don’t think it’s a major thing that you can’t describe content by text, but you’ve got to understand the impact it’s going to have on the visitor and adjust the content to fit that need or think about how you’re going to transfer meaning or really ask yourself is bring your own device the medium of delivering audio guides. That’s a whole different presentation.
But also understanding the content principles. Behind one of those areas where we realize we have a real opportunity. We’re interested in the service design of an audio guide.
What it equates when you get this experience actually delivered. Why didn’t you tell me about this experience? And actually what we tend to do is when you push your time because you’re getting the product out the door, you go, oh, yeah, we need to do marketing. OK. Download the app. Yeah, download the app. Or would you like an audio guide? How many people hear that as they walk into an exhibition. Have you tried an iPad touch tour?
The technology is not there that’s driving the experience, it’s the content ‑‑ delivering the experience. I have to say these are stolen from the people in research, the way they describe an experience if you’re looking for marketing asking people how they ‑‑ they need to feel what’s ‑‑ it’s like someone said, I finally know what it feels like to be inside the painting. I was totally surprised by what you can see.
I think all of those, looking at how we market that experience is much more important.
So what are our take-aways? Use audio guides for what they do well.  You know what you can do and you can use them at the right moment with the right budget at the right time. Focus on delivering meaning over information.
The core of what an audio guide was is a meaning ‑‑ it’s not ‑‑
Understand what makes your content valuable to the experience of the visitor. And that process. Start mobile and stay mobile. As much as you can do in the gallery or in front of the artwork with a human being. That’s the difference. If you’re looking to prototype, prototype the content as much as you do with the user experience design.
[Applause]
>> Jane Finnis: Just one or two minutes. Any quick questions for Lindsey? No? OK. Well, could we just say thank you in the usual way to all of the fantastic speakers? Thank you.
We are now ready for lunch. You have an hour and a half. Be back at 1:30 to see two more sessions. And enjoy. These guys will be around if anybody wants to grab them for more questions.
[Session concluded at 12:02 p.m.]