CART Transcript for How-to Session 4

Friday, April 4, 2014 1:30 p.m. – 2:32 p.m.


Held at:

Renaissance Baltimore Harborplace Hotel

202 East Pratt Street

Baltimore, MD

Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings. 

>> Darren Milligan:   Good afternoon, everybody. I think we’re going to go ahead and get started.

Thanks for coming to the session this afternoon. This is Connecting Learners and Museums Through Education Metadata Initiatives. My name is Darren Milligan, the Senior Digital Strategist at the Smithsonian Institution called the Center for Learning & Digital Access. I’m joined with my colleagues, Melissa Wadman, Manager of Program Evaluation, and James Collins, the Digital Media Project Manager.

I wanted to start this morning with a little introduction, someone that many of you might know, Richard Culatta, the Director of the Office of Educational Technology from the U.S. Department of Education. He gave a talk a little while ago that I thought was a good introduction, some of the things we’re going to talk about today. He identified some specific problems in the structure of traditional education that technology is uniquely positioned to address. These are things like the treatment of all learners with a similar style or similar pedagogy, homogenous scheduling, things like that that are widely identified issues with traditional classroom education and things that technology perhaps has an opportunity to address and have some impact.

I think it’s important to sort of put some things in context. This is a how‑to session. We want to keep things informal and open. If people have questions or need us to slow down or pick up the pace. I think it’s important to make sure we have some context for the way that digital technology can address issues and the reason that these ‑‑ couple of things we’re going to talk about today exist.

Technology can really have an impact on things, especially in things that are connected to museums and museum resources so technology is in a good position to radically improve access to learning resources. These resources are essential components to enablizing personalized learning which has become the buzz word in education, but something that people really are looking into and something that we feel has some potential to have some impact.

We’re going to be talking about two promising strategies that help address that listening specifically. A little overview of what we’re going to do. We’re going to do kind of talk at you for the first half today, first 30 minutes or so, and take a look at some of these things here. The second half we’re going to actually walk through developing some metadata for some educational resources. We picked one we have. If you haven’t come with one in mind, maybe be thinking about an educational resource that’s available on your museum site and we’ll walk through the process for looking at how to address metadata for that in the second half.

A couple of slides here with some data. What they really show are that it really more than ever, which is probably not a surprise, teachers are using the internet to inform and support and enhance and really enable their teaching. So this is some data from a Pew study done last year. Really demonstrating that clearly teachers are not any different than most of us. They’re using internet to find resources to enhance their teaching or to enable their teaching entirely. In the same study, teachers really felt that their students are having a hard time as well, as they may be having isolating and identifying the right content.

A similar survey that was done by the organization that we’re going to be talking about a bit more today, LRMI found similar things. This was a wider survey than the Pew study looked at a wide group of teachers in K through 6. Teachers are using this content but they’re having some difficulty in identifying the content that they need right now for the classroom.

Obviously the internet is impacting the way the teachers and their students are accessing information. So the question I guess at this conference is: How is that impacting museums or what are museums doing to address that?

We thought we would look at the Smithsonian, which is our sort of tag line is usually the world’s largest museum research complex. So the Smithsonian has been around since 1846. It’s a collection of museums, so we have 19 museums and galleries, nine major international research centers, the National Zoo, a nationwide network of more than 180 affiliated museums, and a collection of programmatic offices that focus on strategic initiatives like the office that we work for.

Since 1846, the Smithsonian has been growing in size adding buildings but really also growing in its visitation both physically and digitally. This is actually the Smithsonian’s first homepage. This is from 1995; total Netscape, classic kind of look. Actually kind of complete for an early ‑‑ it’s not just what are our hours. You have activities and resources.

So in sort of looking at 1995, the Smithsonian in that year, the museums that existed in 1995, which are not as many, just over 23 million non‑digital visits to the museum. And just ‑‑ not even 100,000 digital visits. If we jump forward, that’s about 3%, the digital was about 3% of the physical, of the non‑digital. Jumping ahead to last year, obviously those numbers changed quite a bit. So it’s 30 million non‑digital visits, but 140 million digital visits to the Smithsonian museums.

We sort of kind of do a little bit of math there. We see ‑‑ the physical visitation increased by about 27% and the digital visitation increased by 191,833%. You can argue that this is to be expected. Right? We’ve all seen the global massive expansions. The reason this conference exists, the reason that many of us do what we do, the use of digital tools and the expectation of people of all ages to access rich, digital content.

Linda Kelly, who many of you might know, often says museums now operate in three spaces. There’s the traditional physical space of the museum floor, there’s online and fixed web and places like websites and social media, and then most recently in mobile spaces, mobile apps and sites.

This is a bit of an overview of sort of what we just saw from 95 to 2013 looking at the growth of global population using the internet, both globally and in the developed world, comparing that to how the physical visitations of the museum, I think, has really leveled off. I think many museums are serving, especially large museums like the Smithsonian, are serving about as many visitors in the museum that they can handle. They sort of level off at a point. Unless we continue to add museums, we’re at a point where that’s full.

I think this demonstrated to us ‑‑ one of the reasons that an office like ours exists is to begin to address what is a museum. What does a museum complex like the Smithsonian do to confront this reality?

>> Melissa Wadman:   Hi. Is everyone finding the same thing at their institutions in terms of physical ‑‑ excuse me, non‑digital verses digital ‑‑ versus digital visits? Yes? No?

>> Maybe not the same percentage.

>> Melissa Wadman:   But you would agree with that?

>> Right.

>> Melissa Wadman:   Ok. So we’re here to talk about, so what do we do to help educators find all of this great stuff that we have in terms of educational resources.

The survey that Darren mentioned ‑‑ I’m sorry. I can’t do the slides.

Educators reported that what they need most in terms of search results are relevant content, appropriate grade level information, detailed information on the types of resources that are offered, who the resource was made for originally, and rights and copyright information, especially in terms of classroom use. A lot of the teachers wanted to know that.

We think that metadata is the answer to this. And metadata, as we know, is the information attached to the digital resource to help people find, analyze, and we hope to ultimately use the resource.

In the current environment, educators get all sorts of results when they search. This is just a display of Smithsonian‑only resource that they could possibly find in educational‑related websites.

On the Smithsonian’s, the site that our office hosts, educators, when they search for resources, this what they find: a mix of metadata about the resource including the title, the description, state standards, as well as some paradata.

How many people are familiar with the term “paradata?” Not many. Ok. So paradata was created ‑‑ who coined that term?

>> Darren Milligan:   One of the Registry folks.

>> Melissa Wadman:   We think the Department of Ed Learning Registry. It means user ‑‑ things like ratings, comments, reviews to help people analyze the resources as well.

I think it’s worth noting here that the original model for tagging resources, I think especially for museums, was to align them to as many standards as possible to give broad description so that they could be applicable to more. And now this is sort of with the emphasis on customization and personalization of learning the focus is to be more targeted and precise in the tagging for the resources to make them more useful to the educators.

And currently the industry standard for tagging online content is called created for markup schema. Specialized communities have been encouraged to extend it to meet their own needs. This just gives you a sense of all that’s included in the website. And out of this learning resource metadata initiative emerged from a partnership between the association of educational publishers and the creative for educational resources specifically. It was accepted and published into the in May of last year. So once the metadata is embedded within a research page, search engines can hopefully recognize learning resources much more easily and with more accuracy.

And these are the properties of LRMI. They include things like: What was reported in that survey? Things that were important to educators. The time required to complete. And being an activity defining the educational use. The age range that’s appropriate for the type of resource it is. And, again, the rights and copyright information, etc. I’m not going to read through all of them.

Included here we also have the accessibility metadata properties. They’re not part of LRMI, but we are embarking on a project where we are also including tags for the accessibility. It’s another schema adopted by the website. We’ll be telling about that project shortly.

I’ll hand it over to James.

>> James Collins:   I’m going to stand up here.

We talked about why we were moving in that direction. And we also talked about LRMI a little bit. Basically this is the life cycle that we anticipate. This is what we’re working with here. So we have the publishers’ aspect, which is ‑‑ I think this has a laser.

>> Darren Milligan:   There’s a button.

>> James Collins:   Ahh. Ok.

This is where you all are, these circles represent the resources you create; lesson plans, videos, things like that. And you make all of these things. They’re great. They’re really good. And you bring people to your museums and show them to them and they say, hey, those are great.

The LRMI focus of that is really this arrow here. So we’re talking about tagging it so that we can make it more ‑‑ can you all hear me?

>> Melissa Wadman:   It’s recording.

>> James Collins:   Sorry. Wasn’t aware of that. So the metadata aspect of it is to find everything and make it ‑‑ make everything more findable. Have consistency so when we’re talking about things using the same language and that interfaces with machines really well. If you know about computers, you have to talk to them in the same manner if you change your words, it gets confused.

So magic happens here. And then we get to the lessons part over here. So somewhere in between here it magically ends up in classrooms across the nation. We’re seeing the good work that we have doing good in the world.

So this is the LRMI part. What I’m going to be talking about here is a Learning Registry part which is ‑‑ I like to think of this as this inverse triangle. And then the interface that taps into the Learning Registry.

The Learning Registry sounds complicated, scary. If you look it up, it doesn’t make sense, but I’ll tell what you it is now so you can keep it in your mind. It’s just a repository of learning resource metadata. That’s it. And these are stored in what’s called nodes, but you can think of those as servers. And really it’s no different from the internet at large except that it specifically focuses on learning resource metadata.

So let’s walk through this a little bit with a specific learning resource. So this is the National Zoo’s website. It’s a video about watch fishing cats fish. So you can see we’ve got people who publish this, the video, and a little bit of text. This is not findable right now. You can find it on the National Zoo website, if you’re looking for fishing cat videos. But most science teachers aren’t going to say: I think we should have a class on fishing cats. So let me try to find something related to fishing cats.

There’s a lot of good content in here that you might want to use in a biology class or if you want to talk about predator, prey, eco systems. How do we distill that out so that people can find it?

This is a picture of the Learning Registry. It’s a network diagram which are usually not fun to look at but you can learn a lot from it. What we have is these little green boxes, our producers, you, the museums. We have the blue boxes. Those are consumers. And our office a lot of times that’s k‑12 schools. For you it might be visitors who go back home, might be other museums.

Then we have these nodes here, the common nodes. Those are the actual repositories. And there’s also red nodes, gateways. And the gateway node, you don’t need to worry about it unless CIO or whatever tech office you have makes you. Those are just kind of an additional barrier so that you can maintain the common node on your internet and then use the gateway to connect to other networks.

The great thing about the Learning Registry is that you ‑‑ ok. Let’s walk through this. So you have your fishing cat video. It’s over here. Again, it’s great. You put it on your TV, in your exhibit and everything’s great. You want to share it. So maybe you upload it to, in our case, the Smithsonian node. So once it’s on the Smithsonian node, it can go over here to Smithsonian users. And anybody who’s accessing the Smithsonian website ‑‑ which is great. We get a lot of hits, as we showed, but we would like even more of that. You shouldn’t have to come to the Smithsonian website to see a Smithsonian video.

So the Learning Registry takes this resource and it replicates it out to other nodes. And those nodes then replicate it out to even more nodes. And then if we say that this is the Department of Education’s node, which we’ll talk about in a second, that taps into every other node pretty much. Then we can see that everything is spread everywhere.

And really what that means for us is that this user in California who might not be thinking about fishing cats, might not be thinking about the Smithsonian, but they need something that’s biology related and they go to their California Department of Education website and, hey, this is something that comes up for them.

So that’s the power of the Learning Registry. You might have a few questions about the technical aspect of this. There are a lot of technical details in our paper if you want to look that up. And you can contact me, e‑mail, Tweet, whatever.

You’ll notice here that there’s two networks here. These are open networks. That’s why they’re sharing information in between. This is a closed network. You’ll notice that the fishing cat does not go over here.

Learning Registry, which is really just an installation with a Perl script, a small Perl script running on it, it’s flexible enough that you could do different things with it. It was originally a joint project between the Department of Education, the Department of Defense. So you can imagine why the Department of Defense might want to work in something here that they don’t have to share everything that they do out to everybody else. Right? Everybody doesn’t need to see our Army manuals.

So you can keep it close like that. You can make it one way so that it pushes out. You can make it two ways to push back and forth.

I mentioned Department of Education node. They were the other half of this project. And their node is ‑‑ even though this is a decentralized network, the Department of Education node is essentially a central hub so anything that goes into the Department of Education node pretty much pushes back out to everywhere else. And if you access the Department of Education node, you pretty much have everything that’s in the Learning Registry.

So if there’s one thing to take away from this about the Learning Registry, it’s that you should look at the Department of Education’s node before going anywhere else.

Ok. So big success. Right? Fishing cat video is everywhere. This is the part where you might say, hey, this is amazing. It’s also like ‑‑ where I like to say this is a nascent technology. The Department of Education is doing this. There are two interfaces that I’ll show that you are really good but a lot of other people are just figuring it out right now.

I would say that there’s a low bar to participating in the Learning Registry, but don’t start hyping it up at your museum and say, guess what, you know, we’re going to be all over the world with just this one quick trick. But, yeah.

So this is the Brokers of Expertise website. They pulled from the Department of Education’s node, as far as I understand. This is California Department of Education. So, again, you could see here the metadata that we were talking about, the LRMI data, showing up here. We have the title. We have the submitters, conditions of use, description. And we have the screen shot of that.

So all of this metadata is stored in the node itself, in the Learning Registry. Obviously the picture is not. And this is not ‑‑ we’ll talk about this in a second. It pulls all of that data. So then we see here in the educational resources, the Illinois Shared Learning Environment, we see all of the same information showing up here.

That’s it. You publish the Learning Registry, the Department of Education’s Learning Registry node, and, Bam, it shows up on these pages. Not bad.

Ok. So that’s the metadata aspect of it, LRMI, adopted. It’s set in place.

There’s another part of this called paradata which Melissa talked about briefly. We love getting this, distributing this, out. But we would also like to know what that user in California is doing with that data. If you’re going send this all out, we should get something back, maybe. That’s paradata. So that’s usage data, but it’s also very fine grain activity data that we can collect. So it’s not just accesses and bookmarks and favorites and full views, but it’s also: What did they do with it? And all of that, the dream of paradata, is that all of that will travel this route, replicate across these nodes until it gets back up to us here.

So paradata, if the Learning Registry is nascent, paradata is incubating. There aren’t a lot of implementations out on it. That’s scary on one hand but it’s exciting on the other hand because there’s a lot that can be done with it right now.

So you can see here, this is not pulling from the Learning Registry. It will be in the future. Views, full accesses. So instead of aggregating that across multiple sites, imagine if you could just pull one number from the Learning Registry and that will tell you how many times it’s been accessed from everywhere. Pretty useful.

Even more interesting is this fine grain detail that we can get down here. You see the example: Incorporated into a lesson plan by an educator of college, adult and more. It can get ‑‑ it’s so flexible, it can get that level of fine grain detail. People just aren’t uploading that yet.

If you’re interested in this and you have a technical background, I would say get on the developer’s forum, which is on Google right now, Learning Registry Develop Google Group for Developers. They’re trying to figure this out and find best way to make it more common for everybody else.

For the rest of you, you’re probably just, like, well, I want to dip my toes in this but I don’t want to start a huge project. This is going to be your best bet. This is Easy Publish. It’s hub repository. You can fork it and make your own modifications to it. It was developed by Jim ‑‑ I think it was a hack-athon for them. Here’s the address. I believe we’ll be sharing the slides.

This should look familiar. Here are your LRMI fields up here. And here’s actually ‑‑ this is for a single resource. And then here is an import data section. You can actually type in all of your LRMI data into a spreadsheet and drop it in there, press a button, bam, open the Department of Education node.

How long is this going to take you? Probably not that long. Maybe a day, a week, depending on your technical expertise and how many resources you have.

There is one tricky part to this. That’s the credentials box up here. You need a PGP authentication. You need to register that with the Department of Education. There are more details in the paper about that. And, again, it’s a active Learning Registry community, so if you ask around there, people are friendly about helping you out.

That’s pretty much it. We’ll talk a little bit more about our specific example. Again, if you have questions about some of the nitty‑gritty about some stuff, we’re happy to field that afterwards. But we understand most of the people here aren’t, like, hard‑core developers but more like can we do this in a week and if so, what’s the benefit.

>> Darren Milligan:   So before we sort of dig in and try this all out, we want to, I think, share with you from an institutional perspective what the Smithsonian is doing and how we’re approaching, sort of getting caught up in a sense, I think, is the way that we’re thinking about it. We’ve got about 2,500 educational resources from across the museums that have some metadata that Melissa showed but not complete LRMI metadata.

We have a project in place that we’re sort of kind of on step one right now. We’re sort of moving through this process which is to identify the resources that first warrant having this kind of metadata. So we’ve got a huge amount of resources that some are fairly simple and some are incredibly complex.

From an institutional perspective, I think there’s a conversation about how you segment some of the educational content so that this kind of description makes sense. We’re in that process right now.

We’re going to be working a couple of routes to take to develop this metadata. The one is sort of the simple do-it-yourself method. Because we have so many to get caught up, we’re going to be working with a consultant that’s going to work with classroom teachers to develop this metadata for our existing resources. Once we’re caught up, we’re going to be doing the do-it-yourself method that we’ll look at.

Once that metadata is developed, it will be embedded on at least a couple of the websites where those resources exist. And ideally more given sort of the nature of the Smithsonian Museums tend to do what they want in regards to their websites, so we’re hoping for full participation across the Smithsonian Museum sites. We have a couple picked out that are our partners here that the Museum of American History. We’re going to be working with those sites to take a look at what the impact of this process is.

So once that metadata is embedded on the site, pages will be republished, the metadata itself for all the resources will be released into the Learning Registry. And then we’ll take a look at what the impact of that is.

We’re going to make an attempt to change the culture, which we’ll tell you about.

>> Melissa Wadman:   Right now. I would just like to point out before I talk about the evaluation of this that we are doing this project through an internal grant. We felt it was very important and we wanted to get — not just catch up with resources but we wanted to have enough resources tagged so we would be able to study what happens at the search, analysis, and use of this, of these resources.

As Darren was saying, we are working with a partner, the American History Museum, and we are going to be tagging about 2,500 resources in bulk and then evaluate that. And we’re going to use an external evaluator and look at the project from some different angles. We’re going to be analyzing the quantitative data, the web analytics that we can get by tagging all of this content, but we’re also going to be working with groups of teachers to replicate the LRMI Survey that was sent out that we referred to earlier as it relates to museum resources so that we can add to that body of knowledge as well. And we’re going to have ‑‑ take an in‑depth look at educators and how they search and analyze the resources and try to understand if that leads to any difference use in the classroom or wherever because of better tagging.

And we are also going to be doing a formative evaluation of the professional development opportunities that we’re going to be offering through this, which we’re going to be hosting in‑person workshops for Smithsonian educators, about four this summer. And we’re also going to create a training video of how to generate this data. Metadata, excuse me. We’re going to have virtual tag-athons for those who can’t get to the in‑person so we can work through this process with them. And we’re going to be publishing an LRMI guide for museums and cultural institutions specifically because we feel passionate about this; that this is the right way to go.

Just today I saw an e‑mail from the LRMI initiative that was saying they have a new survey that said 72% of educational publishers are aware of LRMI, 60% of them tag with metadata already, and over 50% of them who don’t tag are planning to start within the year. So we really feel strongly that museums and cultural institutions need to get onboard and be a part of this because all the educational publishers are right now. So if we want people to use educators to use our educational resources, we need to make sure they can find them.

So our hope is with all of this professional development that it becomes a part of the process in creating educational resources for these museum educators. I know everyone has, oh, great, another thing for museum educators to have to do. But we really think of it as it’s something that can help think through while you’re creating; kind of can function as a logic model, formative evaluation as you’re tagging because it makes you think about who are you really creating this, whatever it is, for resource for and how long is it going to take and what do we really want people to get out of it, etc.

>> Darren Milligan:   I think it’s important to say. I think what we’re hoping to do is change the culture a little bit. As you’ll see, if you’re not a museum educator or the person who produced the resource, that you’re going to be doing some tagging with, you’ll see that it really is much easier to develop this kind of content when the resource itself is being developed by the person who developed it. There’s some things about how the resource should be used in the classroom that the person who made it and designed that resource has in their head or on paper and often a year later that kind of material tends to maybe not be as available to the webmaster, to the person who might typically be charged with doing this kind of work, that seems technical.

>> Melissa Wadman:   And before we get into actually tagging, I’m curious. How many people here actually already tag to LRMI with their resources?

Cool. Awesome.

>> Darren Milligan:   Great. I think we’re ready to go.

We’re going to be using, to do this, a tagger called the inbloom tagger. So if you’ve got a laptop or some device out there, please pull it up, I’ll do the same. It looks like this here.

This is not by any means the only tag that’s available. This is a pretty simple user interface that’s designed for doing these sort one at a time or at least to introduce you to the idea that there are a lot more streamlined ways of doing it. Even here there’s a form that you can use in Excel to upload your content in bulk. If you’ve got this material or if you have existing metadata for your resources, you can begin to map that over to the fields that LRMI uses using this field and do more of a bulk upload into this tool or into the Learning Registry.

We thought for today we would pick one of our resources. This is one of the most popular lesson plans available from It’s a few years old, four, five years old, a lesson that looks at World War II, U.S. World War II posters and has some activities where kids do some document analysis. It’s really kind of a lesson about civic responsibilities, sort of on the home front during the war.

Everybody got this up. Anybody who’s participating got this up? Cool. Ok.

So the first step here is to sort of create a new tag. So I’m on the left. Click new. The interface has you paste the url and tag. We’ll pull this up so you can do everything from within the interface.

If you want to jump in. Cool. I’m just going to kind of do this step‑by‑step. If people have questions or anything, this is the time to jump in. And I’m just going to walk through this fairly quickly so you can sort of see. It’s simple, really. Most of the pieces are pretty simple. And then if you have some questions and we’ll talk more about this process itself or the others.

There are three tags here. There’s sort of a general, the educational, specific fields, and then the standards alignment piece which is maybe the only difficult piece if you’re not the person who created this resource.

I’ll walk everybody through here. This resource is called “World War II on the Home Front.” We have a description that exists. Paste this. The language.

Many of these fields which we’ll see when we export this, we’ll look at the code, are open text fields. They’re really ‑‑ there really aren’t standardized vocabularies for most of these. They’re fairly open. The fields themselves are standardized through but the contents can be fairly open.

>> Melissa Wadman:   This is the guidance.

>> Darren Milligan:   There is some guidance. These are ones that are generally ‑‑ some of these where you see ‑‑ they’re fairly open. If you have resources in other languages, you see they’re offered by this tagger.

Melissa made this. Most likely have terms of use for your content. LRMI uses a url for that. But that url could be to create comments page or whatever licensing that you use. Publisher.

So “is based on url” is an interesting one. This is a way that tries to connect or maintain a history. So if your resources are built upon others, are next steps of others, I think in the case of this type of resource we will be using this to link back to the original primary source. So this resource is based upon the “We Can Do It” work incentive poster by Westinghouse. So we’re going to use that. That is our “based on.”

So then the educational fields are the ones that if you’re educators doing this step while they’re creating, this is going to be really easy for them. If you’re the kind of the web guy, web gal, who’s doing this, it’s going to be a lot more difficult. These are things that the educator really thinks about when developing these kinds of resources. They’re often sort of buried within the resource.

The end user here is the student. This is developed for late middle school to high school.

Like I said, again, these are fairly agreed upon vocabularies, but they’re not completely fixed. So I’ll show you where we can change these.

This has some presentation. Some note taking. There’s an introduction. And there’s some hands‑on pieces. It’s active. It’s an activity. PDF.

So you can sort of see when you look at these, and you see how really useful these might be for a teacher in the classroom who’s looking for a group activity that’s about, you know, civic responsibility for 9th graders, you know. If she were to put that into a search engine, she’s not going to find this resource. Once this metadata is available, she understands this resource describes specifically for the activity that she’s looking for.

This is a group activity of a small group activity. And for some reason this tagger has the ability to put resources that take up to 12 years. I don’t know what kind of education or resource might take 12 years. Maybe you have one. So this takes about two hours.

The educational alignment, this is set up to easily do common core. Is everyone more or less familiar with common core state standards? It doesn’t require that you use that. That’s the one that most publishers are doing now because it’s a unified tagger. But certainly there are others that people are doing like Next Generation Science. So this is a fairly flexible one.

Once you’ve walked through that for a resource, you’re given some export fields. We’re going to look at HTML because it’s the simplest to look at. You have CSV, JSON. If you were planning on putting this in the Learning Registry. The HTML embedded on this page.

So here’s what we just generated. This is what we walked through in two minutes. This would be put into the head tag within that website here to be used by the search engines.

This in a different format would be put through the eZ Publish interface that James showed  or into some other methods to put into the Learning Registry.

Does that seem fairly straight forward?

>> Melissa Wadman:   That’s how easy it is.

>> Darren Milligan:   Yeah. There’s things you can do. You can do this in bulk. We’re doing a lot of these at once. We’re not going to walk through this for 2,000 resources. We’re going to do it in a spreadsheet so that we can quickly do this. But I think you can see, this is something that most museum educators, when you talk to, say, yeah, this is the kind of no‑brainer. It would take me a minute or two, five minutes maybe, to do when these resources are being developed.

>> Are you talking about the person who does ‑‑ produce the content or you have a lot of content that’s produced by other educators? What is some of the skills that you’ve learned to be able to do that actively? Or is there some sort of communication? Is there a questionnaire you ask the educators? Anything you specifically look for?

>> Darren Milligan:   I think we’re looking directly to involve the educators in using a tool like this. Because it’s not a technical interface. It was really designed for educators to use. It wasn’t designed to go through and code this directly.

>> Melissa Wadman:   What was your question about past resources that have been created?

>> It was sort of length. There are some educators ‑‑ this is something I have to do. I have a hard time getting other educators to think this way is different than a lot of things that we standardize, innovate. There’s a lot of differences. Plus we have ‑‑ [Inaudible] so strategies to try to help people get more organized.

>> Melissa Wadman:   So, yeah. That’s what we’re trying to tackle. What we are doing is we are outsourcing the bulk tagging to get those 2,500 resources tagged at first. And the reason that we’re putting such an emphasis on the professional development, that’s for the museum profession to try to get this to be understood and easily done as they’re creating them to get that buy‑in. So there will be a guide and the video that everyone, you know, hopefully by the end of the year.

>> I think ‑‑ think organized.

>> Darren Milligan:   I think the main pieces, incentive, maybe, that we’ll have, that a big part of this project is the evaluation. It’s really looking at what happens, you know from a quantitative perspective. How does this process dramatically change the activity on these resources? And the qualitatively, to really look at does it affect the operation the teachers use when they’re using Google to find the resources they need?

So I think from a cultural change perspective within our institution, that’s going to be our ammunition to say, look, this is a resource without it and this is a resource with. You spent all of these resources, this time, to create this incredible learning opportunity, let’s do this one more step to ensure people can find it and find it quickly.

>> I’ll check then.

>> Darren Milligan:   Thank you. Please do.

>> I have a whole bunch of questions. You may want to shut me down.

>> Darren Milligan:   All right.

>> One set of questions revolves around the open educational resource community, which is struggling at this point to get some sort of central depository. And my question is, OCW, connections, Orange Group in Florida, how many of those folks have you ‑‑ to your ‑‑ there are hundreds of thousands of resources already out there.

>> Darren Milligan:   Sure. The Smithsonian right now is not pulling content in, pulling metadata in from non‑Smithsonian resources. So we’re in kind of a push stage right now. So we’re pulling that metadata in an effort ‑‑ these other places that you’re mentioning will harvest that metadata and expose these resources to their user on their platform.

>> That’s what they’re saying. We’re kind of operating at ‑‑ if both or all of your repositories say we’re counting on somebody else to get the data to us, and all we want to do is push out the data, we got to get a higher level of collaboration going.

>> Darren Milligan:   I agree. I think you’ll see the main interfaces that have been built that are using the Learning Registry right now are State Department of Educations; so California, pretty robust systems. So they’re the organizations right now that have the resources, the audience, the need, and that are ‑‑

>> California as resources here.

>> Melissa Wadman:   Digital, educational resources.

>> Darren Milligan:   They have the need. And they’re not ‑‑ it’s large publishers of resources, so they want to tap into places like museums that have quality resources and pull this in. But I agree with you, we have these conversations internally about once we are really in a good push‑pull mode here, you know, why wouldn’t we necessarily expose ‑‑ if someone’s at our site, a teacher looking for resources, you know, why would we hide good, quality content from other providers? That gets into a lot of conversations about the role of the museum, and why are we advertising the marketing or promoting content that we didn’t produce on the platform that we paid to produce.

There’s a lot of those internal political conversations I think that will be happening. I don’t see a reason why anyone who’s involved in the Learning Registry wouldn’t be exposed and pushing. If you’ve got a user there and they’re looking for great content and you’re able to expose that, why ‑‑

>> Especially the publishers who love to ‑‑ [Inaudible] you’ve got creative comments ‑‑ are all of these things creative ‑‑

>> Darren Milligan:   Smithsonian resources are not creative comments tagged.

>> What’s the copyright issue, the copyright status? I put my stuff into your registry?

>> Darren Milligan:   It’s not our registry. Learning Registry works by sharing metadata and some paradata, eventually. So what you’re exposing is just the metadata not the resources themselves.

The Smithsonian has an interesting problem — I would say, since we’re being recorded. It’s an interesting conversation going on about, you know, who gets to do with the resource that we have and what they do with them. We have a blanket usage policy for all of our digital content, more or less, that says the education usage is open and fine. That’s a long way away from creative comments or public domain.

There are the issues I think all museums have with this; that there’s not a lot of people who are focusing on this, that we have collections that have artwork that have interesting copyright issues. We have collections that have donor stipulations that effect resources. Many of these things were developed in a time when this conversation wasn’t happening as much, I think, as it is now. Things were in print. That had an established mechanism for dealing with these issues in a world that seems quite different now.

>> James Collins:   This is less about the collection items and more about the learning resources. This is about taking lesson plans or things that your educators are creating, pushing out metadata which Google is already indexing and making it tight so that search engines can aggregate it properly. It’s just about more discover ability for these, not so much about taking our photos from, you know, our astronomical observatory and uploading it to some server. This is just aggregated metadata.

>> We have some ‑‑ education resources and free both behind passwords. Should we even bother with this if Google can’t find it?

>> Darren Milligan:   Do you have ‑‑ certainly the most ‑‑ the people who are using this right now are mostly commercial publishers. Melissa was saying, you know, half of the commercial publishers are in the process of developing this kind of metadata. I think in the most cases they have a landing page for their for‑profit product that Google can find and users can find that then. Are you in a situation where you have some information that is public or accessible?

>> Yes. But we’re just trying to figure out how to tag 800 little ‑‑

>> Darren Milligan:   Paragraphs.

>> Pieces on that main page.

>> Darren Milligan:   I got you. Yeah. It’s difficult. The examples I’ve seen most publishers are breaking that so they have a landing page, for lack of a better term, for each of their books or each of their ‑‑ many of them, they’re tagging non‑digital resources so they’re tagging printed publications, textbooks, and lesson plans and things aren’t open or digitally accessible. So they’re using it to ensure that people who are buying those kinds of things can find them as well. There might be some models there to look at.

>> I think the challenge with that is just when it’s so much that ‑‑ if the tagging does bring into that page, so helping the user, the searcher, know that they’ve arrived to the right place.

>> Darren Milligan:   Mm‑hmm.

>> Melissa Wadman:   You’re saying there would be a main page for each ‑‑

>> Darren Milligan:   It sounds more like a site architecture issue. Unfortunately that’s the harder thing in some respects.

>> Thank you.

>> So you spoke a little bit about how search engines, if you’re embedding this metadata in your content, how the search engines work with it. Can you speak a little bit more about the current state of what search engines can do versus kind of the future potential of what we’re hoping for with that?

>> Darren Milligan:   Google had a great implementation of this a little while ago that they have since turned off for reasons that they’re not explaining. It was always called the potato salad example. So if you were to search Google for potato salad, you’d be presented on the left. So Google would recognize that you were searching for a food‑related — you know, the food tags for recognize that you were looking for food, potentially a recipe. So on the left you’d have some search filters that you could filter by ingredient, by preparation time, these sorts of things. So that was actually grabbing that metadata and allowing you to affect the rank of research results.

So they’ve removed that, which is unfortunate because this would have had a similar search for civic responsibility lesson plan. You would have been presented with the kind of fields that we were just looking at. So it’s unclear whether Google is going to be the interface for that or a new research engine will be the tool for that.

>> That’s few ‑‑

>> Melissa Wadman:   That’s the future potential of it. Right now if you search for potato salad, you get 20 gazillion recipes which is the same thing people get for learning resources. If you search for Abraham Lincoln or whatever, but if you can filter it better, you know, by that metadata essentially.

>> James Collins:   If you haven’t looked at any research on the semantic web, that’s really what you just need ‑‑ it’s the core promise of that, that we’re not going to be seeing these coming in in the end, indexing sites, anytime soon, some smart, intelligent way. So we need to do the dirty work ourselves to enable that technology to come about earlier than it would otherwise.

>> Looking at the research that focuses on the importance of the metadata is great, but does the research also address the ways ‑‑ coming from the other angle, is search the best way for us to find it? What’s the best way for teachers to discover?

>> Melissa Wadman: What would the other options be besides searching?

>> I don’t know. Imagine like ‑‑ it might be impractical, but there are other options. Right?

>> Darren Milligan:   You can look at this site. Well over half of the searches coming in from search engines directly to a resource land page, not people looking for Smithsonian homepage but searching for civic responsibility lesson plan, arriving at this page here. So I think there’s some indicators just in the basic metrics of how people are arriving at our content that eliminates a little bit of that question.

I think, too, the reason that the Illinois and the California sites exist is the states are being challenged with offering ‑‑ both of those sites are actually not just a search engine. They’re kind of a whole learning management system that has recommended resources sort of tied into that.

So some of the states and some of the commercial providers who have many, many of those who are doing it, are offering, you know, a subset of the internet, I guess, of kind of recommended content. So that behavior certainly is a little different than someone using Google and kind of trying to find their way in.

How this affects that, I think most of those sort of front end ‑‑ as James was saying, the Learning Registry is not a front end access point. It’s an infrastructure that enables this metadata to move around easily.

The Department of Ed and Department of Defense have been upfront saying we’re not building a search portal for education for the world. We’re building the pipes, and it’s up to the states or the Smithsonian or commercial entities to build the front end experiences of those.

I’m not sure if anyone’s really looked at the behavior that is affected by using something like that versus a simple wide search.

>> Melissa Wadman:   You started with data, but ‑‑ that teachers do use. Sorry. I’m sorry. I was saying at the beginning of the presentation you were pointing to data that suggests that educators do use search engines to find resources they’re looking for.

>> Darren Milligan:   Can you describe what you mean by serendipitous experience?

>> I think ‑‑ yes, I recognize that a lot of educators are coming, doing searches, I need this now, let me see what I can find with a quick Google search. But this strikes me as an opportunity to say, oh, you searched for this, well, this isn’t exactly what you were searching for but maybe here’s a couple of other things that might inspire you or that might broaden what you’re looking for or, alternatively, further refine what you’re looking for.

It’s almost like an intelligent search but something that’s really, I think, an opportunity to also expand the ways in which teachers are interacting with resources rather than just saying we know you’re looking for something specific, so I’m going to give that to you in as efficient a way as possible. Maybe there’s a way to think about it differently.

>> Darren Milligan:   Yeah. I think there’s some good examples. Our partner, the National Museum of American History, their educator resource pages, the individual layman pages, have some related resources. They have the primary sources sort in the column to the right.

With museums, especially who are building these resources, using the primary resource that we have, all of that content, there’s a real missed opportunity that that’s not happening. I think there’s probably great interfaces that could be designed that would create a more ‑‑ I like the idea of a serendipitous exploration of related resources and related ideas that sort of connect either across the ideas or across themes.

>> Melissa Wadman:   Like Wikipedia rabbit hole.

>> James Collins:   Let’s keep in mind, too, that ‑‑ remember, the whole ‑‑ the four boxes up there, the yellow section is LRMI, right? So we’re just tagging it to make it more semantic accessible. The blue portion is the Learning Registry, which is just distributing that. And then the green portion, which is more to your question, is the interface. So the interfaces we’ve seen a number of different implementations but clearly there is no perfect method yet. I think we’ll see more exploration and experimentation with that in the future.

>> It’s a good thing you mentioned the semantic web or semantic approach. This information, this statement made with the new input, how do you manage to keep it meaningful? Perhaps we can ‑‑ that is actually truthful, if I can say truthful. Anybody can ‑‑ perhaps not meaningful for the search, perhaps not meaningful for the topic.

So usually if we think of semantic web or linked data, we make ‑‑ it makes sure the data is correct or meaningful to the search. What it seems to me, what you have here, you have these taxonomy that you’re not going to be truthful. Even the mobile, network is perfect. Perhaps the creation of the input of the data might not be as accurate as it needs to be.

I don’t know if you have anything behind it to make it more accurate.

>> James Collins:   Unfortunately we’ve been told we have to leave because it’s 2:30. We can add to that outside afterwards.

>> Melissa Wadman:   Thank you.

>> Darren Milligan:   Thank you, everyone.


[The presentation ended at 2:32 p.m.]