Valuing Our Scans: Assessing the Value and Impact of Digitizing Ethnographic Collections for Access


Ricardo Punzalan, University of Maryland College of Information Studies, USA, Brian Butler, University of Maryland, iSchool, USA

Abstract

Over the years, the heritage field has made great strides at making ethnographic and anthropological collections available in their digitized formats. Through digitization, institutions are finding creative ways to make these materials more discoverable and accessible. Access to ethnographic collections is increasingly mediated through digital avenues. Yet, despite these advances in digitization, no clear criteria have been proposed for assessing the impact and use of online digitized collections. Moreover, there is scant understanding of the initial selection criteria used by heritage professionals and administrators for digitizing particular collections. What are the goals of digitizing ethnographic collections? How are these goals set? And how are these goals assessed?

This paper highlights the approaches and perspectives employed in a collaborative project between the Smithsonian Institution and the University of Maryland College of Information Studies. Our project aims to identify important considerations in assessing the value and impact of digitization of ethnographic collections. This paper presents preparatory work for this project and is organized in three parts. The first describes the profound changes happening within the LAM sector. In particular, we look at the effects brought about by the expanding role of digitization and online access as fundamental institutional functions of heritage repositories. The second provides an overview of the current literature on impact and assessment that address the issue of valuing digitized collections. Third, we outline our ongoing collaborative research project that examines cases of ethnographic digitization projects in seven cultural heritage institutions. Finally, conclude by offering “five considerations” that we propose to frame efforts for assessing the impact of value of digitized ethnographic collections.

Keywords: Digitization Impact Assessment; Digitized Ethnographic Collections

Abstract

This paper highlights the approaches and perspectives employed in a collaborative project between the Smithsonian Institution and the University of Maryland College of Information Studies. Our project aims to identify important considerations in assessing the value and impact of digitization of ethnographic collections. This paper presents preparatory work for this project and is organized in four parts. The first describes the profound changes happening within the LAM sector. In particular, we look at the effects brought about by the expanding role of digitization and online access as fundamental institutional functions of heritage repositories. The second provides an overview of the current literature on impact and assessment that address the issue of valuing digitized collections. Third, we outline our ongoing collaborative research project that examines cases of ethnographic digitization projects in seven cultural heritage institutions. Finally, conclude by offering “five considerations” that we propose to frame efforts for assessing the impact of value of digitized ethnographic collections.

Introduction

Over the last two decades, the libraries, archives, and museums (LAM) sector has made great strides at making ethnographic and anthropological collections available in digitized formats. Through digitization, cultural heritage institutions are finding creative ways to make these materials more discoverable and accessible. Access to ethnographic collections is increasingly mediated through digital avenues. Yet, despite these advances in digitization, no clear criteria have been proposed for evaluating the impact of providing online access to ethnographic collections. Moreover, there is scant understanding of the initial selection criteria used by heritage professionals and administrators when choosing collections and particular items for digitization. What are the goals of digitizing ethnographic collections? How are these goals set? And, more importantly, how is progress toward the goals assessed?

Digital collections have evolved to a point where “simply serving useful digital collections effectively to a known constituency is not sufficient” (NISO, 2007: 1). By supporting new forms of use and sharing, digitized collections fundamentally change the range and types of communities that can engage and be affected by the availability of these collections. While a plethora of general standards and guidelines for developing and maintaining quality digital collections are available, and are indeed useful in understanding technical, administrative, and policy aspects of digitization, these models do not attend to the impact of extant digital products and collections on various audiences and stakeholders (UKOLN, 2006; Kenny & Rieger, 2000; Arts and Humanities Data Service, 2008; and Schreibman, 2007).

In our project, we examined the most recent and notable efforts of measuring the impact of digital cultural heritage projects, services, and collections. We wish to build on the achievements of earlier works, including Simon Tanner’s (2012) Balanced Value Impact Model, TIDSR: Toolkit for the Impact of Digitised Scholarly Resources (http://microsites.oii.ox.ac.uk/tidsr/), Archival Metrics (Yakel & Tibbo, 2010), E-Metrics (Fraser, McClure, & Leahy, 2002), and UKOLN’s framework for metrics for JISC programs (Kelly, 2012). These endeavors have come up with conceptual frameworks for planning, structuring, and organizing evaluation efforts to help cultural heritage institutions determine the value of digital services and resources to a variety of stakeholders. However, these standards and evaluation tools address broad swaths of information resources and hence may not be wholly appropriate for measuring the value of providing access to culturally sensitive content. They nevertheless point to a need for developing relevant processes and metrics for articulating impact and value of specific institutional services and resources, such as digitized ethnographic materials.

It is useful here to define some of the terms that we use throughout the paper. Although they are highly interrelated, the differences between goals, objectives, values, metrics, and impact are important for understanding how digitization programs are managed. Digitization project goals and outcomes are distinguishable from institutional goals and outcomes (Fraser, McClure & Leahy 2002). Goals and objectives are expressions of desired results and planned deliverable products. However, these targets and hoped for results, while useful to institutions, may not correspond with outcomes and impacts, which derive from actual and meaningful use of and engagement with digital resources. A plan to digitize millions of images (a project goal), may support institutional goals of making these visual materials accessible online (an institutional goal). As digitization progresses, the collection of digital images will grow (a project outcome), allowing for a greater proportion of the collection to be available online (an institutional outcome) enabling increased scholarly use or significant changes in the interpretation of visual content (impacts). According to Tefko Saracevic (2009), leading scholar in digital library evaluation, “Evaluation refers to ascertaining the performance or assessment of value of some entity” (5).  It is during planning (prioritization of goals) and evaluation (assessment of outcomes and impacts) that the nature of the stakeholders and their values becomes central.

It is in the context goals, outcomes, impacts, and values that metrics exist. We follow Simon Tanner’s (2012) definition of impact as “the measurable outcomes arising from the existence of a digital resource that demonstrate a change in the life or life opportunities of the community for which the resource is intended” (9). Metrics are indicators of outcomes that are typically created with quantitative data, often for use by stakeholders who are not directly involved in realizing the outcomes. Metrics are thus created in consideration of institutional goals and objectives and their outcomes. Metrics are also used as indicators of impact gathered using assessment and evaluation tools.

Ethnographic collections present another complicated concept. These collections encompass various materials in diverse formats, from field notes to photographs, from utilitarian implements to ornaments, from textiles to sound recordings. What distinguished ethnographic collections is the link to one or more source communities. At an abstract level, all material artifacts kept in libraries, archives, and museums, may be linked to a given culture or cultures that produced them. However, our focus on digitization goals, outcomes, impacts, and values led us to focus on collections traceable to identifiable, indigenous source communities. In these cases, the complexity of goal prioritization and impact evaluation is greater due to the need to consider the values and concerns of the source communities along with those of the project and institution. Practically, we have adopted a broad notion of what constitutes ethnographic collections, limiting the scope to materials that are located in repositories that self-identify their materials as ethnographic. In this sense, we relied on our participating cultural heritage institutions to guide us in identifying relevant collections.

This paper highlights the approaches and perspectives employed in a collaborative project between the Smithsonian Institution and the University of Maryland College of Information Studies. Our project aims to identify important considerations in assessing the value and impact of digitization of ethnographic collections. This paper presents preparatory work for this project and is organized in four parts. The first describes the profound changes happening within the LAM sector. In particular, we look at the effects brought about by the expanding role of digitization and online access as fundamental institutional functions of heritage repositories. The second provides an overview of the current literature on impact and assessment that address the issue of valuing digitized collections. Third, we outline our ongoing collaborative research project that examines cases of ethnographic digitization projects in seven cultural heritage institutions. Finally, conclude by offering “five considerations” that we propose to frame efforts for assessing the impact of value of digitized ethnographic collections.

Part 1. Digital Transformations 

Providing access to digital resources for a spectrum of constituencies has gained strong patronage and support from both the scholarly and funding communities. Cultural heritage institutions in turn have made great strides in responding to this enthusiasm by moving beyond digitization to exploring creative avenues for access and use. The importance of capturing and assessing the impact of digitized ethnographic collections must be understood within the context of recent developments in digitization and how they affect institutional change within the LAM sector. Thus, we begin by providing an overview of the significant of the institutional transformations in this arena.

Access to digitized objects has become a common feature on the websites of many heritage repositories. Millions of images are now accessible online courtesy of image scanning technologies, efficient, interactive web interfaces, and faster and broader Web connectivity. Along with many others in the related fields of libraries, archives, and museums, Shan Sutton (2004) observes that “digitization in special collections has begun to shift from a temporary endeavor to a fundamental responsibility” (235). The field is at a critical moment of development that has been brewing for over a decade and has now passed what Sutton (2004) characterizes as “the point of no return” for special collections.

Melissa Terras (2008) describes the evolution of digitization from an ad hoc and largely experimental venture by a few large institutions to a significant part of the day-to-day performance of heritage work. The progress in this area seems unstoppable: from focusing on small sample collections to explore the potential of space-efficient storage at the National Archives and Records Administration and the Library of Congress in the early 1980’s, to mass scanning and the establishment of standards and guidelines in the 1990’s, to the digitization of entire collections and development of techniques and projects that involved cultural heritage materials beyond textual records in 2000’s.

Digitization and online access are becoming the normal modes of fulfilling institutional mandates of access and preservation (Erway, 2008). Reports, such as OCLC’s “Scan and Deliver” (Schaffner, Snyder & Supple, 2011), “Rapid Capture” (Erway, 2011), “Capture and Release” (Miller, Galbraith, & the RLG Partnership Working Group on Streamlining Photography and Scanning , 2010), and “Shifting Gears” (Erway & Schaffner, 2007) speak to the various ways that institutions are responding to digitization as a mode of bringing collections to users. From “digitization-on-demand” (Schaffner, Snyder, & Supple, 2011) to “digital cameras in the reading rooms” (Miller, Gilbraith, and the RLG Partnership Working Group on Streamlining Photography and Scanning, 2010) these reports evidence the ways that institutions are scaling digitization efforts in repositories and developing workflows that respond to demands for creating and delivering digital copies.

While digitization is changing modes of access and delivery for heritage collections, it is also reshaping institutional structures and practices (Koltun 1999). In the last ten years, cultural heritage institutions have been reconsidering staff expertise and responsibilities (Pearce-Moses & Davis, 2006), rethinking resource requirements and allocations, and redefining organizational structure and makeup (Peacock, 2008). Large institutions, for instance, are slowly creating new positions in order to accommodate new digital responsibilities and better coordinate digital initiatives (Boock and Vondracek, 2006). Institutions are also venturing into collaborative digital projects aimed at harnessing the potential of online environments while sharing the cost of digitization (Kalfatovic, et al., 2008). In short, the incorporation of digital technologies in heritage repositories is both an impetus for organizational reinvention and an opportunity to keep institutions and their holdings relevant in the digital age (Witcomb, 2007).

A study published in 2008 by the Institute of Museum and Library Services (IMLS) shows that the availability of heritage materials online does not discourage in-person visits to libraries and museums. The same report concludes that there is “compelling evidence that museums and libraries have leveraged the availability of the Internet to present their resources and services to a broader audience and offered an additional mode of access to them, while traditional in-person visits continue to increase” (Griffiths & King, 2008).

Digitization is redefining relationships among heritage repositories, the collections that they keep, and the patrons that they serve (Samouelian, 2009 and Hirtle, 2002). Online digital images have proven to be useful in the discovery of otherwise hidden or inaccessible materials, and they are transforming research practice (Research Information Network, 2008). Digitization is not only reconfiguring core heritage functions, but it is also reshaping expectations of access and use by both users and creators (Manoff, 2010 and Michel, 2005). The availability of primary sources supports the creation of what Carole Palmer (2004) calls “thematic collections,” or digital materials that span varied sources and organized by researchers to suit specific scholarly endeavors. Digitization is reshaping contemporary “epistemic infrastructure” for all aspects of access (Hedstrom & King, 2007).

Encountering artifacts via their digital and online surrogates is fast becoming one of the usual modes of experiencing heritage objects (Taylor, 2003). Scholars in media, visual, and cultural studies have contemplated this recent phenomenon and have offered up some ways to describe the impact of the digital media in contemporary society. Sarah De Rijcke and Annie Beaulieu (2011), for instance, introduce the notion “image interface” to emphasize how access to knowledge is becoming more and more networked and filtered through visually mediated interfaces. Bella Dicks (2004), in addition, describes online displays of cultural heritage as “virtual destinations,” or sites where cultures are in “visitable” forms.

Many platforms for facilitating online access have emerged. Digital library platforms such as KORA, DSpace, and contentDM are making contributions to scholarly communications by making access to academic outputs easier and faster. In addition, other platforms such as Omeka, Drupal, and Hydra have offered new access layer frameworks for digital content. Online exhibitions provide alternative venues for experiencing artifacts beyond the museum or gallery walls. Some heritage repositories have explored digital archives not only to aid the discovery of archival records, but also to involve users in describing archival collections. Virtual reunification helps to bring together scattered or dispersed collections that are otherwise difficult to recombine in their physical format (Austenfeld 2010).

Archival scholar Paul Conway (2010) notes the continued expansion of digital collections with the ongoing transformation of analog collections through mass digitization. With the availability of digital surrogates, institutions are providing various means for materials to be experienced in the online world. In another paper, Conway (Forthcoming, 2014) suggests that institutions should “treat digital surrogates as archives in their own right.” With resources invested in developing digital products coupled with relatively sophisticated scanning technologies and established standards, the field is now capable of producing very high quality digital surrogates that should merit long-term preservation. Thus, institutions are shifting towards maintaining two collections at the same time: the analog artifacts and their digital counterparts. In sum, digitization and online access to collections influence staff competencies, institutional structure, and infrastructures; since these have become pervasive modes for presenting and accessing collections, our work aims to better evaluate such digital presentations.

Focusing on Ethnographic Collections

LAM scholars have reflected on the potential outcomes of digitization and online delivery of ethnographic contents. A number of reports simultaneously point to the contentious nature and the liberating effects of digital projects involving ethnographic collections (Christen, 2011 and Leopold, 2013). However, anthropological materials also raise special issues, such as conversations around issues of cultural sensitivity (Ogden, 2007). Digital collections stimulate discussions around notions of institutional ownership and authority versus indigenous knowledge and history, the relationship of the object kept in repositories with their source communities, and the ethics of open and unhindered display of indigenous artifacts (Pilcher & Vermeylen, 2008; Vermeylen & Pilcher, 2009; Christen, 2011; and Leopold, 2013). Because the practice of collecting, preserving, and exhibiting ethnographic collections is inextricably linked with the history of colonialism, heritage professionals and administrators responsible for these materials must weigh appropriate cultural protocols in displaying indigenous artifacts. For many institutions, the motivation for digitizing and making these collections available online are different from monographs or journals that are more typical for academic and research libraries. Corporate or personal archival records have their own copyright and privacy complications that are completely different if examined within the context of indigenous rights.

Various reports and projects tout the possibility of digitally “returning” materials to source communities (Shenton, 2009; Austenfeld, 2010; and Lynch, 2008). Digitized collections can offer the unique affordance to return complex and fragile objects more easily through digital surrogates than does physical repatriation. Digital repatriation projects involving archaeological artifacts, medieval manuscripts, or literary works have particular sets of concerns and questions. Indigenous collections, however, may present quite different challenges for repatriation (Robert Lancefield, email message to authors, February 13, 2014). There are studies of community outreach and physical repatriation as well as the impact of these practices on institutional and community relations (Barkan, 2002; Peers & Brown, 2003; Brown & Peers, 2006; and Leopold, 2013). In recent years, authors have paid particular attention to digitization of indigenous artifacts and online representation of indigenous cultures (Vermeylen & Pilcher, 2009; Naka, et al., 2008; Roy & Christal, 2002; Pilcher & Vermeylen, 2008; and Dawson, Levy, & Lyons, 2011).

Digital repatriation is a relatively recent possibility, but the practice has garnered some attention from scholars, owning institutions, and source communities (Smith, 2008 and Henessy, 2009). Kimberly Christen (2011) observes that digital repatriation can be contentious, especially when digital surrogates are considered to be replacement for physical objects. In addition “no one, standard definition, nor agreed-upon terminology, characterizes the multiple practices of collecting institutions, individuals, or local community groups surrounding the return of cultural and historical materials to indigenous communities in their digital form” (Christen, 2011: 187).

While members of the LAM community have begun to address the issues surrounding repatriation practices (Christen, 2011; Boserup, 2005; Lancefield, 1998; and Lyndon, 2010), more work needs to be done to understand the effectiveness of return via digital methods. In particular, the field needs more empirical research on the adoption of online technologies as a strategy for repatriation, including the logistics and negotiations of contacting and working with source communities.

Given their sensitive nature, sheer volume, and complexity, ethnological collections are perhaps among the largest body of “hidden” artifacts in cultural institutions. Through digitization, institutions are finding creative ways to make these materials more discoverable and accessible. Access to ethnographic collections is increasingly mediated through digital avenues. To realize the full potential of digitization as a basis for heritage collections access and delivery, it is also critical that institutional structures and practices change (Koltun, 1999). However, the rapid development of digitization procedures in collecting institutions has primarily focused on preservation, access, and storage, with less attention being given to assessing the impact of digitized materials. Thus, we still need to understand the measures needed in order to appropriately document the value and impact of ethnographic digital resources.

Part 2. Assessing Impact

The advent of mass digitization coincides with numerous calls for more sophisticated approaches for evaluating the impact of LAM collections, programs, and services in the digital landscape. With more and more resources allocated towards digital collections, the need for justifying the value of online programs, projects, and services has become more evident. In the past fifteen years, many studies have noted the inability of a significant portion of institutions to measure the value of their work beyond simple usage statistics and frequency of visits. While LAM institutions routinely compile data on programs and services as well as collections usage and conditions, institutions seem to fall short in analyzing these data to drive decision-making or to institute institutional reforms (Davies, 2002). Moreover, while data collection strategies that record number of visits or frequency of requests or borrowings may provide useful information, these data do not offer reliable measures of institutional impact or nuanced portraits of audience engagement (Saracevic, 2009).

According to Lisa R. Carter, “Curatorial instinct regarding the estimation of research value needs to be confirmed by hard data gathered through assessment” (2012: 90). In the current funding landscape, it is no longer sufficient to highlight the assumed inherent value of collections, institutions are expected to perform “value-adding activities” such as “exhibition, outreach, publication, and digitization” with the noteworthy materials in their care. More importantly, institutions are challenged to provide evidence that demonstrates that the outcomes of such activities are meaningful contributions to both institutional missions and larger research, learning, and teaching environments (Chapman & Yakel, 2012).

The absence of relevant metrics that are applicable across institutions is a significant barrier. Another is what some have cited as a lack of a “culture of assessment” in the LAM community (Lakos and Phipps, 2004). In response, scholars developed a range of assessment tools and reliable measurement metrics. Four significant efforts in this arena are worthy of note, E-Metrics, Archival Metrics, Toolkit for the Impact of Digitised Scholarly Resources (TIDSR), and Balanced Value Impact Model. All address a range of services, tools, and programs that mainly target the perspectives of library, special collections, museums, and archives users. Without going into great detail regarding specific details of the measurement tools created, we will highlight the approaches and perspectives these projects have taken in developing impact measures.

E-Metrics is a research initiative spearheaded by the Association of Research Libraries (ARL) under its New Measures Initiative (Franklin & Plum, 2010). ARL’s project proposes measures of how digital resources contribute to “institutional outcomes,” a perspective on evaluation particularly focused on ways repositories contribute to achieving goals of the larger organizations of which they are a part. This goes beyond documenting inputs and outputs, which proved to be too simplistic. The E-Metrics approach looked at “a number of outcomes-related activities, engaging in an extensive effort to develop frameworks for understanding and depicting measurable library inputs and outputs in the context of indicators of institutional outcomes” (Fraser, McClure, & Leahy, 2002: 505).

Archival Metrics promoted user-based evaluation aimed at understanding the uses and users of digital resources, with the end desire of transforming services and experiences (Duff, et al., 2010). The Archival Metrics project created assessment tools that practitioners can use to evaluate the impact of their services to users. The Archival Metrics toolkits comprised five user-based evaluation instruments for archives and special collections in colleges and universities: researcher, archival web sites, online finding aids, student researchers, teaching support (Yakel & Tibbo, 2010). It also created toolkits to evaluate the economic impact of government archives as well as designing and administering focus group discussions.

Two prominent impact assessment projects that focused on digital scholarly resources are based in the U.K. First is the Toolkit for the Impact of Digitised Scholarly Resources (TIDSR), which provides assessment tools, advice for best practice, and guidelines pertaining to the strengths and weaknesses of various qualitative and quantitative toolkits for evaluating various types of digital scholarly resources (http://microsites.oii.ox.ac.uk/tidsr/). Another initiative is the Balanced Value Impact Model, developed by Simon Tanner (2012), which is “intended to aid the thinking and decision making of those wishing to engage in Impact Assessment” (4). The model focuses on evaluating outcomes and change “within an ecosystem for a digital resource” (4). Both TIDSR and the Balanced Value Impact Model provide guidelines for collecting data to measure impact from the very inception to completion of digitization projects.

Impact assessment and evaluation begin with some determination of “value” to a given set of “stakeholders” (Hughes, 2012).  The E-Metrics, Archival Metrics, and TIDSR projects conducted studies that determined what librarians, archivists, curators, and users consider as key areas for measurement. Similarly, a study conducted by Wendy M. Duff, et al. on user-based evaluation identified “the type of feedback [that archivists] value, methods they currently employ to gather feedback from users, benefits and problems posed by conducting formal evaluation studies, and ways archivists would use standardized questionnaires for user-based evaluation if these tools were available” (144). A similar study is needed for digitized ethnographic collections.

Ethnographic collections are often created by repositories that exist within larger institutions. As such, the findings and frameworks from the E-Metrics projects are likely to be relevant. These collections also often have diverse user communities, making the user-oriented approach taken by Archival Metrics and the TIDSR projects as well as the Balanced Value Impact Model report are also important. Yet, ethnographic collections also have another type of stakeholder – the source community. As a result, any attempt to characterize the value and impact of these collections will need to both include and move beyond the focus of prior work in the area of assessment.

Part 3. Valuing Our Scans

Thus, we found that it is time to assess the impact of digitization from larger and comparative context. Many previous assessment projects have been field-wide initiatives such as E-Metrics for academic research libraries and Archival Metrics for special collections and archives. We can pursue an investigation focusing on one of the most complex types of heritage collections, i.e., ethnographic materials. A first step is to develop an approach that can begin to provide a multi-institutional perspective. Taking a cue from previous assessment projects, we begin by asking how heritage professionals and administrators in institutions currently in the process of digitization wish to evaluate the outcomes of providing access to ethnographic collections online. This is the approach we take in the collaborative project, “Valuing Our Scans: Towards a Metric for Assessing Impact, Value, and Use of Digitization and Digital Surrogacy for Ethnographic Collections.”

Valuing Our Scans responds to two important concerns in the digitization of ethnographic collections. First is the lack of a consolidated framework for assessing the impact of digitizing ethnographic and anthropological collections that considers the perspectives of various stakeholders: the heritage institutions that house them, the source communities from which the artifacts originated, and the various designated and emergent users of these digitized materials. The second is the absence of clear methodology for understanding the impact of digitization projects and their resulting products across multiple stakeholders. By examining these two concerns, the project seeks to develop a general framework, metrics, and methods for assessing the impact, value, and use of digitization for ethnographic collections. The project started in Fall 2013 and it is organized in two phases.

Phase 1 – Pilot Study

In the first phase of the project, we are studying digitization efforts involving ethnographic collections at various units of the Smithsonian and in other institutions with significant ethnographic holdings. Our focus is on web-based digital projects at institutions that primarily deal with cultural heritage collections. The range of digital projects we consider range from online exhibitions, to publicly accessible online catalogs, to digital repatriation projects. Our choice of institutions is guided by their location (they are all located in the East coast) and the variety of digital projects they represent. These units are:

  • National Anthropological Archives and Human Studies Film Archives
  • Department of Anthropology of the National Museum of Natural History
  • Smithsonian Center for Folklife and Cultural Heritage
  • National Museum of the American Indian
  • American Museum of Natural History (New York, NY)
  • Harvard’s Peabody Museum of Archaeology and Ethnology (Cambridge, MA)
  • University of Pennsylvania Museum of Archaeology and Anthropology (Philadelphia, PA)
  • American Philosophical Society (Philadelphia, PA)

We are currently conducting interviews about anthropology-related digitization projects at the subject institutions, to document project goals, outcomes, and methods used to assess collection value and impact. In addition, for background purposes, we are also gathering information about the procedures used to bring digital objects from creation through to presentation, publication, and exhibition.

This stage of the project documents emergent approaches for assessing the impact and uses of digital products and projects. These approaches will be characterized based on systematic analysis of qualitative data gathered from semi-structured interviews, focus group discussions, and archival research.

Phase 2 – Valuing Our Scans Workshop

The second phase of the study is a daylong experts’ and users’ discussion in April 2014 to be conducted in College Park, MD. The aim of this activity is further data gathering,  environmental scanning, and agenda setting. The workshop is designed to solicit ideas of what has been done and what still needs to be done in developing assessment metrics and methods for digital ethnographic projects. We will also report salient findings of our research study before this audience. The goal is to supplement the plan and generate interest in a larger effort develop and empirical substantiate a framework for characterizing and measuring the value, impact, and use of digital ethnographic collections.

Prospective participants in the workshop include those heavily involved in digitization projects in general as well as those engaged in developing and studying digitized ethnographic collections. We are also inviting heavy users of online ethnographic collections.

Part 4. Five Considerations

We have so far considered the literature about and implications of digitization and impact assessment for heritage institutions. Our project triangulates existing metrics with current assessment goals of heritage institutions as well as desired institutional outcomes for digitized collections. As we move forward, we are pursuing five major areas of consideration that are likely to be critical for advancing state of the practice in creation, management, and use of digital ethnographic collections. These are presented here in the spirit of discussion, and we hope that further discussion with the audience will develop these five considerations even further.

1. Do digitized ethnographic collections require a unique set of metrics for assessing their value and impact? While the argument can be made that ethnographic collections have distinctive features, empirical evidence is needed to support the claim that separate measurement scheme are necessary and will not simply replicate existing measures. What are the benefits and drawbacks of designing assessment by collection type and content rather than institution type?

2. What do heritage professionals and administrators consider as the most important areas for assessing the impact of digitized ethnographic collections? On one hand, some of the power of metrics arises from their usefulness in setting standards applicable across institutions.   On the other, measures are more likely to provide a basis for change if they reflect the priorities and values of specific organization. Our project gathers existing evaluation measures, if any, from all the participating repositories in order to identify both divergence and commonality and to understand how those features serve the potentially conflicting functions of value assessment.

3. How do heritage institutions understand and respond to changing users and uses of digitized ethnographic collections over time? What are major drivers of institutional decisions? Are investments in digitization driven by data about user needs, or other concerns? We therefore aim to document and compare the institutional decision-making processes and development of projects at the participating institutions. To do this, our project traces the history and development of digitization programs in various repositories. In particular, we seek to identify the motivations that mobilize heritage professionals and administrators to embark on digitization projects. It is important to account for the evolution of our programs and projects, but we must also trace how these changes are transforming our audiences. 

4. What are the institutional effects of digitization in cultural heritage institutions that primarily collect and keep ethnographic collections? Many claims have been made about ways in which mass digitization has impacted and changed the structural make up of cultural heritage institutions. Most of these have been about ways in which access has been transformed, so we want to look at institutions. As more and more ethnographic materials are made available online, it is important to understand how this impacts resource allocation within institutions, building or changing expertise of institutional staff, and work practices associated with these collections.

5. How are the additional stakeholders associated with ethnographic collections included in the measurement of value, impacts, and outcomes? Impact assessment for ethnographic collections cannot be exclusively institution-centric; it must also take into account “designated communities” of collection users (The term “designated communities” comes from the OAIS reference model and refers to users of digital archival materials, whether institutional, research-focused, and others. See: Consultative Committee for Space Data Systems, 2012). We plan to incorporate perspectives beyond heritage professionals and administrators; other stakeholders, including funding agencies, academic researchers, educators, and most importantly, indigenous source communities will be considered. If putting cultural objects online creates an avenue for indigenous communities to access their cultural heritage, we would be remiss to neglect considering the outcomes interactions with these diverse groups. When considering ethnographic collections, source communities are particularly important stakeholders to include in the designated community. Our project also explores ways in which source communities can be involved in measurement and evaluation. Source communities may have completely different notions of value for digitized ethnographic collections than curators, librarians, and archivists.  If this is the case, it is likely to present challenges for professional staff in repositories who may not have the knowledge necessary to incorporate these outside perspectives in their assessment effort and collection management decisions.

Valuing Our Scans is a first attempt to develop a multi-institutional assessment framework for measuring the value and impact of digitized ethnographic collections. As we expand from our preliminary focus on seven institutions to involve more heritage repositories from across the United States, we will continue addressing these five areas of consideration and the overall challenge of characterizing and demonstrating the complex value of these materials and institutions.

Acknowledgement

This project has received funding support from the University of Maryland, College Park and the Smithsonian Institution Seed Grant in 2013. We wish to thank our collaborators at the Smithsonian Institution, Dr. Robert Leopold and Diana Marsh and our graduate research assistants, Ann Abney, Erin Antognoli, and Dr. Massimo Petrozzi for the energy and insights they brought to the project.

References

Arts and Humanities Data Service (AHDS). (2008). Guides to Good Practice. Consulted January 14, 2014. http://www.ahds.ac.uk/creating/guides/index.htm

Austenfeld, A.M. (2010). “Virtual Reunification as the Future of ‘Codices Dispersi’: Practices and Standards Developed by e-codices—Virtual Manuscript Library of Switzerland”. IFLA 36(2), 145-154.

Barkan, E. (2002). “Amending Historical Injustices: The Restitution of Cultural Property,” In E. Barkan and R. Rush (Eds.). Claiming the Stones, Naming the Bones: Cultural Property and the Negotiation of National and Ethnic Identity. J. Getty Trust, 16-46.

Boock, M. and R. Vondracek. (2006). “Organizing for Digitization”. portal: Libraries in the Academy 6(2), 197-217.

Boserup, I. (2005). “The Manuscript and the Internet: Digital Repatriation of Cultural Heritage”. IFLA Journal 31(2), 169-173.

Brown, A.K. and L. Peers (with members of the Kainai Nation). (2006). ‘Pictures Bring Us Messages’: Photographs and Histories From the Kainai Nation. Toronto: University of Toronto Press.

Carter, L.R. (2012). “Articulating Value: Building a Culture of Assessment in Special Collections”. RBM: A Journal of Rare Books, Manuscripts, and Cultural Heritage 13(2), 89-99.

Chapman, J. and E. Yakel. (2012). “Data-Driven Management and Interoperable Metric for Special Collections and Archives User Services”. RBM: A Journal of Rare Books, Manuscripts, and Cultural Heritage 13(2), 129-151.

Christen, K. (Spring/Summer 2011). “Opening Archives: Respectful Repatriation”. American Archivist 74, 185-210.

Conway, P. (2010). “Preservation in the Age of Google: Digitization, Digital Preservation, and Dilemmas”.  Library Quarterly 80(1), 61-79.

Conway, P. [Forthcoming 2014]. “Traces and Transformations: A Case for the Archival Nature of Digital Surrogates,” Archival Science 14.

Consultative Committee for Space Data Systems. (2012). Reference Model for an Open Archival Information System (OAIS). Washington, DC: CCSDS Secretariat. Consulted January 14, 2014. http://public.ccsds.org/publications/archive/650x0m2.pdf.

Currall, J., M. Moss and S. Stuart. (2004). “What is A Collection?”. Archivaria 58, 131-146.

Dawson, P., R. Levy, and N. Lyons. (2011). “‘Breaking the Fourth Wall’: 3D Virtual Worlds As Tools for Knowledge Repatriation in Archaeology”. Journal of Social Archaeology 11(3), 387-402.

Davies, J.E. (2002). “What Gets Measured Gets Managed: Statistics and Performance Indicators for Evidence Based Management,” Journal of Librarianship and Information Science 34(3), 129-133.

Deegan, M. and S. Tanner. 2002. Digital Futures: Strategies for the Information Age. London: Library Association Publishing.

De Rijcke, S. and A. Beaulieu. 2011. “Images as Interface: Consequences for Users of Museum Knowledge,” Library Trends 59(4), 663-685.

Dicks, B. (2004). “Virtual Destinations,” In Culture on Display: The Production of Contemporary Visibility. Open University Press, 170-194.

Duff, W., et al. (2008). “Archivists’ Views of User-based Evaluation: Benefits, Barriers and Requirements”. American Archivist 71(1), 144-166.

Duff, W., et al. (2010). “The Development, Testing, and Evaluation of the Archival Metrics Toolkits”. American Archivist 73 (2), 569-599.

Erway, R. (2011). Rapid Capture: Faster Throughput in Digitization of Special Collections. Dublin, Ohio: OCLC. Consulted January 14, 2014. http://www.oclc.org/content/dam/research/publications/library/2011/2011-04.pdf.

Erway, R. and J. Schaffner. (2007). Shifting Gears: Gearing Up to Get Into the Flow. Dublin, Ohio: OCLC. Consulted January 14, 2014. http://www.oclc.org/resources/research/publications/library/2007/2007-02.pdf.

Franklin, B. and T. Plum. (2010).“Assessing the Value and Impact of Digital Content”. In S.H. Hee (Ed.) Bridging the Gap: Connecting Users to Digital Contents. New York: Routledge, 41-57.

Fraser, B.T., C.R. McClure, and E.H. Leahy. 2002. “Toward a Framework for Assessing Library and Institutional Outcomes”. portal: Libraries and the Academy 2(4), 505-528.

Griffiths, J.M. and D.W. King. February 2008. Inter-Connections: The IMLS National Study on the Use of Libraries, Museums, and the Internet (Conclusions Summary). Institute of Museum and Library Services. Consulted January 14, 2014. http://interconnectionsreport.org/reports/ConclusionsSummaryFinalB.pdf.

Hedstrom, M. and J.L. King. (2007). “Epistemic Infrastructure in the Rise of the Knowledge Economy”. In B. Kahin and D. Foray (Eds.). Advancing Knowledge and the Knowledge Economy Cambridge, MA: MIT Press, 113-134.

Hirtle, P.B. 2002. “The Impact of Digitization on Special Collections in Libraries”. Libraries and Culture 37(1), 42-52.

Henessy, K. (April 2009). “Virtual Repatriation and Digital Cultural Knowledge”. Anthropology News, 5-6.

Hughes, L.M. (2004). Digitizing Collections: Strategic Issues for the Information Manager. London: Facet.

Hughes, L.M. (2012). “Introduction: the Value, Use and Impact of Digital Collections”. In L.M. Hughes (Ed.). Evaluating and Measuring the Value, Use and Impact of Digital Collections. London: Facet Publishing.

Erway, R. (2008). “Supply and Demand: Special Collections and Digitisation”. Liber Quarterly 18(3/4), 324-336.

Kalfatonic, M.R., et al. 2008. “Smithsonian Team Flickr: A Library, Archives, and Museums Collaboration in Web 2.0 Space”. Archival Science 8, 267-277.

Kenney, A.R.  and O.Y. Rieger. (2000). Moving Theory into Practice: Digital Imaging for Libraries and Archives. Consulted January 14, 2014. http://www.library.cornell.edu/preservation/tutorial/.

Koltun, L. (Spring 1999). “The Promise and Threat of Digital Options in an Archival Age”. Archivaria 47, 114-135.

Lancefield, R. (1998). “Musical Traces’ Retraceable Paths: The Repatriation of Recorded Sound”. Journal of Folklore Research 35(1): 47-68.

Lakos, A. and S. Phipps. 2004. “Creating A Culture of Assessment: A Catalyst for Organizational Change”. portal: Libraries and the Academy 4(3), 345-362.

Lee, H.L. (2000).“What is a Collection?”. Journal of the American Society for Information Science 51(12), 1106-1113.

Leopold, R. (Spring-Fall 2013). “Articulating Culturally Sensitive Knowledge Online: A Cherokee Case Study”. Museum Anthropology Review 7(1-2), 85-104.

Lyndon, J. (2010). “Return: The Photographic Archive and Technologies of Indigenous Memory”. Photographies 3(2), 173-187.

Lynch, C.L. (January/February 2008). “Repatriation, Reconstruction, and Cultural Diplomacy in the Digital World”. EDUCAUSE Review 43(1), 70-71.

Manoff, M. (2010). “Archive and Database as Metaphor: Theorizing the Historical Record”. portal: Libraries in the Academy 10(4), 385-398.

Michel, P. (2005). “Digitizing Special Collections”. Library Hi Tech 23(3), 279-395.

Miller, L., S.K. Galbraith, and the RLG Partnership Working Group on Streamlining Photography and Scanning. (2010). Capture and Release: Digital Cameras in the Reading Room (Dublin, Ohio: OCLC). Consulted January 14, 2014. http://www.oclc.org/content/dam/research/publications/library/2010/2010-05.pdf.

Naka, M., et al.  (December 2008). “Indigenous Digital Collections: An Early Look at the Organisation and Culture Interface”. Australian Academic and Research Libraries 39(4), 223-236.

National Information Standards Organization. (2007). A Framework of Guidance for Building Good Digital Collections. NISO: Baltimore, Maryland.

Northeast Document Conservation Center. (2000). Handbook for Digital Projects: A Management Tool for Preservation & Access. Consulted January 14, 2014. http://www.nedcc.org/assets/media/documents/dman.pdf.

Ogden, S. Summer 2007. “Understanding, Respect, and Collaboration in Cultural Heritage Preservation: A Conservator’s Developing Perspective”. Library Trends 56(1), 275-287.

Palmer, C.L. (2004). “Thematic Research Collections”. In S. Schreibman, R. Siemens and J. Unsworth (Eds.). A Companion to Digital Humanities. Blackwell Publishing, 348-382.

Peacock, D. (2008). “Making Ways for Change: Museums, Disruptive Technologies and Organizational Change”. Museum Management and Curatorship 23(4), 333-351.

Pearce-Moses, R. and S.E. Davis, Eds. (2006). New Skills for a Digital Era. Washington, DC: National Archives and Records Administration, Society of American Archivists, and Arizona State Library.

Peers, L. and A.K. Brown. (2003). Museums and Source Communities: A Routledge Reader. New York: Routledge.

Pilcher, J. and S. Vermeylen. (2008). “From Loss of Objects to Recovery of Meanings: Online Museums and Indigenous Cultural Heritage,” M/C Journal 11(6) (2008). Consulted January 14, 2014. http://journal.media-culture.org.au/index.php/mcjournal/article/viewArticle/94.

Research Information Network. (2008). Discovering Physical Objects: Meeting Researcher’s Needs. London: Research Information Network.

Roy, L. and M. Christal. (April 2002). “Digital Repatriation: Constructing Culturally Responsive Virtual Museum Tour”. Journal of Library and Information Science 28(1), 14-18.

Samouelian, M. (Spring/Summer 2009). “Embracing Web 2.0: Archives and the Newest Generation of Web Applications”. American Archivist 75, 42-71.

Saracevic, T.  (2009). “Introduction: the Framework for Digital Library Evaluation”. In G. Tsakonas and C. Papatheodorou (Eds.). Evaluation of Digital Libraries: An Insight Into Useful Applications and Methods. Oxford: Chandos Publishing, 1-13.

Schaffner, J., F. Snyder, and S. Supple. (2011). Scan and Deliver: Managing User-Initiated Digitization in Special Collections and Archives. Dublin, Ohio: OCLC. Consulted January 14, 2014. http://www.oclc.org/content/dam/research/publications/library/2011/2011-05.pdf.

Schreibman, S., Ed. (2007). Best Practice Guidelines for Digital Collections at University of Maryland Libraries, 2nd ed. Consulted January 14, 2014. http://ourdigitalworld.org/wp-content/uploads/2012/04/DigitizationBestPractices_Schreibman.pdf.

Shenton, H. (2009). “Virtual Reunification, Virtual Preservation and Enhanced Conservation”. Alexandria 21(2), 33-45.

Sutton, S. (2004). “Navigating the Point of No Return: Organizational Implications of Digitization in Special Collections”. portal: Libraries and the Academy 4(2), 233-243.

Smith, D.A. (2008). “From Nunavut to Micronesia: Feedback and Description, Visual Repatriation and Online Photographs of Indigenous Peoples”. Partnership: the Canadian Journal of Library and Information Practice and Research 3(1), 1-19.

Tanner, S. (2012) Measuring the Impact of Digital Resources: The Balanced Value Impact Model. King’s College London. Consulted February 13, 2014. Available at: www.kdcs.kcl.ac.uk/innovation/impact.html.

Taylor, B. (2003). “Enhancing the Value of Museum Websites:  Lessons from the Practical Engagement Front,” Journal of Library Administration 39 (2/3), 107-122.

Terras, M.M. (2008). Digital Images for the Information Professional Ashgate, 99-139.

UKOLN. (2006). Good Practice Guide for Developers of Cultural Heritage Web Services. Consulted January 14, 2014. http://www.ukoln.ac.uk/interop-focus/gpg/

Vermeylen, S. and J. Pilcher. (2009). “Let the Objects Speak: Online Museums and Indigenous Cultural Heritage”. International Journal of Intangible Cultural Heritage 4, 60-78.

Washington State Library. Digital Best Practices. Consulted January 14, 2014. http://digitalwa.statelib.wa.gov/newsite/best.htm

Witcomb, A. (2007). “The Materiality of Virtual Technologies: A New Approach to Thinking About the Impact of Multimedia in Museums”. In F. Cameron and S. Kenderdine (Ed.) Theorizing Digital Cultural Heritage: A Critical Discourse. Cambridge, Massachusetts: MIT Press, 35-48.

Yakel, E. and H. Tibbo. (2010). “Standardized Survey Tools for Assessment in Archives and Special Collections”. Performance Measurement and Metrics 11(2), 211-222.


Cite as:
. "Valuing Our Scans: Assessing the Value and Impact of Digitizing Ethnographic Collections for Access." MW2014: Museums and the Web 2014. Published January 16, 2014. Consulted .
https://mw2014.museumsandtheweb.com/paper/valuing-our-scans-assessing-the-value-and-impact-of-digitizing-ethnographic-collections-for-access/


Leave a Reply