Augmented Reality for Interpretive and Experiential Learning

Karen Elinich, The Franklin Institute Science Museum, USA


The Franklin Institute in Philadelphia has been investigating the use of augmented reality technology in exhibit-based, hands-on learning environments. Fixed-position augmented reality is having a positive impact on the learning outcomes for middle-school students who participate in the ARIEL project research.

Keywords: augmented reality, science learning

1. Introduction

In 2008, The Franklin Institute began to explore the use of augmented reality technologies in science museum exhibits as a tool for better integrating the experiential and interpretive aspects of hands-on engagement with scientific phenomena. Augmented reality combines real and virtual objects in the real environment. It changes in real time in response to manipulation, and it aligns real and virtual objects with one another. Our ultimate goal was to leverage these characteristics to strengthen the informal science learning experience that we provide for our visitors.

The Franklin Institute is a science museum located in Philadelphia, Pennsylvania, USA. The organization dates back to 1824, with its origins rooted in the promotion of the mechanical arts. In 1934, the Institute opened its current incarnation as a hands-on science museum. Throughout its history, its paramount mission has been advancing the spirit of inquiry and discovery embodied by its namesake.

Like at most science museums, traditional exhibit practice at The Franklin Institute has been to design a thematically linked series of devices that invite learners to encounter individual scientific phenomena in some interactive, hands-on way. To understand magnetic forces, for example, one must first feel the forces by holding two magnets near each other, so we provide bar magnets for play. To encourage extended exploration and  provide interpretive content, a graphic panel with label copy is placed beside the magnets. The expectation—or hope—is that the learner will play with the magnets, read the graphic panel, play some more, refer back to the panel, make sense of the experience, and ultimately walk away having learned about magnetic force fields. This ideal scenario is occasionally manifested, but the far more prevalent reality is that the visitor engages with only one aspect of the experience; some visitors only play with the magnets, while others only read the label copy.

As augmented reality technologies were becoming accessible in 2008, we wondered if they might be useful in exhibit design practice. As we explored the technology, we realized that they might help to integrate the interpretive and experiential aspects of our typical informal science learning experiences. The Augmented Reality for Interpretive and Experiential Learning (ARIEL) project is our effort to research and develop the transformative potential of the technology in service to informal science learning.

Our project structure has three parallel yet interwoven strands: prototyping, learning research, and platform development. Each has its own focus and goals, but the findings and outcomes reinforce the activity and process in the others. For example, as we prototype new device interfaces, we engage the learning research to test learning impacts with student populations. The learning research findings then inform the next stage of prototyping. Likewise, the technical needs that become evident during prototyping inform the platform development process. At this stage of the project (early 2014), all three strands have reached a mature state that is informing both our ongoing internal work as well as the field. Each strand is presented individually here, but one of the keys to our success has been that the three are in parallel and informing each other continuously.

2. Prototyping

In order to learn and develop our capacity, we began using augmented reality technologies in our prototyping workshop. We were concurrently in the process of redeveloping our electricity exhibit, so we selected a prototype called “Be the Path” from that exhibit. We based our decision on several factors. It was convenient, so we knew we could move quickly. It is a fairly classic device across the field of science museums, so the results might have wide applicability. Also, the principles of electrical circuits and conductivity that it presents are important and common across the educational spectrum.

Be the Path, in its non-augmented classic form, invites learners to complete a simple circuit by using their bodies to conduct the flow of electricity. A learner grasps two metal balls, and a bulb lights to show that the circuit it complete. Using EyesWeb and Basic Stamp and a background-differencing approach, we added a digital overlay that showed an animation of the flow of electricity along the learner’s arms, illustrating the flow of electrons through the circuit (Figure 1). One essential element of our approach to using augmented reality is that the experience be entirely “gear-free” and not introduce any barriers between the learner and the device (Snyder & Elinich, 2010). Our use of augmented reality is in fixed-position implementations where the technology is responsive to triggers at the device. Many of the learners who used “Be the Path” never questioned the source of the digital overlay. Their presence and action triggered the layer of content in an entirely natural and intuitive way.

elinich-fig1 300w, 400w, 2048w" sizes="(max-width: 526px) 100vw, 526px" />

Figure 1: Be the Path in its augmented condition

Another classic device that we augmented was “Magnet Maps.” It is rare that a visitor might never have felt the force of a magnet before encountering this device. We deliberately wanted to leverage that common prior knowledge, as we added a field-mapping simulation to the device. We wanted to explore how augmented reality could enable us to deepen a simple, familiar experience. In our augmented version, the learner rotates two bar magnets and sees a real-time visualization of the magnetic forces changing around the poles (Figure 2). Arduinos beneath the magnets register position so that the ARIEL Builder software (described below) can render the field lines accordingly. The essential experience of moving two bar magnets that is the classic approach remains the core of this experience. All of the digital and augmented aspects are entirely responsive to that basic hands-on play. The actual experience is highly enriched, however, because the learner can now “see” overlaid around their hands the interpretive content that would typically appear on a graphic panel nearby for reading.

elinich-fig2 300w, 476w" sizes="(max-width: 500px) 100vw, 500px" />

Figure 2: Magnet Maps in an early prototype stage

Having now strengthened our technical capacity, we moved to a more challenging, yet equally classic, device: the “Bernoulli Ball Blower.” Most every science museum includes some variation on this device in order to engage learners with scientific principles of basic fluid dynamics. The playful engagement with a ball floating in a stream of air can seem magical to many learners. Traditionally, we expect them to stop playing and focus on interpreting a graphic panel with complex diagrams and illustrations of physical formulas. Not surprisingly, many learners ignore the label copy and just play with the ball, walking away with a fun memory but no better understanding of the phenomenon than they had when they began. We used augmented reality to position the interpretive content directly in the learner’s line of sight so as to increase the potential for understanding. As the ball is floating in the airstream, the camera is tracking its position (color blob tracking) and feeding that information to the processing program, which then renders the visualization that illustrates areas of higher and lower air pressure (Figure 3). When the ball finds its balance between the higher- and lower-pressure areas, it floats. As it drifts, the display provides a visualization of the changing pressures.

elinich-fig3 300w, 479w" sizes="(max-width: 500px) 100vw, 500px" />

Figure 3: Bernoulli Ball Blower positions the digital augmentation directly in the learner’s line of sight

These first three devices—Be the Path, Magnet Maps, and Bernoulli Ball Blower—were used extensively for the learning research described below. From a prototyping perspective, they also enabled a strong foundation on which to continue testing and adapting our approaches. They provided our team with a basic vocabulary and reference set as well, which we routinely referenced as we continued to select devices for augmentation. We looked for devices that represented different augmentation challenges—devices that were not similar to the first three.

The “Sand Pendulum” was one of those challenges. There is an elegant simplicity about the typical sand pendulum—but what’s the science? We decided to use augmented reality to demystify the Lissajous figures produced by the pendulum as it spilled sand on the surface below. Our first approach was to use potentiometers below the surface to track the path of the sand and feed that data into the ARIEL Builder software for display on the sand (Figure 4). The digital path ultimately represented a visualization of time and position for the pendulum.

elinich-fig4 300w, 476w" sizes="(max-width: 500px) 100vw, 500px" />

Figure 4: The first augmented Sand Pendulum prototype

While the augmentation was successful, we struggled with two key issues. First, as the visualization begins to form, it is appealing, but by the time it is completed, it seemed to be detracting from the elegance of the sand figure. We feared that we were degrading, rather than enhancing, the primary experience. Also, we struggled to articulate how the visualization aided understanding. We decided to try again.

In the second attempt, we separated the compound pendulum’s two axes and project the force along each separately alongside the growing Lissajous sand figure (Figure 5). We were now working above the surface, so some of the sand issues were resolved. However, we determined that the visualization was adding only minimal value to understanding: it was clear now that two axes were involved. But we also found that the visualizations were drawing focus away from the actual Lissajous figure as it was forming. As a standalone device, we decided that the augmentation was not appropriate. However, at this point in the project, we discovered another role for augmented reality: as a tool for facilitator interaction with a learner. When one of our exhibit educators stood beside the device and facilitated engagement with it, the augmentation was a definite asset. The augmented reality effectively enabled the facilitator to “slow down” the physics. We developed a “switch” for the device so that it could easily exist in its elegantly simple, non-augmented state while also being ready for use by a facilitator. We also decided to select another device that is traditionally facilitated and further explore the potential for augmented reality to support the facilitation practice.

elinich-fig5 300w, 474w" sizes="(max-width: 500px) 100vw, 500px" />

Figure 5: The second augmented Sand Pendulum prototype

“Gravity Well” is one such device. While it is a classic device, it typically requires facilitation if the learner is to understand any of the physics involved. Countless learners will happily roll the marble or small ball and watch it spin its way into the well, but the science involved is beyond the capacity of a graphic panel. How can augmented reality help the facilitator provide the needed interpretation? As we listened to facilitators, we frequently heard phrases like “when it did that” or “did you see that?” In other words, the facilitator hoped that the learner noticed something that had just happened. Our experience with Sand Pendulum had shown us that we could use augmented reality to “slow down” the physics. Our approach was to capture the roll of the ball and then, once the ball reached the well and ended its spin, “replay” the physics so that the facilitator could point out key features and explain what the spin pattern showed. The path of the ball is tracked and layered directly onto the well. Using background differencing techniques, we used infrared lighting on the well and a camera with an infrared filter to track the ball, feeding the information into the ARIEL Builder software, which renders the visualization for projection directly onto the surface of the well (Figure 6). The result is a powerful tool for interpretation of a brief experiential encounter with an essential scientific phenomenon.

elinich-fig6 300w, 474w" sizes="(max-width: 500px) 100vw, 500px" />

Figure 6: The augmented Gravity Well prototype

Throughout the project, augmented reality technology was entering the mainstream as a commercial application, especially for consumer purposes. This presented a challenge for us as we sought to differentiate how our approach was different from the typical smartphone or tablet-based interaction with information that had become commonplace. Most frequently, people understood marker-tracking (or glyph-based) applications as being the extent of augmented reality technology, since that approach was being used in advertising and marketing for commercial products or experiences. That confusion was a minor frustration for us, but it also prompted us to decide that we should develop a prototype using the marker-tracking approach. There were several reasons why we had not been using marker tracking previously. In exhibit design and environmental installation, there was a real concern that the tracking was not robust enough to support our objectives. Also, most glyphs are sufficiently “odd” in their appearance that they serve as a distraction. Our fundamental belief in the primacy of the hands-on encounter with a scientific phenomenon had always influenced our decision making; the augmentation must always enhance and never distract from the primary engagement. But we felt compelled to develop a prototype using markers so as to be certain we had fully engaged the spectrum of augmented reality approaches.

In many science museums, you will find microscope stations where a learner can look at specimens on slides via a scope. There are many variations on the design. The commonality, of course, is the ability to see something small enlarged. We thought it would be helpful to add interpretive content to the enlarged view as a way to increase the learning potential. We developed a device so that the scope was reading markers on the slides and relaying positional coordinates to the ARIEL Builder software in order to respond dynamically to the current view and reveal real-time information (Figure 7). For example, as the wings of a fly come into focus under the scope, the viewer also sees label copy pointing directly at the wing. As the view changes to the eyes, different copy appears.

elinich-fig7 300w, 473w" sizes="(max-width: 500px) 100vw, 500px" />

Figure 7: The microscope station prototype

Our findings confirmed our hypothesis: marking tracking was not robust enough (at that time) to support our objectives. The tracking simply lacked the precision we needed in order to render the information in the right places at the right time. We set this device aside in hopes that the approach might become more accurate as it develops. Our current consensus remains that marker tracking is not sufficiently robust for augmenting hands-on interaction with devices, although it may be entirely appropriate for less-complex purposes, such as overlaying information onto products or revealing hidden messages.

3. Learning research

When we began the ARIEL project in 2008, there was a need for systematic and rigorous studies of learning designs in informal science education settings in order to understand the real potential for impacting science education. We responded to this need in three key areas. First, we wanted to provide clear evidence that conceptual gains result from informal engagement with science content. Second, we wanted to learn how emerging digital platforms improve the learning experience in informal settings. Finally, we sought to determine the emergence of higher-order cognitive skills such as critical thinking and theorizing through informal science experiences.

We designed our studies around the central premise that the status quo in museum exhibit learning anticipates that the learner will engage both the primary hands-on experiential aspect and the secondary interpretive aspects of a designed informal science learning experience. Yet a wide gap exists between the two aspects, with many learners unable to engage the secondary aspect on their own. The ARIEL project was launched to attempt to use augmented reality technology to bridge that gap.

Because informal science learning is a voluntary activity, there is a level of unpredictability associated with it that presents a challenge for educational researchers. This variability has historically complicated the design of learning research within informal environments. In order to overcome some of the complicating factors, the ARIEL research focused on one key segment of the museum’s visitor population: middle-school students who visit the museum on school field trips. While they are indeed a convenience population and may not perfectly represent the true informal learner—since their visit to the museum is determined by their teachers—school-group students are a major segment of the overall population of museum visitors and, therefore, it is important for us to understand how they learn in exhibits.

Our ARIEL model involves four variables: augmented reality technology, knowledge-building scaffolds, small-group collaboration, and informal engagement. For our studies, we arranged and manipulated a series of quasi-experimental conditions in pursuit of an ideal configuration of the four variables.

In all our studies, our protocols and instruments included pre- and post-intervention surveys of conceptual knowledge related to the device’s scientific phenomenon. These brief surveys were administered in the students’ classrooms on the days before and after their field trips. The instrument was designed to measure conceptual understanding via a series of both multiple-choice and open-ended questions. We also used a student response sheet during the intervention to collect responses to open-ended questions that probed cognitive and critical thinking, such as theorizing. We used an observational instrument derived from the Critical Thinking Skills Checklist (Yoon et al., 2011) and conducted interviews with 20 percent of the students to capture qualitative data related to the students’ understanding of the intervention.

As mentioned above, our learning research involved the first three prototype devices: Be the Path, Magnet Maps, and Bernoulli Ball Blower. The research results being reported here are for our studies with Be the Path. Figure 8 shows the conditional arrangement of the testing area for Be the Path. The other two devices had a similar configuration during testing.

elinich-fig8 300w, 318w" sizes="(max-width: 700px) 100vw, 700px" />

Figure 8: The conditional arrangement for the Be the Path intervention in the testing area

For Be the Path, our testbed included 307 students (52 percent female, 48 percent male) from nine local middle schools (grades six, seven, and eight). All nine of the school populations were at least 50 percent eligible for free or reduced lunch. (“Free or reduced lunch” is a term used in American public schools to indicate eligibility for federal assistance. It is also an approximate indicator of socioeconomic status. Students receive free or extremely low-cost lunch in the school cafeteria.) Six of the nine schools were at least 80 percent eligible for free or reduced lunch. The students were randomly assigned to conditions.

Table 1 shows results from the conceptual knowledge surveys administered before and after the students’ field trip. The questions probed basic understanding of electrical circuits and conductivity. Knowledge gains in all conditions (except condition 1, in which students interacted only with the non-augmented version of the device) were statistically significant. Students in condition 4 demonstrated the greatest gains, with condition 3 a close second.


Mean Difference




Sig. (2-tailed)

1 (Device Only)






2 (Digital Augmentation)






3 (Posted Questions)






4 (Collaborative Groups)






5 (Posted Knowledge Building)






6 (Recorded Knowledge Building)







Table 1: Results of paired-samples T-Test comparing means within conditions *p<0.05

Figure 9 shows results from the cognitive understanding questions asked during the field trip, essentially asking students to theorize in response to the question, “What are you supposed to learn by using this device?” There is an increasing trend from condition 1 to 6 in the means for higher-order reasoning.

elinich-fig9 300w" sizes="(max-width: 400px) 100vw, 400px" />

Figure 9: Results of cognitive understanding measurement

Our findings across studies (Wang et al., 2012, Yoon et al., 2012a, 2012b, 2012c) include compelling evidence that digital augmentations can serve as valuable scaffolds for conceptual learning in science museum exhibits. Which combination of learning scaffolds is optimal for use in a science museum, given the particular characteristics of informal learning? In other words, how much scaffolding is acceptable before we lose the benefits of informal engagement? Our findings demonstrate an apparent tension between efforts to increase science knowledge-learning outcomes with more formalized scaffolds and the ability to preserve voluntary participation characteristics that make informal learning attractive in the first place. The concept of “overformalization” (Yoon et al., 2013) emerged from our studies as we focused too heavily on cognitive gain at the expense of informal play (Yoon & Elinich, 2013).

Cumulatively, our evidence points toward the favorability of using mixed-reality applications (traditional devices mixed with augmented reality overlays) within the context of an informal science learning experience. The results to date suggest that digital augmentations positively impact the hands-on science learning experience. Conceptual knowledge and cognitive understanding are both favorably impacted.

4. Platform development

While we were developing prototypes of augmented devices and using them to conduct learning research, we were also working to develop an open-source software application that would simplify the process for other exhibit developers. The result is the ARIEL Builder platform, which is freely available for use. Since it was developed to facilitate the use of augmented reality technology in informal science learning experiences, the target user community includes exhibit and program developers who work at hands-on science museums. While the software was developed with this community in mind, all developers are welcome to use ARIEL Builder—in fact, we believe that it might be easily adapted for other purposes.

ARIEL Builder is a node-based visual programming application. System requirements currently include Mac OS X 10.6 or higher, a minimum of 2 gigabytes RAM, and 300 megabytes hard-drive space. The application is downloadable via the project’s website ( or GitHub.

5. Summary

The larger significance of our investigations is that the process for exhibit development may be transformed. To date, the prototype devices that we used have featured digital augmentations layered on top of an existing “classic” device that was originally designed for traditional use. The availability of mixed-reality applications may change the way that hands-on science learning exhibits are designed and constructed for maximum learning impact.

The impact of the ARIEL project on our exhibit team at The Franklin Institute has been transformative. We currently have several new exhibit projects in development, and we now consider mixed-reality approaches from the beginning of the prototype process. We now include augmented reality as one of many tools in our toolkit.


This material is based upon work supported by the National Science Foundation under Grant No. 0741659. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.

Over the life of the ARIEL project, there have been dozens of team members involved, too many to acknowledge here. The team currently (early 2014) includes Dr. Susan Yoon (Co-PI), Dr. Jayatri Das, Eric Welch, Brian Kelly, James Sannino, Elizabeth Reynolds, Alexander Bruno, Joyce Lin, and Emma Anderson. Dr. Steve Snyder was the original PI for the project.


Snyder, S., & K. Elinich. (2010). “Augmented reality for interpretive and experiential learning.” Proceedings from EVA 2010: Electronic Visualization and the Arts, London, UK.

Wang, J., S. Yoon, K. Elinich, & J. Van Schooneveld. (2012). “Investigating the effects of varying labels as scaffolds for visitor learning.” Proceedings of the International Conference for the Learning Sciences (ICLS), Sydney, Australia.

Yoon, S., & K. Elinich. (2013) “Doing augmented reality and knowledge-building in a science museum: Formalizing an informal learning experience.” Proceedings of the annual conference of the International Conference on Computer Supported Collaborative Learning, Madison, Wisconsin.

Yoon, S., K. Elinich, J. Wang, C. Steinmeier, & J. Van Schooneveld. (2011). “Fostering critical thinking in science museums through digital augmentations.” Proceedings of the annual conference of the International Conference on Computer Supported Collaborative Learning, Hong Kong, China.

Yoon, S., K. Elinich, J. Wang, C. Steinmeier, & S. Tucker. (2012a). “Using augmented reality and knowledge-building scaffolds to improve learning in a science museum.” International Journal of Computer-Supported Collaborative Learning, 7(4), 519-541. doi:10.1007/s11412-012-9156-x

Yoon, S., K. Elinich, J. Wang, C. Steinmeier, & J. Van Schooneveld. (2012b). “Learning impacts of a digital augmentation in a science museum.” Visitor Studies, 15(2), 157-170. doi:10.1080/10645578.2012.715007

Yoon, S., K. Elinich, J. Wang, & J. Van Schooneveld. (2012c). “Augmented reality in the science museum: Lessons learned in scaffolding for conceptual and cognitive learning.” Proceedings of Cognition and Exploratory Learning in the Digital Age (CELDA), Madrid, Spain.

Yoon, S., K. Elinich, J. Wang, J. Van Schooneveld, & E. Anderson. (2013). “Scaffolding informal learning in science museums: How much is too much?” Science Education, 97(6), 848-877.

Cite as:
. "Augmented Reality for Interpretive and Experiential Learning." MW2014: Museums and the Web 2014. Published February 2, 2014. Consulted .

Leave a Reply