Here students and teachers can explore the STEM concepts and skills found in the artists work. The STEM Concept tool provides a basic definition of a key concept in Science, Math or Engineering and suggests possible ways it has been applied or illustrated in each artist's work. It also includes artist tips and views on STEM for a personal perspective on the STEM + Art connections.

Nettrice Gaskins and Laurie Marion
What is your name and where are you from? Tell us a little bit about yourself.

Nettrice: My name is Nettrice Gaskins and I’m a PhD candidate and researcher with Georgia Tech's Experimental Games Lab. My work investigates culturally situated technologies. This includes new media tools and platforms, and existing cultural artifacts. My mother was a computer programmer and my background is in art and technology.

Laurie:  My name is Laurie Marion.  I was born in North Carolina and received a good foundation in music from private teachers and the North Carolina School of the Arts.  In college I became interested in philosophy and earned a Ph.D. in philosophy at Emory University in 1996.  Because communications technologies have greatly expanded our possible modes of expression, I began studying digital media at Georgia Tech two years ago.  I became fascinated with mobile augmented reality because it offers the potential to create philosophical experiences by enabling people to place virtual comments within physical contexts using mobile devices.

Are you a digital native or digital immigrant?

Nettrice: Digital native.

Laurie: From a child development perspective, I believe we are all digital immigrants.

What is the purpose behind your ISEA2012 piece and what inspired you?

Nettrice: When I was a high school freshman graffiti artist and pioneer Futura (formerly Futura 2000) came to my school to teach us spray can art as part of a community mural project for the local transit authority. So, in a sense, doing AROS with local youth in Albuquerque (at Wells Park Rail Corridor) brings me full circle as an artist, educator and researcher. Futura and street artists like him 'make do' with the materials or information around them and they reflect whatever is popular in culture. For example, Futura is known for using the atom/atomic symbol in his artworks. He also experiments with the spray can in ways that go beyond the traditional graffiti art that is seen on the street and his art reflects abstract themes such as cells or the universe. This way of producing art has inspired my projects, including my digital media Ph.D. research.

Laurie:  For this particular work, we are creating a dialogue between images from an ancient past and designs from an oracular perspective on the future.  We fill in some details by studying works by archaeologists, cultural and social historians, and contemporary indigenous scholars and artists.  At the same time we cultivate an open attitude to insights that may emerge from the process of the group that is working together to produce this piece.

What do you hope an audience takes away from this piece? 

Nettrice: I hope that the audience comes away with a deeper sense of Indigenous, or non- Western artistic, literary, and cultural/aesthetic aspects of the world, especially artistic styles or movements that combine elements of science, mathematics, science fiction, fantasy and magical realism in order to investigate contemporary issues of people of color, but also to re-examine linkages to historical events of the past. This is a model for how I approach working with local or Indigenous youth and help them combine their own ideas with existing artifacts from their own cultures to create a mural (for AROS).

Laurie:  We hope our audience will become aware of the potential that these new web-based applications have to offer for communicating important information about the local community.  Mobile Augmented Reality can be a powerful force for educating the public about resources available in a given area.

How is your medium or technique unique? How have you integrated, adapted or recombined STEM components to create something innovative?

Nettrice: In 2010 I was commissioned by IBM to create a virtual art simulation based on Afro (black)-futurist cultural production. One of several installations included a double helix tower and flowing water beneath it. I consulted with a real scientist about how to build it. "Multiversal Flow" explored concepts such as duality, Diaspora, double consciousness, and dance. In the center is a double helix (DNA) tower that signifies genetics, reproduction, freedom, and interconnectedness as part of life and the timeless dance with the sun. Each installation I created for the IBM show explored something that connects to STEM. For AROS I consulted with a mathematician who specializes in cultural designs. Like my earlier experience working with graffiti pioneer Futura 2000 I made do' with the materials or information around me and I created images and objects that reflected contemporary issues. The process of 'making do' with materials on hand or producing with limited resources or creating from the imagination often leads to innovation.

Laurie:  Our work blends traditional media with computer vision-based augmented reality displayed on the screen of a camera-enabled mobile device.  For the mural, we used geometry techniques to scale up the images in the original design created on poster board.  For the mobile augmented reality part of the piece, I taught coding in KML, HTML and JavaScript and led the group through a design process that included debugging and introduced the coding for the next version of the Argon Augmented Reality browser which will become available through the App store within the next year.  The underlying STEM concept that makes vision-based mobile augmented reality possible is the concept of electromagnetic waves and how they are converted to voltages in the camera that form a binary pattern processed by a chip.  Software has been developed that can recognize patterns and produce images on a screen in response to what a computer “sees.”

What would you suggest as a STEM activity or resource for a student that would like to explore the type of work that you do?

Nettrice: My research touches on so many different ideas or concepts: sacred geometry (math), hip-hop (graffiti, rap, dance, DJing), science, science fiction, game design, film/video, animation, fantasy/dreams, etc. For example, I have been covering the Afro-futurist influence in contemporary art for Art21 (PBS Art21). The artists whose works I write about are peers and friends, for example: http://blog.art21.org/ 2012/06/19/sanford-biggers-conundrum-the-mothership-lands-at-mass-moca and http://blog.art21.org/2012/07/17/cauleen-smith-a-star-is-a-seed-a-seed-is-a-star. These artists are the same age as I am and are on different tracks but we grew up around similar things, especially in popular culture. What inspired me as a teenager learning from Futura 2000 (hip-hop, soul music, etc.) is what inspired them. We are working from a mutual sense or spirit of the times. As an activity I suggest coming up with a theme, then interviewing peers or friends about their experiences or knowledge of it. Make note of what is the same and what is different. Explore either through writing, sketching or both your own ideas that come out of your research/interviews. Find the apparent linkages or connections to science (sci-fi), math, etc.

Laurie:  Argon is a free app available through the App store.  It is currently available only for Apple devices but aversion for Android devices is being developed this year.  Example code can be found on the KHARMA website: https://research.cc.gatech.edu/kharma/examples.  We used the code sample entitled “TrackerFrameSimple.kml” as the basis for the channel we made in Argon.  We have also made a PowerPoint presentation for educators that shows how to code an Argon channel.  This can be viewed on SlideShare: http://slidesha.re/SkeM3y.  To obtain the Frame Marker patterns you must register as a developer through the Qualcomm Vuforia web site:  https://ar.qualcomm.at/. 

Can you share your methods for brainstorming and how you get your ideas?

Nettrice: Watching films like "Inception" or "Prometheus" gives me ideas, as well as my interest in combining the virtual/real: http://blip.tv/nettricegaskins/cry-immersive-art- installation-in-second-life-2437151. Visiting exhibits/studios, interviewing and talking with other artists, and writing for my blog or Art21's blog also gives me ideas. By writing about the art in the film Inception I learned more about deeper themes and topics like "Powers of Ten" by Charles and Ray Eames. The film depicts the relative scale of the universe in factors of ten. For AROS, youth interns learned about ancient Mimbres and mandalas, as well as mathematical concepts such as symmetry, polar and Cartesian coordinates. They used software to create designs, as well. I showed them work by contemporary artists who make use of mathematics, or science in their work such as Sanford Biggers and Casey Warr who reference sacred geometric principles. Casey Warr has work on view in a local sculpture garden. Once students/artists become conscious of these abstract ideas they can name it and see it everywhere. My point. Once you can name something, youʼre conscious of it. You have power over it. You own it. Youʼre in control. This means a lot, especially when working with populations that are marginalized or underrepresented.

Laurie:  I find that my creative goals emerge from what I see and experience within my community, so I tend to think of projects that will serve the community in some way.

Once you had your idea how did you approach the phase of designing and planning for its realization?

Nettrice: When I am creating new work I reference images, or texts and look for other things that reflect what the images/texts are trying to convey (tell the viewer or reader). Artist Dorothea Rockburne explains that “astronomy combines shapes, math, metaphysics and feelings that the cosmos will continue to exist long after the lives of humankind.” The ancient Incas regarded space and time as a single concept, named "pacha." In The Time Machine, H.G. Wells writes,
"There is no difference between time and any of the three dimensions of space except that our consciousness moves along it."

Dorothea Rockburne states: “astrophysicists say that we are made of old stars from the time of the Big Bang. When I look at my hand, I know that I am made of old stars, and that I am a very tiny part of a vast universe.” The idea that the universe is all around us, including us is a very hard concept to start from and this is why I look at Rockburne's artwork as well as Josiah McElheny's Island Universe and whatever else I can find in my research. For the IBM exhibition I started constructing objects that reflected these ideas or concepts. Certain things were combined or interwoven into the structures. This process is similar to what youth participants in AROS began to do when looking at ancient Mimbres and mandalas, as well as contemporary art and artists.

Laurie:  We used a participatory design process in which a community of young people created the work together.  This encourages each person to take responsibility for a certain part of the work, while the whole group produces something greater than the sum of its parts.

Did you build a model or prototype for this piece? if so, how did you decide what to make it out of and can you describe the process?

Nettrice: In Second Life I often build virtual (sometimes interactive) 3D structures that reflect what I am researching. I created a simulation of 'Wildstyle' graffiti in Second Life to explore what happens when a two dimensional graffiti tag becomes three dimensional and interactive. What I discovered is that in order to do this transformation I needed to dance and move in real and virtual space. I invited others to see the work-in-progress and they also wanted to dance. I experiment with textures/images, shapes, colors and in-world scripting to make objects move or the virtual avatar do something. For AROS I helped youth create their own designs based on ancient Mimbres artifacts and mandalas. They also learned how to simulate some of these designs using culturally situated math software (Graffiti Grapher). They learned about math concepts used to create the designs. The final step of planning is to combine all of these ideas and designs. For AROS (and SL Wildstyle) it was a collaborative effort, like 'call and response.' Call and response is inherently connected to the historical African religious roots, which served as the foundation for African American religious activity. Call and response participation closely relates to iterative (repetitive) digital media design (testing, prototyping, participating in the design process). This also relates to the "cipher" in breakdancing and freestyle rapping and the Brazilian martial art Capoeria.

Laurie:  The prototype for this piece was a mobile augmented reality experience that accompanied a mural painted by a high school history class in north Georgia last fall.  In that piece, students worked together to create a design for a mural depicting important sites in ancient Rome.  Frame Markers were placed on the mural in strategic locations to deliver links to information students had placed on the web detailing their research on a particular topic in Roman history and culture.  For that piece, I made four-inch square markers out of card stock and cut them out with an Xacto knife. They were applied to the wall with removable adhesive tape. For this piece, however, I made a stencil of a 42-inch square marker to be painted on the wall.

Did this piece require doing research and if so can you share why it is important and how you go about it? Is there any advice you can offer about this phase?

Nettrice: Research is a huge part of my creative process. In fact, I am studying to become a better researcher of digital media but I also do heavy research for my artwork. My recent batik art explored Hurricane Katrina and I referenced Amadeus Mozart's "Requiem" and Shakespeare, among other things. My biggest work to date has been in virtual 3D environments where I am inspired to think outside of the box. In the virtual 3D world visitors can enter as virtual avatars that fly around and do things that are difficult to do or impossible in real life. The virtual 3D world is perfect for exploring art, science, math, etc. Augmented Reality is also another way to go beyond the real and explore the hyperreal (simulation of reality). AROS participants created an AR experience based on sound and imagery, triggered by frame markers (similar to QR codes) and a mural as an image target. Something as complex as this requires lots of research.

Laurie:  The main research this piece required was to determine the optimal size of Frame Marker to use given the unusual placement and conditions under which the AR content can be viewed.  The AROS mural is painted on the wall of a warehouse facing the rail corridor, so that viewing the mural can normally occur only from a moving train.  I initially tried using 30-inch markers.  The camera is able to track them from 60 feet away, but the display did not seem stable enough to work from a moving vehicle.  I performed another experiment at home with a column of 42-inch markers and drove past it at about 15-20 miles per hour.  The camera was able to track the markers and the augmented reality content was displayed on the device in a moving vehicle.

How did you test and evaluate your design? For example, did it work the first time or were there many versions before the final one? Do you have standard ways of testing your work?

Nettrice: The main difference with virtual 3D artwork is that it often includes scripting objects using Linden scripting language which is very much like web-programming (HTML, Javascript). All of scripting is about trial and error and experimentation. When IBM opened my show in Second Life I received a lot of feedback from visitors that made me go back an change some things. This included tweaking scripts or scripting new objects that transported things someplace else. Again, in virtual 3D space almost anything is possible if you can build it or script it. IBM supported this iterative design process and I continued to modify and build new installations throughout the duration of the exhibition. This process is very similar to how designers create games or how teams create Augmented Reality experiences for mobile devices or even collaborative murals – it's all about being iterative (repetitive) in the design process.

Laurie:  Digital art is by its nature an iterative process.  New ways of presenting information are constantly being developed, new technologies that shape interactions emerge practically every day.  This design extrapolated from a previous work to push beyond the boundaries of usage suggested by the inventors of Frame Markers.  As better technologies for vision-based augmented reality become available freely on Argon within the next year, this type of piece could display AR content just using patterns within the design of the mural itself rather than using the fiducial markers invented by Qualcomm. 

The test and evaluate phase is where we confront our limitations and “failures”. Often we prefer not to talk about them because we are encouraged to only talk about our successes. What can you share with students about the process of “success and failure” that emerges in the test and evaluation phase?

Nettrice: With iterative (i.e. game) design it is important not to start with a completely blank slate. Artists/designers must begin with goals that are ideally paired with some initial concepts that will guide early prototypes. For AROS one goal was to test and prototype an Augmented Reality application of culturally situated designs that, at an early stage, clarifies exactly what users need to learn and how. Participants start with an early prototype or design that works as a real project, if not a very interactive one. Through testing, we assess why certain things work and others do not. Most important, we can see how all of the various concepts interact. Although AROS for ISEA2012 was three weeks in duration we could have used a few more weeks to process, prototype and test out the AR with the mural to ensure that the mural was trackable on mobile devices. The two completed project tracks (mural and AR) never really came together at the end because we ran out of time. Since this was the first time a project like this has been attempted we expected some challenges and we've learned about how to better facilitate the process.


Laurie:  The first time the group coded an Argon channel it did not work at the end of the day.  This was a great teaching moment because any kind of computer code can have bugs.  It can be as simple as a semicolon in the wrong place – a typographical error – that makes the whole program not run correctly.  When I pored over the code that evening, sure enough, there were some typos that had to be fixed.  The next morning I gave everyone copies of the non-running code and the running code to compare.  If something does not work, you have to look at all the realistic possibilities available to fix it.  We had hoped to be able to use the mural itself as an “image target” to display the AR content, but there were two problems:  the design did not have enough trackable “features,” i.e. corners and high local contrast in non-repeating patterns, and Argon 2 is not yet available to the public on the App store. So the next best solution is to use Frame Markers since this is a technology for vision-based augmented reality that currently works on Argon.

What criteria did you use to evaluate your piece, or your work in general?

Nettrice: For AROS there were two project tracks: mural production and Augmented Reality production. The goals for each were to create a project that accomplished different results. Together, the mural would trigger mobile devices to display or overlay Augmented Reality content such as sound or images. We can evaluate this production based on our initial expectations (I had youth write theirs down on the first day). We wanted an audience to have the experience of interacting with the mural through touchscreen, camera-enabled mobile devices blends virtual and physical spaces and results in a greater appreciation for STEM learning, culture and art. This can still happen but it's possible that more work needs to be done to ensure the mural is trackable for AR and the software is able to read such an image.

Laurie:  My goals for this project were to stimulate a desire to continue learning and to model a pattern for how to do so.  The students we worked with were introduced to web programming, computer graphics and 3D modeling.  If they continue working in some of these media, I will consider our work a success.

What do you get from sharing your work with others? This question addresses the greater question of why we create art in the first place? What is its role in society? Why is it important for us to create and share art?

Nettrice: My answer goes back to referencing "call and response" participation and collaboration: I create art to encourage spontaneous verbal and non-verbal interaction between speaker and listener in which all of the statements ('calls') are punctuated by expressions ('responses') from the listener. This can be done via creative Web-based or mobile technologies like Augmented Reality that allows users to interact with images, or objects in physical, or virtual environments. Personally, there's a natural and cultural need to engage in this manner that goes back to the activities of my ancestors. Since I can remember I have sought ways to engage others in discussions about art. It has the potential for social change, or transformation.

Laurie:  The arts (and I include literature as an art) are the primary means by which we become aware of our identity as a people.  We use images to define who we are and to convey our best attempts to comprehend the ineffable.  Regular spiritual practices within a faith community have historically provided both a foundation and a source for artistic creation.  Living within a commitment to each other’s spiritual growth allows one to be receptive to new insights that manifest in response to changes in the world.  Communicating these insights through works of art deepens the commitment to the common good.

Did you have to collaborate to realize this piece? If so why? Is there anything you would like to share about the collaboration process?

Nettrice: This question has been addressed by my previous responses. I co-facilitated Augmented Reality for Open Spaces (AROS) in Albuquerque My project partner
and I were brought in as visiting artists to work with eight youth interns from the Explora museum to produce an interactive, outdoor mural as part of the Wells Park Rail Corridor Murals Project. This community-based project is a collaboration between 516 ARTS for the ISEA012 conference and Albuquerque’s Public Art Program. We also received support from the STEMArts (science, technology, engineering, mathematics and arts) program. Our aim was for an audience to have the experience of interacting with the mural through touchscreen, camera-enabled mobile devices blends virtual and physical spaces and results in a greater appreciation for STEM learning, culture and art. As a result, no one person or artist is responsible for the success or failure of the project. We all play a role in it's success (or failure) and share in the learning process.

Laurie:  Collaboration succeeds when each member of a collaborative group does his or her share of the work and when realistic and honest goals are articulated.  With a few exceptions, I have found that teamwork is a good way to achieve any goal, whether it is a work of art or making a home run in a ball game.  This particular piece required the cooperation of several organizations, two artists, and eight youth interns.  We also received help from other members of the Albuquerque community who were not part of the official collaboration but whose help was invaluable.  In particular, I would like to thank Prof. Diane Torres-Velasquez of the University of New Mexico for providing space for me to work late at night to do research that was vital to the success of this project.

Do you come from a STEM background or an Arts background? What is the STEM skill or concept that inspired or formed this piece, or your work in general?

Nettrice: My current research: Interest in culturally situated STEM learning is growing. Researchers posit a dynamic interplay of four factors that shape competence – cultural artifacts, cognitive domains, interpersonal contexts, and individual schemata – and link culture to competence within everyday life activities. Thus, there is a need to develop a framework built upon existing theories to bring
together clear and growing findings on the development of how people learn in a cultural context. There are several examples of creative works with an ‘ethnomathematics’ component that have not been made clear for STEM learning. Using "A Geometrical Bridge Across the Middle Passage: Mathematics the in Art of John Biggers" my advisor Dr. Ron Eglash links culturally situated artifacts in Biggers' mural to mathematics principles such as geometry. This research is what inspired AROS, as well as learning about cultural heritage Augmented Reality projects at Georgia Tech.

Laurie:  My Ph.D. is in a liberal arts discipline, but before I studied philosophy I studied organic chemistry, calculus and biology.  I have worked in a medical research laboratory and I am now engaged in the study and practice of computational art.  Each discipline has specific methodologies and key practices that one must perfect to develop expertise, but there is a common thread of basic intelligence and critical thinking that accomplished people in all disciplines share.  When you are placed in a new situation, you draw from your well of experiences to discover the best possible actions.  This particular art piece draws from experiences with computer vision which works by converting electromagnetic radiation (light) to voltages on a sensor which become signals processed by a chip.  I did not invent the chips or the sensors or the lenses that make all this happen, but as a human being living in 21st century America I have access to these things and to the technologies that have been developed that allow these things to be combined to make mobile augmented reality experiences.

Has working with science and technology improved your professional career or life and if so how?

Nettrice: My younger sister is a chemical engineer and my mother was a computer programmer (both are black American women). Thus, I am an artist, educator and researcher with a STEM background. In high school I majored in visual arts and a teacher convinced me to take a computer graphics class. Until then I did not know that the same computers my mother used at work could also be used to make art. This class lead to a full tuition scholarship at Pratt Institute in New York City and later grad school and Georgia Tech where I am exploring how digital media (and art) can be used to engage learners from underrepresented minority communities.

Laurie:  In midlife I decided to return to school to study digital media.  The past two years have been a period of rapid growth and consolidation of a new skill set, resulting in an expansion of opportunities for creative work.

What was your experience with STEM in middle and high school and what would you change if you could?

Nettrice: I loved chemistry and biology but hated math in middle and high school. It wasn't until graduate school that I become interested in mathematics, as a bridge to learning and art production. If a teacher had made this connection earlier in my schooling I would have been more engaged and positive about mathematics and technology.

Laurie:  I loved science and excelled in mathematics in middle and high school.  My favorite class was Advanced Chemistry, in part because of the camaraderie we shared in the lab in the afternoons.  I feel that experimentation is very important especially at that age – you want to learn by experiencing things for yourself – but it is also important to develop a disciplined process for making sense of those experiences.  Working in small groups is perhaps the most effective way to convey how a disciplined process can yield scientific understanding.

How has your creative work influenced your use of technology and/or how has technology changed how you work or the pieces you make?

Nettrice: I've been using computers to create art since my senior year in high school and that over half my life. As an undergraduate I majored in Computer Graphics, as a graduate I majored in Art and Technology and as a Ph.D. student I am using research into culturally situated design for art and learning to bring these areas of study together. It also informs my artmaking. My mother was a computer
programmer/analyst, so I've been around technology all my life but was only interested in it as a tool for making art. Now I move in and out of the virtual environment into physical/real environments and I am comfortable working in both worlds.

Laurie:  Whereas I used to express myself primarily by writing, now I think of ways to create a web-based design including text, images, sounds, video and interactivity to convey my thoughts.

Can you share any differences and/or similarities between artistic and scientific creativity that you may have personally explored by uniquely merging the two in your work? Or you may choose to share the more general question of what the arts and sciences have in common, or differ?

Nettrice: Author Leonard Shlain proposes that the artist is the first member of a culture to see the world in new ways. Then, nearly simultaneously, the scientist discovers new ways to think about the world. Shlain explores the classical, medieval, Renaissance and modern eras, to show how art when superimposed on scientific concepts create a compelling fit. The final chapters explore possible reasons why these connections occur. The split brain phenomenon and Greek mythology are used to explain Western culture's division of the two seemingly disparate fields of art and physics. This book was instrumental in encouraging me to explore STEM as part of my research. I've superimposed some of this work with the cultural and creative technology studies.

Laurie:  Both art and science involve discipline, and both result in a kind of invention or discovery.  Artistic creativity seems oriented more towards expressing some truth about ourselves, while scientific creativity is oriented towards increasing our systematic understanding of nature.

How do you think artists can benefit from science / scientists? and/or visa versa, how do you think scientists/science can benefit from artistic creativity?

Nettrice: By making the connections more explicit (clear), i.e. in the language of the specific discipline and giving professionals from these disciplines more opportunities for exchange and collaboration there could be more integration. In a recent blog entry for Art21 I mention the contemporary “spirit of the times,” in which artists and mathematicians, for example, intermingle different perspectives and creative ideas to produce new spaces for production and discourse – see Cai Guo-
Qiang‘s and Hiroshi Sugimoto‘s contributions in Mathematics: A Beautiful Elsewhere. In a way, the arts and sciences need one another to progress.

Laurie:  Artists can take scientific ideas and ask how does this understanding of nature reflect something within human beings or change our vision of who we are.  Science can give artists new tools to work with and a stimulus to the productive imagination.

Do you think the arts are as important as science? If so why, if not why not?

Nettrice: Of course! I am an artist, first and foremost. As Leonard Shlain argues the artist is the first member of a culture to see the world in new ways. Social change and transformation without vision is dangerous. We need art to connect the mind to the soul.

Laurie:  The process of creating images that define and express our identity as a people is every bit as important to society as the process of formulating theories about how nature operates.  Art enables us to come to know ourselves; it generates reflective moments in which we may see something like what God sees of our inner life.  The scientific endeavor takes us deeper into the mysteries of nature and gives us the ability to create technologies that make life more convenient.  Art opens us up to the mysteries within ourselves and makes it possible to accept life as it is.