Creating Augmented Reality Content

As part of our project, Shared Spaces has been working on developing an augmented reality (AR) application. We are currently in the prototyping phase, learning and observing how members from our communities respond to this type of technology. For those unfamiliar with how AR works, here is a brief explanation: Upon opening an AR app, the user first scans a “trigger”. This can be a photograph, scanned image, QR code - basically any digital image file that is visually unique. The app reads this trigger and activates some other form of content; this can be a video, animation, link, etc.

Shared Spaces was fortunate to be invited to share our prototype at the Canadian Crafts Federation conference, Ten Digit Technology, in collaboration with Ruth Cuthand. Ruth is a Saskatchewan artist of Plains Cree and Scottish ancestry and 2020 recipient of the Governor General's Award in Visual and Media Arts.

In preparation for the conference, we decided to employ Artivive, a free, pre- existing AR application that allows users to link content to their artwork. The process of adding content can be as simple as linking the trigger image to a pre-existing website, or something a little more complicated, like creating an animation or video. With the right skills and tools, this process can be very efficient. The programs that I utilize for creating quick animations are Unity and Adobe After Effects; I like that both programs have hundreds of free tutorials online.

For Ruth’s first piece, a stack of Canadian Forces blankets beaded with small pox viruses, we decided to create a short video of Ruth speaking about her work. Lisa Birke, Michael Peterson, and I filmed a brief interview with Ruth using a green screen set up in Lisa’s office. Lisa edited the video while I animated an image of smallpox viruses to be placed as Ruth’s backdrop. The next day the video was complete and uploaded to Artivive along with its trigger, a photograph of Ruth’s smallpox beadwork.

The next piece we augmented was Ruth’s beading of Hepatitis C. We decided to create a 3D animation of a rotating Hepatitis C virus to provide a different perspective than what Ruth had beaded. We selected a pre-existing model that resembled the protein-covered surface of the virus. The rotating animation was created using Unity, an open-source game developing engine. After I imported the 3D virus into the scene, I Googled some code for continuous rotation and pasted this into the C#script window, which I linked to the object by dragging and dropping. C# is the programming language Unity uses to control actions. This could be complicated if we were the ones writing our script, but Luckily we were not; someone with a Comp Sci background has graciously done it for Unity users. All I had to do was Google “Unity 2019, rotation C# script” and, once I found what I was looking for, cut and paste the text. This sometimes requires a bit of trial and error, as there are multiple ways to do the same action and various individuals writing the script. If code is unavailable to directly copy and paste, tutorials on platforms like YouTube provide step by step guidance, showing users how to input text into the script window.

Since we wanted to create the visual effect of the virus floating in front of Ruth’s beadwork, our next step was to create a single-colour background to make use of Artitive’s Chroma Keying Feature. Chroma Keying is a visual effect used to remove a flat colour background (the most common example is a green screen) and replace it with a transparent background or some other footage. For our model, since the 3D virus was yellow, I made sure to select purple (it’s complimentary colour) for the background to make chroma keying easier. I recorded video from within the Unity game window and imported this file into After Effects to create the correct aspect ratio (Artivive suggests using the same aspect ratio for the trigger image and linked content). I then uploaded the trigger image (a scan of the beadwork) and the video to Artivive, which allowed me to chroma key directly within the app; I simply used the dropper tool to select the colour that would become transparent. Now when the trigger image of the beadwork is scanned, a rotating model of the virus can be seen floating in front of Ruth’s work.

Written by Lauren Warrington, Development Team Member