[00:00:00] SY: Big announcement! Early bird tickets for our Codeland conference are now available. You can get them at codelandconf.com. It's our second annual conference happening in New York City, May 4 and 5 of 2018. So get your tickets while supplies last. Link is in your show notes. (Music) Welcome to the CodeNewbie podcast, where we talk to people on their coding journey in hopes of helping you on yours. I'm your host Saron. And today we're talking about augmented reality.

[00:00:35] Earlier this year at WWDC2017, Apple introduced ARKit, the augmented reality platform for developing augmented reality apps, and once it was released developers started playing with it and built fun artistic and sometimes even useful apps and demos. One of those fun artistic demos was created by Zach Lieberman.

[00:00:55] ZL: Hi my name is Zach Lieberman. I'm an artist, educator, and also co-run the studio YesYesNo.

[00:01:02] SY: He built this really cool thing that records audio visually. In the video of the demo (the link to that is in your show notes) there's a person beatboxing and every noise they make creates this burst of white lines, almost like a strange cloud. As the beatboxing continues, there are more bursts, more clouds creating a visual trail of sound in the space in front of you. But that's not the cool part. The cool part is what happens next when you walk through those clouds tracing that trail of sound with your phone. The sound plays back. Those clouds aren't just artistic - they're actual recordings. It's one of many AR demos Zach has created over the years but he doesn't do it alone.

[00:01:44.07] MK: Hi my name is MolMol. I'm an artist, I'm a researcher, and I'm a partner at a studio called YesYesNo.

[00:01:50.11] SY: And they're not just coding partners. They're life partners, making them the second coding couple of the CodeNewbie podcast. And today they're going to give us a newbie- friendly introduction to augmented reality. After this.

[00:02:06] Flatiron school teaches you how to code from anywhere. They've got an awesome community of career-changers, and a number of different options you can pick from to become a software engineer. They've got full-time in-person courses, self-directed introductory courses, and remote online web-developer programs. They even have a free, 75-hour online prep course, so you can dive into JavaScript, Ruby, and interview prep. To learn more, go to FlatIronSchoolNewbies.com. That's FlatIronSchoolNewbies.com. Link is in your show notes.

When I first learned to code, all I wanted was to be a developer. But then I actually learned to code and realized that you don't become a developer. You become a front-end developer, or a Rails developer, or a full-stack engineer, or a back-end engineer, or the million other job titles that involve coding. So how do you pick? And once you get that first job, how do you turn it into a career? You can use the Dice Careers mobile app. This is the tool I wish I had when I first started. You pick the tech skills you either have or hope to have in the future, you type in your designer job title, and Dice helps you find other job titles you might also be interested in and maybe didn't know about. But they take it a step further, by telling you what skills these job titles require, how much they pay, and based on your profile, they tell you what skills you might want to learn so you can one day apply for those jobs. They simplify a lot of the chaos of job-hunting, and it's totally free. So check out the Dice Careers mobile app. Go to dice.com/CodeNewbie for more info. That's dice.com/CodeNewbie.

[00:03:35.05] And what's really cool is one of you is more of like the software person and the other one is more of the hardware person -is that right?

[00:03:39] MK: Yeah we kind of just moved towards the things that we love to do most. And I happen to love physical computing - electronics and sensors - and he happens to just be really good software-based things.

[00:03:52.29] SY: Man, you all are such a great power couple. Okay so MolMol tell me the truth - what is it like to work with Zach.

[00:03:58.02] MK: Oh my God. Zach is good to work with because he's the type of person who you know that can solve a problem. This is the kind of person that I enjoy working the most, it's people who take joy spending hours and hours just to solve the problem.

[00:04:20.29] SY: Ok, Zach it's your turn. Tell me the truth.

[00:04:23] ZL: So what I love about working with Molmol is that she's really tough on me and I know when I've made something good when Molmol likes it. She is really that honest and I think it's amazing to work with people who are really honest and give you direct feedback and we really have enjoyed working together for over five years now.

[00:04:43] SY: So is your working together you doing something and then she'll give you feedback, or she does something and you give her feedback, or have you collaborated from start to finish on a technical project?

[00:04:54.22] MK: Most of the projects we collaborated from beginning to end we have also have some personal projects.

[00:05:03] ZL: We have a studio so we do you know when projects come in, we talk about them we talk about the technologies that we want to use the directions or approaches we'll go in for our clients, and we work together on all the sort of aspects of the process of making work, whether it's a public art commission or commercial work. You know we do a variety of things.

[00:05:22] SY: Do you ever get tired of each other, because you were together, you have a company together, you build together and then you had to go home to each other too. Right? So does that ever get like a little old?

[00:05:32] MK: Of course we get tired of each other all the time.

[00:05:37.17] SY: There is that honesty. I love it.

[00:05:41.07] MK: But you have to find someone who you get tired of but still want to work together with.

[00:05:44] SY: Exactly. Not someone that you're fed up with to the point where it's like OK I just I can't see you anymore. So we are here to talk about augmented reality and I know you both do a lot of different types of tech, use a lot of different tools. How would you define augmented reality. What is that?

[00:06:00] ZL: I would say augmented reality is basically taking some sense of space. So in the most common case are the things that people are used to, it's having a camera and knowing where you know using maybe a marker or a markerless system to understand where the camera is in space and then adding graphics, but it doesn't necessarily have to be visual, it could also be auditory. But I think it's about kind of a spatial awareness and that people have a view into the real world but then an augmentation on top of.

[00:06:36.23] MK: It's like a piece of technology generated image or video that's being put on someone's view of the real world.

[00:06:45] SY: If someone hears about augmented reality they may also think of all the other types of reality. There is virtual reality, there is mixed reality, there's probably some other ones I don't remember - what is different about AR compared to the other realities?

[00:06:59] ZL: I think with virtual reality I mean, you're wearing a headset. You know and you're in another world. You're not connected to the world that you're in. You're really you know disconnected because virtual reality is sort of all-encompassing.

[00:07:11.26] MK: So far in virtual reality you besides the glasses you can wear, you can also add a lot of elements into it, like sensory touches. With augment, you can add like vibration, you can add wind, you can add all kinds of like not visual aid to your VR experiences, but for AR so far we are exploring really kind of screen-based type of experience.

[00:07:46.20] SY: Sounds like in some ways and might be a little more limiting.

[00:07:49] ZL: I actually think AR is in some ways less limiting, like right now VR requires a really massive investment in hardware. So you have to have a fancy computer. There's a big investment and the nice thing about the stuff that we're exploring with ARKit or using devices that you hold in your hand is that it doesn't cost that much. So you're able to explore the same ideas without a huge outlay of money.

[00:08:16] MK: Yeah I would really say AR gives the kind of excitement, like me as an artist, is more dramatic than VR.

[00:08:26] SY: So I want to talk about a specific project, Zach, that you made and this was this is really awesome because my husband, you know was looking at different AR stuff and he came across this thing you made that was about audio and translating audio into visuals into AR. And I'm going to have you explain it better than I just did. Tell us about what that demo did and what it looked like.

[00:08:48] ZL: So, I mean to back up a little bit, Apple announced the kit at a WWDC last year. Over the summer I was like kind of dismissive of it, but Molmol kept saying you know we should try AR, to really push me to take a look at it and start experimenting with it. And we came up with a bunch of demos at the beginning that were just about - what does it mean that when you have a screen, a microphone, a speaker, and a camera in space, and what kind of experiences what sort of interactions. what kind of creations can we make when we know where these things are in 3-D? And so a lot of the early experiments that we made were about photography or shooting video in space. And this one was an extension of that, which was, could we record audio in space. Then the next step was okay could we actually play it back. You can imagine like as we're speaking or making sound you know and moving the iPad through the room, you're drawing like actually making a drawing the same way you would draw like with Tilt brush or any kind of drawing in space program. [00:09:54] Then the next step for us was to really figure out you know how do we replay it, how do we move through it, and develop software to scrub the sound. So basically like a D.J. would take a sound on a turntable and kind of scrub through it, it's the same approach where you would move through the sound and as you get close to a sample of audio that you've made, it replays that so that you can kind of play it in forwards or reverse. And for some reason a bunch of beatboxers reached out to us and we had some beat boxers come to the studio and kind of play and jam and it was like a really nice day and I think these demos have captured a lot of people's attention just because people are excited about the technology, but a lot of the initial demos of ARKit were - they were not imaginative and they were more like, you know, demos. And we were trying to make poetry.

[00:10:46.08] SY: Very interesting. OK so Molmol when you saw this idea, what do you think, what was your reaction?

[00:10:52.13] MK: My background’s in film and like for years like filmmaker has been dreaming about this kind of AR experience where you know you can take the narrative and put the audience in a different space in a different city and still kind of experience the narrative. I just really wanted to try it.

[00:11:20.23] ZL: Yeah, in some ways it feels like we're doing a lot of experiments but we're we're trying to develop a grammar. Yeah so they're like really simple but they're just trying to figure out a grammar of like, okay we put an object in space, now if we get close to it, can it change? The closer we get you know how can we draw in space.? Like what are the UI questions? There's a ton of unanswered questions that we're trying to develop a grammar for, that would allow us to then use in different ways.

[00:11:50] SY: So when we talk about best practices for augmented reality, who gets to decide that? Is that just you know everyday developers like you all and doing demos or just putting your work out there to the point where you kinda end up being the you know the authority figures so to speak on best practices or is there, I don't know, like an AR governance body - who makes those kind of decisions?

[00:12:23] MK: For us, it takes a lot of kind of consideration into building a piece of installation, interactive installation, that is going to put out on the street or in our festival or you know in a children's festival where thousands of kids are going to play around. It takes lots of philosophy about how we should play with children, or not how we should do something, iti's philosophy about how why we are doing this and what kind of experience we're trying to provide and what kind of interaction that we expect people to have, and what do we want people to walk away with. So it's not really just, it's not, it kind of goes beyond computer science and moves into social studies, psychology, philosophy. A lot of science and a lot of just like, how do we make something beautiful and enjoyable.

[00:13:18] ZL: Also it's, I say, like there's, you know, Apple puts out human interface guidelines and they have some guidelines about AR designed to be, you know, help app developers or people that are interested in creating software in this space. And I feel like it's both like early days but it's also been around for a long time. It's new in that there are these toolkits that make it easy to get started with it but it's been around for awhile so we've had marker-based AR systems you know since, I don't know the late 90s. And we've had this kind of markerless system which is based on tracking visual features, and in the case of ARKit and ARCore, it's combining this visual system and also using the accelerometer, that IMU, the sensor that you know senses the orientation of your device. It's combining those. But that's, that's been around for 10 years. So there are examples of projects and artworks and the exciting thing for us is that we've been experimenting with these ideas for a while. I remember participating in an exhibition at Ars Electronica around AR in 2002. You know, so it's been in art festivals it's been in academia, but now it's reaching product and now it's in our daily life. And that's where I think things get really interesting.

[00:14:46] SY: Yeah and even, you know, when I think about things like SnapChat filters, I didn't really think about that as AR, but that also is leveraging AR and augmented reality and that technology to put bunny ears on your head and things like that.

[00:15:01] ZL: And I think that also speaks to I think one of the powers of AR, is that it also it works for very short videos and for expressing ideas, like that the experiments that we're doing we're putting them on Instagram and they're short movies like 30 seconds or so. But you can communicate the idea really quickly. It's easy to understand. And it's the same thing for the masks that you see on SnapChat or Facebook that they're you know augmenting your face they're transforming your face. And what I love about those experiences is that they're really performative. And you see people like, they look away and the mask disappears and then they look towards the camera and the mask appears and there's like there's a game, there's a dance, and then they make a short video and they can share it with other people, and I think AR presents a lot of opportunities for magic, for creating moments of magic, and that are shareable. And you know I think we're constantly as a culture trying to find new ways of visually expressing our feelings, and that's why things like emoji are so popular, or stickers, or you know, gifs. Like how do we capture our feeling, our essence, and share it with other people? And this technology I think allows us to play and express ourselves in new ways.

[00:16:14] SY: Yeah. When you think about using AR to tell a story how does that work? Give me give me an example of what that might look like.

[00:16:24] MK: I started to develop this concept of using AR to build a new lens and kind of a new tool to redefine our physical presence in public space in more of a whimsical and poetic way. What we're doing is we're experimenting some kind of life experience where I can film a person and I can create a character while filming. How can we build a life experience by manipulating a person's body or things around a person's body that a person can create in that moment in time and kind of visually present it.

[00:17:11.03] ZL: It might be good if you can explain why we're developing this tool for the project that you want to do.

[00:17:16.07] MK: So illegally if you're an immigrant you're an alien, right, if you're not a citizen, you're an alien. And yet to kind of prove that I can work as an artist and just please let me stay in this country as an artist, I have to prove that I have extraordinary abilities. So what it does I have to provide a lot of truth to say that I am a legit artist and please let me stay. In 2017 this becomes like a difficult year for immigrants and I wanted to kind of visualize that kind of feelings with AR. How do we present this kind of a feeling of alienation, the feeling of being an outsider in a society, or just walking down a foreign street. And it's a feeling not just as an immigrant, but it's a feeling sometimes we all have when we walk into a party or walk into a new place with strangers and we kind of feel like a little bit different, we feel a little bit strange and don't feel like we belong there, we don't know. And those are the kind of emotions that's kind of common. And I'm wondering if I can visualize that. And I think AR is a really good tool to kind of visualize emotions in space. You can do a lot of things with that, you can use drawings, you can use computer generated images, and and just kind of visualize that experience. think it's the project that we're working on right now.

[00:18:49] SY: I love that so much. OK, so let's say I have the app, I have the the alien experience app, and I'm holding it up and I'm walking down my neighborhood street with that app. What are some things that I might see or experience that give me that feeling of isolation and that feeling of what it might be like to be an immigrant?

[00:19:11] ZL: We're building tools for filmmaking but adding kind of live augmentation on top of, you know, as you're filming. So that involves doing things like face tracking or tracking bodies moving in space and adding augmentation on top of that. So, we've been exploring different visual looks to explain the feelings that MolMol was describing, and they are driven either by understanding where your body is in space or where your face is in space, and there are really interesting problems to solve. Using AR where you have, you know where the camera is in 3-D, then if we know where the face is, how do we combine that, and you know there's interesting things to figure out there.

[00:19:56] SY: What's an example of an augmentation that I might see with this lens?

[00:19:58.20] ZL: Swarming particles around your face, you know seeing your body kind of explode out 3D as you speak, seeing your voice in space. We have a bunch of different looks that we've been developing and that we're continuing to develop. So we're sort of in the early days of this project.

[00:20:13.28] MK: The idea is to have some kind of particle system that can represent that type of alienness that comes in and out of your skin or surrounded around your body in 3-D or come up from their face when you speak or when you move your head around. But the idea is to have the lens pointed at you so the camera is pointing at you, and not so much as like looking outside and you're not the person behind the camera anymore. You're the person in front and examining yourself and your feelings.

[00:20:47.12] SY: Coming up: we get a bit more technical in creating augmented reality experiences, and where tools like ARKit fit into that process. After this.

[00:21:01] Getting a job is one thing. Building a career is another. With Dice, you can do both. They've been helping tech professionals advance their careers for over twenty years, which means they have a ton of data and tools to help you navigate yours. Looking for a job right now? They've got thousands of positions available from top companies, like AT&T, Dreamworks, Adobe, IBM, and Dell. Wondering what's next in your career? Use their career pathing tool to find new roles based on your profile and see how much more money you can make. Not sure if you're getting paid what you're worth? Use the dice salary predictor to see real numbers on what your skills are worth. Don't just look for a job - manage your career, with Dice. For more info go to dice.com/codenewbie.

You keep saying you're going to get serious about learning to code, but where do you start? Flatiron school's got the perfect thing. They're offering their free, 75-hour online prep course, where you can dig into JavaScript, Ruby, and more. If you're not sure where to start, start there. And when you're done, you can keep learning, with their self-directed introductory courses, remote online web developer program, or full-time in-person courses. Whatever your schedule, they've got options to help you reach your coding goals. To learn more, go to FlatIronSchoolNewbies.com. That's FlatIronSchoolNewbies.com.

[00:22:12] So, going back to the more technical aspect of creating some of these demos, how much of it is leaning on or using ARKit and how much of it are you having to create and come up with on your own?

[00:22:33] ZL: Yes, so ARKit basically gives you positional information, so when you run an app you instantiate and a session with ARKit and it tells you the position and the orientation of the device, and at the beginning, it doesn't know where it is and you sort of wave it around it it finds visual markers, and and then it'll start telling you kind of in space where it is and it's all relative. So where you started as the center of the world and as you move it, it gives you numbers which are basically how much it's changed or how much it's moved. For the demos that we're using, we're using an open source tool called openFrameworks, which is a C++ tool. If you're familiar with Processing, it's not dissimilar to Processing. It's basically a framework. But more than that it's a community of people that share code and ideas and add-ons and samples and whatnot. And there's an add-on that somebody made for ARKit. So I am used to openFrameworks. I helped create it and I've been working with it for a long time. And so for me it's quite easy to sketch in openFrameworks, but there are other tools that wrap ARKit, so you could be programming directly in Objective C or Swift. You know if you were just doing kind of straight iOS development, or you could be using Unity, or I'm sure there's other tools to talk to ARKit. But the tool that I've been using is OFX ARKit, which is an add on that wraps ARKit, and it gives you positional information.

[00:24:10] You can also place anchors in space so you could say like this is a spot that I care about, and then you know, it could store that information, kind of where that anchor is in space. But then I've been writing code on top of that to either draw geometry or create different interactions. The main thing is the API, or the what you can do with ARKit is quite limited in some ways. It doesn't give you a ton of information, it basically just gives you position. You can also detect horizontal planes which we haven't experimented with but a lot of the demos where you pointed out a table and then it finds the table and something happens, like those use the, there's a horizontal plane detection. I guess when planes are detected they get reported back to you. I've seen examples where people, like there's an AR example where somebody solves Sudoku puzzles, and that's using like - Apple has a rectangle detector you know, to find rectangles in an image, and then combining that with AR to you know make something happen. But in general the API is super limited, it doesn't do that much, which actually makes it quite easy to get started with.

[00:25:21] SY: Interesting. OK. So it basically, when we think about the construction of an AR app, we have ARKit which we need to give us just the raw data information, but once we have that it sounds like it's up to us to write the actual app that determines what to do with that data.

[00:25:39] ZL: Yeah definitely. I mean it's it gives you more than position, it's like position orientation, it's basically a matrix. It's just a box - it gives you a box of numbers. It's a box of numbers and then you have to figure out like okay, that means that I'm you know in this point in space. And I mean, there's a lot of stuff that we had to figure out. You know we had to figure out, if I touch the screen where's that in space. The ARKit tells me where the camera is in space but if I want to build a drawing tool, I have to figure out like when I touch the screen, where is that in 3D space. And there were a lot of like little challenges to - we still have a bunch of questions that we have to answer.

[00:26:19] SY: It sounds like there's a lot of pieces, and frankly there's more pieces than I thought because I kind of thought ARKit would give you a little bit more than just that, for some reason I assumed it would have like some really neat API that I don't know, like an endpoint called "put on rug" or something like that. I think that's what all the newbies were hoping for. But given that it's a box of numbers it sounds like I have to kind of understand space.

[00:26:50] ZL: Yeah. I would say also if you're programming on the iOS side, there are a ton of examples that combine ARKit with something called SceneKit, which is Apple's API for working with 3-D graphics and setting up scenes, and you could think about it as this tiny tool that does one thing really well but then you can combine it with other things.

[00:27:10] MK: Okay so, what the difference is we're using ARKit with openFrameworks, so we basically have to write our own library and after that we have to create whatever we wanted to create, creatively with our own. Like, we have to write code. But there are other plugins, like AR plugin for Unity, which just gives you kind of all the functionality that ARKit provides for now and then you can use Unity, like without writing so much code you can build scenes or you can build AR experiences with Unity. It's just kind of, we're using different tools here.

[00:27:50] ZL: I think as a beginner the simplest way to think about it is, you could think about it even just as a camera, right. That it's like a camera information. The way you would have a camera in a 3D game, you can think about it as a point in space but you could also think about it as a camera, so you put a 3D model at some location and then as you move around you're basically changing the camera view and then compositing that on top of the -

[00:28:11] MK: But also if someone's really wanting to get into using AR with processing, or you know P5JS for web, there are also a lot of documentation on line to work with these tools.

[00:28:24] ZL: There's also Web AR. I think the interesting thing is that these, there's a bit of a kind of arms race down between Google and Apple around AR and on mobile devices. And then there's a secondary race between companies like Facebook and SnapChat and Google that are like, have products that were living in all the time that also have AR components and that just means there's going to be a ton of things being built around this technology and a bunch of platforms and ways of entering. So I don't have any experience with web AR, but that's another way of doing it where it's combining AR and something like 3JS. [00:29:09] And so if you're comfortable in JavaScript you can develop demos and experiments that way.

[00:29:12.03] MK: And you would just use a browser so you wouldn't really need to download any apps. It's important to first identify what kind of project you want to build, like what kind of AR experience you want build, and then from there you can kind of look for clues as to which platform or which software that you can then use to develop that.

[00:29:38] SY: Yeah exactly, and that was actually going to be one of my next questions, about the process. You know let's say you have a really cool AR idea but you're new to AR, you've never really worked with that platform, that medium before, and you're not very familiar with the tools. What's a good place to get started? How do you begin bringing your idea to life?

[00:29:59] ZL: I think it's about finding community more than anything. So let's say you have an idea something that you want to make. I always recommend to students that you catalog like all of the cool things that you see, things that you're interested in, projects, examples. You got a sense of the different tools that are out there and that you search for a community, like a really active and friendly community, and I always recommend students just go on if it's if it's a tool like openFrameworks or P5 or 3 or you know one of these tools, you jump on the forum or a mailing list, you ask a question, and you just get a sense of like what does it feel like to be in this community. Because I think these tools more than anything are places for people to share ideas and share resources and work together and then it's just much easier to do this stuff with other people. And that we're all figuring it out. I mean for example with ARKit and openFrameworks, like there's a group of us that have all been just kind of developing and proving the add on and sharing what we learn and working together and learning from each other, and it's just much faster when you're with other people.vSo yeah that's probably my first advice.

[00:31:08.17] MK: Coding needs to be social.

[00:31:12.13] ZL: Totally.

[00:31:12.26] MK: There need to be like physical social coding sessions happening in your neighborhood. If there's no, then you should start one. The other thing that I always like every time I teach a class, and I always ask my students to begin with research, doing research on the project that's already existing and doing research on the idea that you have. Look around and see if it has been done before, and what are the kind of different style or format that it's been done, it's been used. And from there you can kind of figure out if you're really into this idea, what can you do, how can you do it, because you've done your work doing research.

[00:32:01] SY: Yes. And I feel like AR feels like such a naturally collaborative medium because you can include filmmakers, you can include artists, you can include software people, you can include hardware people too, because after all hardware's still using phones and in the future we might even get rid of phones or just use glasses. So it feels like a really great playground to bring in all kinds of people from different disciplines and skill sets to make something really cool.

[00:32:29] ZL: Yeah I mean one of the really exciting things about putting these demos has been just the number of people who have reached out who are curious, who really want, you know filmmakers and musicians and exactly what you say, like people that really want to explore what this means for their discipline and how they can use it to tell stories or make things in new ways. And the other thing I would say is that although these things have been around for a long time, it's really quite early days and I think there are sort of core technical problems that need to be solved for this stuff to take off, like you know where it's going, where it's going to be in six months or a year five years, ten years. And it's an exciting time to be working in this space, it's really exciting because you know right now it might seem like a lot of the apps that you find in the App Store, you know not childish but like gimmicky - it's early days. But I have a fundamental belief that like it's going to be really important, and a really important part of computation.

[00:33:28.26] MK: Yeah, for a long time, artists have been kind of treated or be seen or used as like a R&D department for technology and sometimes for technology company. They create a new tour and a new piece of technology, they kind of release it a little bit, and then artists and technologists get excited, and then they hack it the way and then build things on top of it that you know sometimes people who build it or the technology company who develops it have never think of. So it's really interesting to see technology company when they creating a product - what if they involve artists in the process before they release the product. Maybe they can open up their production and invite people from different backgrounds professionally and have more diverse background to join the process of creating a product. If these kind of products are going to affect our daily life.

[00:34:40] SY: So what advice do each of you have for people who hopefully are excited about AR after listening to this episode. They're looking to get started. What advice do you have, what are some things that maybe you wish you knew when you first dabbled in augmented reality?

[00:34:55] MK: I started to learn programming in a very kind of, a really diverse environment where there are definitely advanced programmers, designers, and there are also lawyers and writers and artists, and so that is really encouraging in a way, where you know, we are just people learning how to do this thing. And as an artist I have to sometimes, you know, learn specific skills in a very short period of time just to be able to make this one specific project. So it's really, really helpful to be able to talk to different people while you're learning something, and just kind of get a general sense. It definitely helps than you know like staying in your on your desk by yourself staring into your computer into the void in the middle of the night.

[00:35:41.10] SY: Zach, what about you?

[00:35:46] ZL: I would say that it's really helpful to take a lot of notes and to draw a lot and to try to engage with paper. I ask my students on the first day of class, I ask them to write down all the questions that they have. And it's really beautiful to just sort of articulate your questions, to write them down. And the school that I help run, we do a lot of work on paper and I think that paper is kind of under under explored as a means of teaching or learning programming, that we spend so much time on the screen and we spend so much time like on the Internet, or like just kind of engaging with it with a screen which is kind of antisocial. Programming is filled with layers and layers of abstraction and that when we're drawing when we're using metaphor when we're trying to break things down and explain them in playful imaginative ways, it's a way of combating abstraction.

[00:36:41] SY: I'm a huge fan of paper as like a very reliable and really helpful tool, especially when it comes to more creative planning strategy stuff, so I feel you on that one.

[00:36:51] ZL: I think generally it's under-utilized for learning code. One thing I recommend to people is to actually print out code which can be a challenge, like to figure out how to format it right. But just print out a piece of code that you're interested in and just take highlighters, like mark this is what I do understand and this is what I don't understand and draw on the paper and engage with the text not on the computer screen. I feel like when you're beginning as a programmer, you can't see all the code at once, you can't see - you don't get a sense of context, so it can be quite challenging the longer that your logic gets. the harder it is to see it, and when you work on paper, it allows you to see this as text, you're using text to encode logic. It just feels really useful to have another way other than the computer screen to engage with that.

[00:37:33.23] MK: For people who are interested in physical computing, it's also a really intuitive way to actually learn the code because when you're writing a piece of code you can then see the output as a motor spinning in different directions or something is moving or that kind of physical output that can help you understand what your code does.

[00:38:05] SY: Yeah, makes sense. So how hard was it for you all to actually learn augmented reality when you did the very first demo, first looked into it - was it was it hard for you personally to get that up and going?

[00:38:20] ZL: First ever thing I did a long time ago was in 2002 - you would wear these glasses, they were like VR glasses but we did cut holes in them so that you could see through them. Very like primitive Hololens, I guess. We had magnetic trackers so we could figure out where those devices were in space and when you said something, your voice came out like a out of your mouth like a noodle and it was amazing it was a table where you could have a collective hallucination. Six people could sit and wear these goggles and it was 2002 we programmed it, it was like these Linux machines. It's very like early early days. For that project, I think the hardest part was not necessarily the AR but all the networking and all of the code to make because they really needed many machines to run that project.

[00:39:07] SY: Nice and Molmol, what about you, especially coming from virtual reality to augmented reality, what was that transition like?

[00:39:11] MK: 10 years ago when I was doing film, I remember from there I was doing a VJ for like brave party for a little while. This is a funny story. There was this one time we were in in Asia and we had to do this outdoor improvised music festival under the bridge. So we brought this projector. I don't know if you remember those big clunky overhead projectors, like a photo projector. And then we have those like lab glasses. There is a little container and we start adding water in it and then we start adding oil and then we start adding like oil with water, so they kind of don't mix well together. They started moving around once you projected out, it's like oh my god, now it's like moving image of like color animation. Then my friend finds this bug that's like huge, it has like a really weird horn on its head.

[00:40:14] SY: It's like is a literal bug.

[00:40:11.24] MK: A literal bug, we found a bug, like a real bug. We put in on this container, and it started to like swim or like just walk around between these colored oil and water mixture and then we projected it onto people, and that was my first kind of augmented reality and kind of really trippy experience that I've ever had.

[00:40:43] SY: Wow. So when you think about the future of augmented reality, do you feel like investing in learning augmented reality, learning these new tools. Is that a good long term investment when it comes to skills and career?

[00:40:58] MK: Yeah for sure. We just recently spoke with someone who had been doing since the 80s. She's doing very well and she was able to like promote a young generation of artists that work in AR and VR and career wise you first have to really believe that this is something you want to do and be able to do it for a long time, and from there you can kind of extend your branch a little bit. You might not just be doing AR-related stuff, you might move into film, you might be moving to live events. There are many possibilities and I think the most important thing is to identify what you are really passionate about.

[00:41:36.27] SY: Yeah. Zach, what do you think?

[00:41:42] ZL: The exciting thing is that a lot of these things that we've been experimenting with for a long time are moving their way into product and that means that a lot of people are able to interact with them. And you know you're able to reach a really large and engaged and curious audience. But I also feel like although I'm super excited about AR, I feel like it's never good to be like excited about the technology, that it's good to be excited about the ideas, and that technology lets you implement those ideas. So in our case it's it's about understanding space or new types of cameras or seeing sound, but it's it's driven by the what if question, it's driven by the thing that you want to see and not necessarily the technology itself. This field is super interesting and there's going to be a ton of changes, there's going to be a ton of things happening and I think it's really exciting. [00:42:36] It's an exciting time to be you know working in AR.

[00:42:36.26] SY: Well, thank you both so much for being on the show and schooling us and showing us about how awesome AR is. Do you all want to say goodbye?

[00:42:49] MK: Bye!

[00:42:46.27] ZL: Bye, thanks for having us. I just want to say that we are really big fans of this podcast and the community. And you know, I'm just really excited and honored to be here.

[00:43:01] SY: And that's the end of our seventh episode of Season 2. Let me know what you think. Tweet me @CodeNewbie or send me an e-mail, hello@codenewbie.org. If you're in D.C. or Philly, check out our local CodeNewbie Meetup groups. We've got community coding sessions and awesome events each month. So if you're looking for real life human coding interaction, look us up on Meetup.com. For more info on the podcast, check out www.codenewbie.org/podcast. And join us for our weekly Twitter chats. We've got our Wednesday chats at 9:00 p.m. Eastern time and our weekly coding check in every Sunday at 2:00 p.m. Eastern Time. Thanks for listening. See you next week. (Music).