[00:00:00] (Music) SY: Welcome to the CodeNewbie podcast, where we talk to people on their coding journey in hopes of helping you on yours. I'm your host Saron and today we're talking about building augmented reality tools for astronauts. (Music) Jesslyn recently graduated college.

[00:00:25] JT: My name is Jesslyn Tannady.

[00:00:27] SY: But while she was there getting a double major in computer science and media arts and sciences, she also got to work with NASA. She built a tool that uses augmented reality to help astronauts walk around on Mars. And to do that she had to learn game development. She got to go to a volcano to test her tool and ended up publishing an academic paper on the topic. She tells us all about it. After this.

[00:01:01] Flatiron School teaches you how to code from anywhere. They've got an awesome community of career-changers and a number of different options for you to pick from to become a software engineer. They've got full-time in-person courses, self-directed introductory courses and a remote online web developer program. They even have a free 75-hour online prep course where you can learn JavaScript, Ruby and do some interview prep. Go to flatironschool.com/podcast to learn more. That's flatironschool.com/podcast. Link is in your show notes.

[00:01:30] One of the best parts of being a coder is finally being able to bring your passions to life. You have the skills to design, to code, to create the thing you're excited about and share that passion with the world. And Hover can help you with the first step of sharing your passion with the world: getting your domain name. They've got a really beautiful and easy-to-use interface where you can find and register your new domain name in just a few steps. And to give you full control, they separate your domain name from your hosting so you're never stuck with one service. They keep your domain name safe while giving you the flexibility to use whatever hosting service is best for you. They also give you free WHOIS privacy, so your personal information is safe, too. To get started, go over to hover.com/newbie to save 10% off your first purchase. That's hover.com/newbie. Link is in the show notes.

[00:02:18] DigitalOcean provides the easiest cloud platform to deploy, manage and scale applications of any size. They remove infrastructure friction and provide predictability so you can spend more time building what you love. Try DigitalOcean for free by going to do.co/codenewbie and get $100 of infrastructure (Music) credit. Link is in your show notes.

[00:02:42] SY: And what is that? What is Ctrl-labs?

[00:02:44] JT: Ctrl-labs is a startup based in New York City, and what they do is they work on technology so that you can control computers and robots using your brain.

[00:02:53] SY: Wow. That is so cool. So does that mean it's possible or we're still figuring out how to do that?

[00:03:00] JT: We have demos that work. We have the technology for it. I mean we're always...

[00:03:04] SY: Nice.

[00:03:04] JT: ...constantly improving the algorithms and the hardware and also like what we do with that technology, but it works. Yeah.

[00:03:12] SY: Wow. Oh my goodness. That is—that's just so exciting. Ok. So but today though we are here to talk about what you did right before that actually. I think this job is a, a few days old.

[00:03:22] JT: Yeah.

[00:03:23] SY: Is that right?

[00:03:24] JT: Fourth days—fourth day on the job.

[00:03:24] SY: Congratulations.

[00:03:25] JT: Thank you.

[00:03:26] SY: But before that, you were working with NASA and you were building AR tools for Mars astronauts. And that is a direct quote, by the way. That's straight from the bio. "I make AR tools for Mars astronauts for NASA," which sounds so, so cool. Let's start with the tools part of that. What tools were you working on? 

[00:03:44] JT: Sure. So when we think about space travel, we often only think of the astronaut, but something that I feel is a little bit unseen as all the people that are involved in getting them up there 'cause there are so many tools and so, so much like maintenance and so much science and research that has to go on before we dare send a human anywhere, right? We just wanna make sure that they're safe, make sure that things are good in going up there. So I guess I'm one of the many, many, many behind the scenes people. And I guess like in relation to augmented reality navigation tools, I think a good place to start is history of people walking on other surfaces outside of Earth. So back during the Apollo missions, we sent people up there with paper maps and...

[00:04:26] SY: Wait, really?

[00:04:27] JT: Yeah, yeah. We had someone sent up there with like laminated paper maps.

[00:04:30] SY: Whoa.

[00:04:31] JT: Yeah.

[00:04:31] SY: I guess that makes—ok, if I had to think about it, that actually makes a lot of sense, but I just can't imagine laminated paper in space. (Laughing) Like that just... 

[00:04:39] JT: Yeah.

[00:04:39] SY: ...sounds so weird.

[00:04:40] JT: Yeah.

[00:04:41] SY: Ok. Ok, cool.

[00:04:41] JT: We came up with rockets, but nothing beats laminated paper at the time. (Laughing)

[00:04:46] SY: Right. Yeah. 

[00:04:47] JT: But we had one mission—one of the Apollo missions—where someone was sent out to walk on the surface of the moon. We call those extravehicular activities, so EVAs 'cause you're walking outside of your vehicle. And the EVA was extended for an extra 30 minutes because the astronaut couldn't properly triangulate himself. And he basically...

[00:05:07] SY: Oh no.

[00:05:07] JT: ...got lost on the moon for 30 minutes.

[00:05:10] SY: Oh my god. Sounds like a horror movie.

[00:05:12] JT: Right. And obviously that was stressful for a lot of people. (Laughing) And two big reasons why they think that happened is—there are very like sparse landmarks on the surface of the moon, right?

[00:05:21] SY: I was wondering that. I was wondering like what is on the map. Like there's a dent here...

[00:05:25] JT: It's kind of like there's a crater here.

[00:05:26] SY: ...and a bigger dent here.

[00:05:27] JT: Yeah, exactly. 

[00:05:28] SY: Right. (Laughing) Exactly.

[00:05:29] JT: Which I think makes sense if you're viewing from above, but when you're on the ground, I imagine it looks quite different.

[00:05:34] SY: Yeah.

[00:05:35] JT: Yeah, so there are very sparse landmarks on the moon. And the other thing is that the atmosphere on the moon is also different from Earth. So it's actually difficult for people to gage distance.

[00:05:44] SY: How is it different? What does it look like or feel like?

[00:05:46] JT: I think things either look closer or further away. I can't quite remember.

[00:05:52] SY: Ok.

[00:05:52] JT: But yeah basically had someone walking around who couldn't really triangulate and work out where they were. So we learned from those mistakes. And moving forward, the science... I guess like EVA community started to look into 2D digital interfaces. So kind of like wrist watch apps. Think of like if you had like an iPad strapped...

[00:06:09] SY: Oh. Ok.

[00:06:09] JT: ...to your wrist. So that's like a slight upgrade, right? There are a lot of I guess like advantages there because if you think about Google Maps it tells you exactly where you are and it dynamically updates and automatically rotates the map for you so you don't really have to work out from scratch like exactly where you are. And we like to think that it gives you a little bit more information and you can display more dynamic things.

[00:06:32] SY: But unlike Google Maps on Earth, if I take a big iPad 2D navigation thing to space, I can't get like a software update, right?

[00:06:40] JT: Oh probably not a software update.

[00:06:42] SY: Like if there, like if there was a mistake on there, like that, that's still kind of it. Right? Ok. Just checking.

[00:06:48] JT: I think the people on Earth will try to like make sure that doesn't happen before...

[00:06:51] SY: Yep.

[00:06:51] JT: ...sending you out there. 

[00:06:52] SY: Right. (Laughing)

[00:06:54] JT: That'd be so problematic.

[00:06:56] SY: Like oh no, north is south and south is north. Oops, it's too late. Ok.

[00:07:00] JT: But I guess like back to 2D watch interfaces, there's still a lot of problems with that. If it's attached to your arm, then that means that you can't really use your arms while you're...

[00:07:09] SY: Oh.

[00:07:09] JT: ...looking at the map, right? 

[00:07:10] SY: Right.

[00:07:10] JT: So if you wanna do useful science like picking rocks up or getting soil samples—all that kind of stuff—it's hard to do that while you're trying to—or I guess like we just wanna make sure that your hand is used for useful things.

[00:07:21] SY: Right.

[00:07:22] JT: And the other big problem is that if you have to keep looking at a screen and looking up again and down in a screen and looking up again, that leads to a lot of like loss of situational awareness which is also not great. Like have you ever been on Google Maps and it says that there's a building in front of you but and is next to like that other building but you still have to look down and up and think about it. Like wait, this one is directly in front of me.

[00:07:42] SY: Right.

[00:07:42] JT: Next to that one. There's still a lot of cognitive space that goes towards navigation that really doesn't have to go towards navigation. Again, could be used for more useful things. The other really big issue is that as we're thinking of Mars missions, Mars is obviously a lot further away from Earth than the moon is.

[00:07:59] SY: Yeah.

[00:07:59] JT: So we expect signal latency to be a big issue 'cause the way things were done when people were on the moon is that they were very dependent on Mission Control on Earth, right? It's kinda like they're always asking for instructions and mission control will tell them where to go. There's a lot of like dependency there. But as we get people further away on Mars, we wanna make—like we'll still have that kind of communication, but there will be an estimated 20-minute latency one way so...

[00:08:21] SY: Oh, wow. 

[00:08:21] JT: ...let's say there's like an astronaut on the surface of Mars, and she's like, "hey guys, which way do I go now?" It's like 20 minutes back to earth. And assuming they respond immediately, that's like another 20 minutes back. And that's just like...

[00:08:30] SY: Right.

[00:08:31] JT: ...40 minutes of an astronaut sitting there like not doing... 

[00:08:34] SY: Oh man.

[00:08:34] JT: ...anything, which is like a waste of time and taxpayer money and like energy.

[00:08:38] SY: And sounds boring.

[00:08:40] JT: And sounds boring. (Laughing)

[00:08:41] SY: Just standing and waiting for... I can't even go to the toilet without my phone, too.

[00:08:44] JT: Oh yeah. Yeah. So we wanna make sure that the astronauts are able to be a lot more independent. So I guess to summarize, our big goals were we wanna make sure that people aren't looking at screens. We wanna make sure that the information is presented in a more intuitive way than kind of just like looking at a screen. And we wanna make sure that it's presented in such a way that people—that the astronaut would have a lot more autonomy so that they don't have to keep asking for instructions and going back and forth. So what we proposed was an augmented reality navigation tool. We prototyped on the Microsoft HoloLens, which is their like AR mixed reality headset. And it was a team of three. So it was me, another undergrad student and our grad student advisor. And we...

[00:09:27] SY: Wow, only three people. It seems like such a huge project. (Laughing)

[00:09:31] JT: We were part of like a larger NASA lab, but the three of us were working remotely and this was like our project.

[00:09:38] SY: Cool.

[00:09:38] JT: Yeah, and I guess in terms of what we used to build it, a lot of what we used was Unity, which is a game engine. And the thing I really love about Unity is that there are a lot of tutorials out there, a lot of like really user-friendly tutorials that come with subtitles, which—and code snippets as well so you can copy and paste code and try out your own projects. And I really love that there's a whole community of developers out there who make tutorials and videos. So you go on YouTube.

[00:10:04] SY: Oh, that's awesome.

[00:10:04] JT: There's so many Unity tutorials out there, and it's pretty accessible to pick up. So that's what we developed it on. And the hardware that we used was Microsoft HoloLens. And then the other big, important thing after you develop your tools is that you wanna test it, right? 

[00:10:20] SY: Right.

[00:10:20] JT: But the big problem with testing software that you wanna use on Mars for humans is that we don't have humans on Mars yet (Laughing) as far as we know.

[00:10:27] SY: Right. As far we know. Key words.

[00:10:30] JT: As far as we know. So what we ended up doing was we went to a volcano in Hawaii or like a crater.

[00:10:37] SY: Seriously? 

[00:10:38] JT: A volcanic crater. Yeah.

[00:10:39] SY: What?

[00:10:40] JT: Yeah.

[00:10:41] SY: Wow. So ok. Explain to me again why...

[00:10:44] JT: Yeah.

[00:10:44] SY: ...the volcano? How? 

[00:10:46] JT: Yeah.

[00:10:46] SY: Yeah, why. Yeah.

[00:10:47] JT: The volcano craters have miles and miles of lava terrain. It's just rock as far as you can see. And that's very... 

[00:10:53] SY: Wow.

[00:10:53] JT: ...almost like otherworldly. So (Laughing) when we went there with the NASA BASALT team, it was—we had kind of like our Mission Control-style area. And then we had some people out there. Some of them were assigned to some of them were retired astronauts. I think we had like two or three astronauts there. We call these analog missions basically trying to simulate what a mission would be like. So they just kind of walked around, and I think half the time I was there I was in the mission control area just like debugging and just was like watching the video feeds and just trying to make sure that everything is good. And they let me to go out there and just like follow them on the field. There was one day when we were out there in the field, and I think there was some sort of issue with like a data package, so I had to upload something, and it's—let me tell you it's not fun debugging while you're sitting on a volcano. (Laughing)

[00:11:43] SY: Ok, so in, in that example, how do you know it is working? Is there like a specific—you mentioned kind of these analog missions I think is what you called it.

[00:11:54] JT: Yeah.

[00:11:55] SY: What does one look like? Like how, how do you know that it's done its job?

[00:11:58] JT: Sure. So it depends on the mission, on the type of mission and the mission objectives, right? Something we also wanted to test out was the 20 minute latency thing like how that dynamic would work between like the "mission control" and the astronauts out there, right? So like something that we set up was a video feed, but the video feed would only come into Mission Control 20 minutes later even though Mission Control was like not that far away for this...

[00:12:22] SY: Yeah.

[00:12:22] JT: ...for this test. But we wanted to test out that dynamic. So I guess there were a bunch of like questions and metrics and just like trying to work out whether good practices for like giving instructions and how people should conduct themselves. I guess like if Mission Control on Earth is like, "oh, that rock over there is really interesting. I want the astronaut to pick it up, sample it and bring it back to Earth." You know, it's like how would that dynamic work knowing that the astronaut isn't, probably isn't there anymore and that they're probably like somewhere else in the 20 minutes like since that video came in, you know? For me specifically, something that was really interesting is I wanna know whether my or like the tool that we developed is better for helping people stay on task or stay on the path, right? So we took some data about the proposed path, which was projected in front of the user as they're walking as if like it's really there, you know? But really it's a hologram. Do people better follow this path using our tool or do they better follow the path using a 2D wrist device. And we found out that our tool was better, and it's—like people stayed more on...

[00:13:24] SY: Yeah.

[00:13:24] JT: ...track, which is really great. 

[00:13:26] SY: That's right.

[00:13:26] JT: Yeah. And the other big thing that we wanted to know was whether people liked it, you know? 'Cause it's like it's not just about...

[00:13:32] SY: True.

[00:13:32] JT: ...the numbers, right?

[00:13:33] SY: Yeah.

[00:13:33] JT: Like how do people feel about it? A lot of people said that they found it a lot more intuitive 'cause they didn't have to look at numbers or work out maps or work out like this is a mountain and this one goes below ground. Like, like working out all that kind of information. But another thing that we found really interesting is that some people preferred their interface like their display to look a certain way, and some other people preferred their display to look another way.

[00:13:54] SY: Oh.

[00:13:55] JT: Like one of our settings I guess had kind of like a red carpet-style path in front of you or like a ribbon trail so that you...

[00:14:02] SY: Oh, cool.

[00:14:02] JT: ...could essentially like walk on that trail. And it's like really easy and intuitive. Some people prefer to have that trail right on the ground so that they could almost like step on the hologram and follow it.

[00:14:10] SY: That's what I was thinking. Yeah.

[00:14:12] JT: Yeah. No, but some people preferred it above their head 'cause they didn't want to...

[00:14:15] SY: Whoa.

[00:14:15] JT: ...have to look at it. So there is an... 

[00:14:17] SY: Oh, ok.

[00:14:17] JT: ...astronaut called Stanley Love, who preferred it above his head 'cause he felt like he wanted to be able to actually see the rocks 'cause it's like potentially dangerous terrain, and he felt like the hologram was distracting. So that was really interesting. 

[00:14:28] SY: That makes sense.

[00:14:29] JT: Yeah.

[00:14:30] SY: How did you get to the volcano? Like if it's miles and miles of, you know, like rocks far as the eye—like is it, is it... Was it like a helicopter just dropped you down? Did you have to walk? Like how—is there a bus that go? Like I'm just—like how did you, (Laughing) how did you physically do that?

[00:14:47] JT: Let's see. I arrived at the airport on the big island, and then the grad student that I worked for picked me up from the airport and brought me to the military base where everyone was living. And then every day from the military base, we took some cars to the actual, to like the corner of the crater. And then we would just walk until we were satisfied like in the middle of the crater or yeah.

[00:15:09] SY: Ok, so let's go back to the building of the—so I wanna call it a tool, but it kinda almost sounds like a game, right? Especially since you're doing it on Unity and stuff.

[00:15:19] JT: So the reason we went with Unity was because Microsoft and Unity were talking to each other as HoloLens was developed. So there were a lot of like very handy tutorials that make it really...

[00:15:27] SY: Ok.

[00:15:27] JT: ...easy for developers to make things on Unity. And Unity already handles a lot of things like how colors work, how things like rotate and scale in 3D space. So that's why we went with that. I feel like when I think of a game I think of something with a mission or an objective.

[00:15:44] SY: Yeah. Yeah. 

[00:15:44] JT: I guess—I don't know, getting from point A to point B is arguably in objective, but... (Laughing)

[00:15:49] SY: That's true.

[00:15:49] JT: Yeah.

[00:15:50] SY: But yeah. Ok. Ok. That makes sense. So was that your first time working with Unity?

[00:15:55] JT: No. 

[00:15:56] SY: Ok.

[00:15:56] JT: I've worked on some other Unity projects before that. 

[00:15:59] SY: So how was the learning curve for you? And I know you graduated with a double major in CS and media arts and sciences, and you've done like a lot of coding in school. But when it comes to this project, it just feels—it feels so different, you know? Like it feels like a very different type of coding project than let's build a web app to organize projects. You know what I mean? Like it just feels like so different. I'm wondering what was the learning curve like? 

[00:16:27] JT: Yeah. So I feel like it's—yeah, it's like more different than building a website, but I feel like in terms of the skillset and the tools I use, it wasn't too different from building a video game. And something that I was really interested in or something that I've, I feel like I've always been interested in is games. It's something that I kind of picked up on the side as well just because I was interested in it. But the other big really helpful thing for me was having a community around it. So like I said before, there are a bunch of communities of game developers in Boston. But another really big thing was the hackathon scene for me. So there's this, there's this event called Global Game Jam, or just like game jams in general, which are events where people come together and they build games. You don't necessarily have to be a programmer, and that's what I really loved about it. Like you could be a programmer or, you know, you also have to work with audio people. You have to work with art people. You have to work with writers and level designers—all that kind of stuff. And I really love that it allows a lot of people to get involved in game development without having to know how to code or without needing a strong coding background. I got into it back when I was still starting to learn how to use Unity, and I would draw for my teams 'cause I also do some art. And I feel like just observing the other programmers on my team—even though I wasn't necessarily doing so much coding—it really helped me familiarize myself with my tools and like the workflow and all that kind of stuff. Yeah, just like having a community about it, around it and having people to talk to and collaborating with people just makes it more fun and more accessible. 

[00:17:52] SY: Tell me a little bit more about community 'cause I feel like that's a really popular answer that we get that we also really love. You know, whenever I ask people, "how do you learn a new thing?" And they say, "step one, like find a community." How do you leverage that? Does it come from getting ideas for projects? Is it about getting recommendations? Like how specifically do you take advantage of having awesome people around you to learn a new tech skill?

[00:18:18] JT: I feel like the biggest advantage of having a community, especially as a beginner, was mostly exposure to things 'cause there's only so much you can...

[00:18:25] SY: Yeah. 

[00:18:25] JT: ...Google. And you can't Google when you don't know...

[00:18:28] SY: Right.

[00:18:28] JT: ...what to—how like what you don't know to Google, you know? (Laughing)

[00:18:30] SY: Yes.

[00:18:32] JT: So I feel like one thing I really love about the communities I was involved in was that they would host talks and bring in speakers and just like showcase what other people were working on. And I really loved that. It just exposed me to what was out there. Through that I'm able to make my own decision about whether this is something I'm interested in or whether this is something that I want to pursue or like all that kind of stuff. I don't know. I feel like it helps a lot to be able to talk to someone about like, "hey, I'm working—I'm thinking of working on or like building this website or working on this project. What do you think? Do you have any advice?" I guess like finding people who maybe have a little bit more experience and being able to bounce back ideas and just having that personal feedback is really helpful. But I think the other really big advantage of having a community for me is that I guess just like having that network which also leads to opportunities. I wanna say that...

[00:19:20] SY: That's true.

[00:19:20] JT: ...that's like oh, like your skills will speak for itself, but I feel like knowing people is just like really, really important. I feel like it's...

[00:19:26] SY: It's so important.

[00:19:27] JT: It's so important. Yeah.

[00:19:28] SY: It sucks that it's important, but it is.

[00:19:30] JT: It is. Exactly. (Laughing) And just like getting to know people has allowed...

[00:19:35] SY: Right.

[00:19:35] JT: ...me to go from someone who, you know, is like trying to learn it and doing it in my spare time to someone who, who gets paid for it. Like through that I'm able to like learn about projects that people are working, and I can volunteer to help on people's projects. And eventually like people remember me. And then that leads to other opportunities, which leads to like collaborating on projects or even like speaking opportunities.

[00:19:53] SY: Oh yeah.

[00:19:54] JT: Yeah. I, I think it kinda sucks that is like—so much of it depends on like who you know. But I feel like I'd be lying if I said that that wasn't a huge advantage of being a part of a community.

[00:20:03] SY: Absolutely, yeah. Who you know is a lot of times like all you need.

[00:20:07] JT: Yeah.

[00:20:08] SY: You know? Who you know can be a, a huge, huge—and it's one of those things where, you know, there's a, a virtuous cycle there because if, if you know people, they get you to the first door. You do—hopefully you do a good job, and then that opens more doors. You do a good job, and you just open... You know, more and more doors can open with just like the first door, you know? So yeah. That's definitely a really important thing that, that is a, a really great benefit of being part of a community. Lots of...

[00:20:31] JT: For sure.

[00:20:31] SY: ...potential doors to open. Yeah.

[00:20:33] JT: For sure. And it just snowballs from there.

[00:20:35] SY: Oh, yeah. Absolutely. Ok, so with something like this that feels important but also kind of futuristic, right? Like we don't have people on Mars right now, so we can't actually use this. How do you see this applying to our world?

[00:20:54] JT: Our world as in like life on Earth? Or our world as in like the actual Mars mission? (Laughing)

[00:20:59] SY: That is a great question, and that was a terrible way for me to word that question. (Laughing) As it applies to I guess like life today, you know, given the fact that we're I think still kind—actually, you tell me. Are we still far away from living on Mars? Because I think that we are, but I don't really know.

[00:21:15] JT: I mean we're not even living on the moon, and we got there... 

[00:21:18] SY: Ok. There we go. 

[00:21:19] JT: ...like a while back so...

[00:21:19] SY: Cool. Ok, so assuming it'll take a while, what do you feel like is the, the application of this research or the value of this work today?

[00:21:26] JT: Let's see. In terms of like everyday pedestrian life, I feel like the research is applicable just for like navigation tools in general.

[00:21:34] SY: That's true, yeah.

[00:21:35] JT: It was actually really interesting 'cause right before we published this paper, there was that Google developer conference, Google IO, and they showcased this Google Maps thing that was quite similar to what we were working on. So it's like oh my gosh...

[00:21:46] SY: Oh, interesting.

[00:21:46] JT: ...someone is actually building this as well on Earth. That's really cool.

[00:21:49] SY: That's so cool. Yeah.

[00:21:49] JT: But I think it involves like pointing your phone at things. So if you've ever played like Pokémon Go it's like it overlays that kind of information on your actual environment. I guess wouldn't it be really cool if you could have like arrows on the ground or like arrows pointing you to where you need to go or things that will tell you like directly where to go in real life, you know? So I think that's like a really fun application for life on Earth. But in terms of the upcoming Mars mission, something that we acknowledged as we were building this is that by the time we get people to Mars, we really, really hope that the hardware will be better as well. And we know that what we're working with right now is not—it will hopefully be obsolete by the time we get there. So...

[00:22:27] SY: Yeah.

[00:22:27] JT: ...I guess what we hope out of our research is that whoever ends up building the actual tools that do get used by the time we get people on Mars will read our paper and get some good knowledge and good practices and just like things that we tried out, things that we found worked/don't work. I think our code is also out there as well. So we're hoping that this will like help contribute for—yeah, contributes for when like we have better hardware if that makes sense.

[00:22:52] SY: Yeah. That's really interesting because this project involves a lot of different pieces. You have to know about Mars, and you know about astronauts and their needs. You also have to know how to code stuff. You have to know how to use HoloLens and how to build on HoloLens. You have to understand, you know, hardware and what the limitations of that are. Of all of those pieces, when you think about what's going to happen in the future when we if we hopefully are able to go to Mars and, and, you know, like live there, what of these pieces do you think might still prevail?

[00:23:27] JT: I think understanding human needs is something that will last a long time. I feel like software and tools and like hardware change pretty quickly, but what's intuitive to people and how people interact with things and how we interact with 2D information in 3D spaces, that's I think going to be there for quite awhile. Yeah. 

[00:23:47] SY: So I love this project because you're working with the future in many different ways. You know, the fact that like AR is—I can't really say that it's new 'cause it's—like different versions of it have been around, but the most recent iteration feels very new. Like, you know, Pokémon Go was a huge sensation 'cause it, it felt like a new experience, but then also the fact that you're building tools for astronauts for Mars is like very into the future. So I'm wondering how does that inform what you want to learn about or focus on as an engineer, as a, as a programmer yourself?

[00:24:21] JT: Well one thing that was really, that was like quite difficult for me when I first started was that everyone around me was so much smarter than me and like a lot of them had PhDs or were pursuing PhDs, and everyone was just so—everyone just like knew so much. And I think it was initially uncomfortable being in a space where I knew that I was the most unexperienced person in the room. But I think that being a part of this project for the last year and a half has really helped me reshape how I want to position myself in these spaces. It kind of made me feel a little bit more comfortable asking questions instead of hiding and like pretending that I know things that I don't really know. But also just like learning to value being the least smartest person in the room because that means that I can learn a little bit from everyone, right?

[00:25:06] SY: Absolutely.

[00:25:07] JT: And...

[00:25:07] SY: Yep.

[00:25:07] JT: Yeah. I like that really helped inform like what I ended up applying to forwards, like moving forwards. And yeah, like making sure not to bar off opportunities that were like way over my head. (Laughing) Yeah.

[00:25:19] SY: Yeah. How long did it take to get the first working prototype for the tool?

[00:25:25] JT: I wanna say like a week or two after we got the hardware.

[00:25:27] SY: Oh, wow.

[00:25:28] JT: Yeah.

[00:25:29] SY: That's pretty good. That's awesome. And how many iterations did you end up going through before you took it to the volcano?

[00:25:36] JT: All right. So I mean it was kind of like an iterative process, right? Because we had our minimum viable product like our very first version. And then after that, we just kept—it was like an iterative process where we kept bringing people, and they would give us their opinion on it. And then we just kept adding features. So it wasn't so much like a version like 1.0, 2.0, 3.0. It wasn't like releasing iPhones. You know, it was more like, you know it's like this is one version, and this is like a slight tweak and that's like a slight tweak. So sometimes even in a new update, it's like, it doesn't feel like it's really like a big improvement but... 

[00:26:06] SY: Yeah.

[00:26:07] JT: ...incrementally it just like...

[00:26:08] SY: Yeah.

[00:26:08] JT: ...evolves into something else.

[00:26:09] SY: Yeah. That's kind of like the right way to do it, you know?

[00:26:13] JT: Yeah.

[00:26:13] SY: You wanna push like a little by little and feature by feature. So that's awesome. 

[00:26:16] JT: Yeah. 

[00:26:16] SY: That's great. Yeah. So when you were building the, the final version, what was the biggest technical challenge that you had to deal with?

[00:26:26] JT: I think the biggest technical challenge for me was working with networking. We also had like Bluetooth device that we had to bring on as well. And I've never quite worked with that before, and I've never taken a class on it. And my—the other person in my team actually handled most of it. But I guess like I wanted to learn, and I wanted that to be useful as well. So he would write code and I would like read his code and be very confused and overwhelmed and then like (Laughing) try to go back and ask questions. And then just like trying to get to a place where I could also like operate this independently was, was like technically challenging for me 'cause I didn't really have a background in it, but also quite scary for me 'cause it's like people would be like, "hey, why isn't the code working?" And I'm like, "I have no idea." (Laughing) So...

[00:27:12] SY: So in those moments especially when you're working with so many awesome people and they, they know stuff that you don't know and you kinda have to, you have to catch up but you have to be able to, you know, keep up with them, how did you manage that? How were you able to, to deal with that knowledge gap, I guess? That, that difference between where they were and where you were.

[00:27:33] JT: I think the biggest—this sounds kind of generic but just like ask questions and ask for clarity.

[00:27:37] SY: Yeah.

[00:27:38] JT: I think asking questions wasn't necessarily the big issue for me. I think the big thing was knowing when to ask questions 'cause like something...

[00:27:46] SY: Oh.

[00:27:46] JT: ...that would happen is that I would like make a big list but then it's like I wanna ask someone but they seem busy. So it's like should I like talk to them? And I think just like having dedicated times regularly to ask these questions and ask things or even like askings on a spot like learning to do that was helpful. Yeah. When people around me like spoke jargon or talked about things that I didn't understand, often it was because they felt like everyone in the room already knew it.

[00:28:13] SY: Oh, yeah. Oh, I hate that.

[00:28:13] JT: And like they want to me to succeed is—yeah. I felt like they want everyone to succeed as well. So it's like they're not doing it to like confuse people. They're mostly doing it because they didn't (Music) realize that I didn't understand it, you know? So it's like they don't mind it when I ask.

[00:28:28] SY: Coming up next, Jesslyn talks about how she got started in coding. Spoiler alert she was not one of those kids coding since age 2. She actually got into it in college. She tells us more about her coding journey after this.

[00:28:44] When I learned to code, I was so excited to finally bring my passions to life. I could build things that I really cared about and share them with the world. And the first step in sharing is getting a great domain name. That's where Hover comes in. They've got a really slick east-to-use interface. They've got awesome domain names to pick from and they separate your domain from your hosting so you have full control and flexibility over your online identity. So go to hover.com/newbie to save 10% off your first purchase. That's hover.com/newbie. Link is in the show notes.

[00:29:17] You want to get serious about learning to code, but where do you start? Flatiron School's got the perfect thing. They're offering their free 75-hour online prep course, where you dig into JavaScript, Ruby and more. If you're not sure where to start, start there. And when you're done, you can keep learning with their self-directed introductory courses, remote online web developer program or full-time in-person courses. Whatever your schedule, they've got options to help you reach your coding goals. To learn more, go to flatironschool.com/podcast. That's flatironschool.com/podcast. Link is in your show notes.

[00:29:54]  DigitalOcean is the easiest way to deploy, manage and scale your application. Everything about it was built with simplicity at the forefront. Setting, deploying, even billing. Their support is amazing. They've got hundreds of detailed documentation and tutorials, so if it's your first time deploying an app, they've got great tools and community to make it nice and easy. Try DigitalOcean for free by going to do.co/codenewbie and get $100 of infrastructure (Music) credit. Link is in your show notes.

[00:30:25] SY: So you mentioned that you had some experience working with Unity. And as we talked about earlier, you studied computer science in school. I'm wondering for this project how much of it were you pulling from what you were learning as part of your degree? And how much of it were you learning on the spot and as you were working on this project?

[00:30:47] JT: To be honest like 95 percent of it was stuff I learned on the spot.

[00:30:51] SY: Really? Wow.

[00:30:53] JT: Or just like I mean I self-taught myself game dev crew like Unity tutorials that I saw online, so I feel like from that a lot of my AR/game development journey has been pretty self-taught. I felt like what I was learning in school was very theoretical, and what I was—when like for this particular like Mars project they cared more about whether it works or not.

[00:31:14] SY: Yeah.

[00:31:14] JT: So I felt like at that point, my coding knowledge kind of like split two ways. There's like what I know theoretically and what I know to do practically.

[00:31:23] SY: Yeah.

[00:31:23] JT: And this project kind of demanded more of like that practical side of it.

[00:31:26] SY: Did the theory part shape or influence or have any real effect on the practical stuff that you were learning?

[00:31:35] JT: Not for this particular project. No.

[00:31:37] SY: Interesting. That makes me hopeful honestly for myself. (Laughing)

[00:31:41] JT: I hope. I hope none of my professors are listening because I'm so sorry.

[00:31:43] SY: That's very encouraging. But I, I think that's a really encouraging for a lot of—for me and for a lot of folks listening because we don't have computer science degrees, you know? And I think that, you know, it's, it's always a question, you know, like should, should we go back to school? Does it actually matter? You know, how much—and, you know, when you hear someone say, "well you know I did—I worked on a NASA-funded research project, and I didn't really use the theoretical parts of my degree." That, you know, that's encouraging. That, that gives us hope.

[00:32:16] JT: Yeah.

[00:32:17] SY: So I'm glad for myself. (Laughing) For selfish reasons, I'm glad that is true.

[00:32:22] JT: I mean this is like very individual I guess, but I feel like...

[00:32:25] SY: Yeah.

[00:32:25] JT: ...the biggest thing that my computer science gave me was legitimacy.

[00:32:30] SY: Yep.

[00:32:30] JT: It taught me a lot of skills as well, but none of it was as helpful as like having that yeah, legitimacy and connections, I think.

[00:32:37] SY: Yeah 'cause that was the other thing, too. I was looking at, at your resume and you've done internships at Apple.

[00:32:43] JT: Yeah.

[00:32:43] SY: I believe. 

[00:32:43] JT: Yeah. 

[00:32:44] SY: Right? And MIT Media Lab. And I assume a lot of that, a lot of—the fact that that opportunity was even available was because you were studying computer science.

[00:32:53] JT: Well the MIT Media Lab thing and this massive project, they all happened because I went to a school that I went to. And I just, I just want to be real like I don't think any of this would have happened had I not gone to this specific school. Like I don't think it was because of my ability at all. You know? Just like a right place at the right time. 

[00:33:11] SY: Well I'm sure both were important. Yeah, you—I mean I feel like you can't really work at NASA and be an idiot, you know?

[00:33:17] JT: I mean I guess I just wanna be real about that like...

[00:33:20] SY: Yeah.

[00:33:20] JT: I feel like, like, like we said before, you know, like often it's less about your technical skill and, you know, it's—a lot of it depended on like who you knew and...

[00:33:29] SY: Where you are. Yeah.

[00:33:31] JT: Yeah.

[00:33:31] SY: Yeah.

[00:33:32] JT: Oh, being in Boston I think was also another... Like I'm—I feel very lucky to be in a place with a lot of schools and a lot of students. Yeah.

[00:33:39] SY: Yeah. Oh, such an academic culture there.

[00:33:41] JT: Yeah.

[00:33:41] SY: Absolutely. So for people who are listening who aren't in Boston who don't have a computer science degree but who would love to one day do some cool stuff like work on a NASA-funded project, what advice do you have for them? What can the rest of us do to have some of that experience and launch our own tech careers?

[00:33:59] JT: One thing that was really helpful for me was to start thinking of opportunities as a liability game if that makes sense.

[00:34:07] SY: Oh, interesting.

[00:34:07] JT: So like when I think about like employers, when they hire me, they're hiring because they want me to get a certain job done, right? And as long as I can prove that I can get that job done whether it's through past projects or whether it's through whatever means, they just want that kind of assurance that like yes, I hired the right person to get the specific job done. And I feel like that's sometimes a little bit overlooked 'cause, you know, it's like, "oh, I got an A in this class" or "my GPA was a certain thing," you know? It's like—I guess as a software engineer, people aren't necessarily hiring me for that. It's like great, you know, it's like I know how to follow instructions, and I know how to do homework I guess. (Laughing) And like take tests. But at the end of day, it's like—I think like once I realized I wanted to go more into this like game dev field, I started dedicating a lot of my free time towards building projects so that if people wanted to hire me I could show them that like yes, this is something I've implemented before. And no, you're not taking like a crazy risk by hiring me and giving me this opportunity. And I feel like being able to show that was really helpful.

[00:35:04] SY: That's such an interesting framing of it, you know? Because you're right. Whenever someone hires you, they're taking a risk. And so the more you can de-risk yourself...

[00:35:15] JT: Yeah.

[00:35:15] SY: ...the easier it is to actually get that opportunity. And in a way, when they see a computer science degree, it looks like you're less risky. That's kind of the...

[00:35:24] JT: Yeah.

[00:35:25] SY: ...you know, the idea.

[00:35:24] JT: For sure. For sure.

[00:35:26] SY: Yeah. 

[00:35:26] JT: I think the really great thing about the tech industry at least in the U.S., which is where I'm most familiar with the tech industry is that often they don't care as much where you went to school or your GPA, it's like—a lot of it boils down to whether they like you in like your like coding interviews, you know?

[00:35:42] SY: Yeah.

[00:35:42] JT: I feel like work on projects and show and apply to things and show people that you're not a liability.

[00:35:48] SY: Yeah. Oh, that's great advice. Shall we move on to fill in the blanks?

[00:35:52] JT: Sure.

[00:35:52] SY: Number one, worst advice I've ever received is...

[00:35:55] JT: One of worst advice I've ever gotten was that people who have made it know what's best for you. So I remember when I first started, I would talk to older students or who have like, you know, like they work at Google already, interned at Facebook. Or it's like oh, like this person is like now a manager at like some really cool company. And it's like I would talk to them, and they would give me advice. And I would like really take it to heart and like listen to it and apply it to my life like word to word. And I feel like there's a lot of problems with that, and I think it's like difficult to not like completely absorb everything 'cause it's like oh my gosh you're someone who has made it, you know? I just feel like there's a lot of like survivorship bias there.

[00:36:31] SY: Oh, yeah.

[00:36:31] JT: You know, you could go up to someone and ask like, "oh my gosh, how did you end up working at Google?" And they'll be like, "oh, it's really easy. I just applied." You know? But it's like what that doesn't talk about is that like they, you know have a computer science degree, and they went to an engineering school, and also a U.S. citizen. And, you know, they've been...

[00:36:45] SY: Yeah, yeah.

[00:36:45] JT: ...coding since they were 13 'cause their parents were also engineers or whatever. It's like it doesn't talk about a lot of that. So I'm Chinese Indonesian. And I think like when I first came—when I started looking for tech jobs, I found a lot of like Asian American women in tech. So people who looked like me but had very different life experiences from me. And I think that one thing I wish I was more aware of earlier is that there are people who could look like you who may not necessarily understand or share your life experiences. 

[00:37:12] SY: Interesting. How was yours different from the, the woman that you, you met?

[00:37:18] JT: Oh, I mean this is kind of like compiling a lot of stories from a lot of different people. I like to ask advice from people, and it's like people say different things. But I don't know, like I didn't grow up in the U.S. I only came here for my undergrad. I grew up in Indonesia, and it's like I only learned to code when I came to college. So it's like, you know, I didn't like—I haven't been coding since I was like 13, you know? Like nobody in my family works in tech. I guess growing up it's like I never thought that I would work in tech 'cause it's like "oh, those people who work in tech are like smart people in the United States on the other side of the world," you know? It wasn't until like quite recently that I was able to identify as a developer. So I feel like I—especially when I first started to learn how to code I didn't have that kind of like confidence. And I feel like if you're at a place where you're like a senior engineer at like some tech company, you may or may not have already forgotten what it's like to just learn how to code, you know"

[00:38:08] SY: Yeah. Oh, yeah.

[00:38:09] JT: Something that's really meaningful to me is like finding people who share similar life experiences on certain axes. So whether it's like, "oh, this person also came from not the U.S. like me, you know, even if they came from a different country from me. But it's like we share that. Or it's like, "oh, this person, you know, like only learned to code very recently like me." You know, that, that's—I guess like taking bits and pieces of people's experiences and advice I think is really helpful. I guess like...

[00:38:36] SY: Yeah.

[00:38:36] JT: ...another one that— that I was talking to a friend about was the whole thing about working in tech with a really high salary when you came from a low income background and just like dealing with that kind of, like that kind of like dynamic of knowing that you make more than your parents or whatever, you know? It's just like, just like having those kind of conversations.

[00:38:56] SY: Yeah.

[00:38:57] JT: And I feel like, like those are things that like people don't talk about and... 

[00:39:00] SY: Yes.

[00:39:00] JT: ...back to the whole survivorship bias thing, like it's easy to say like, "oh, I just applied to Facebook, and I got the job. And it was easy." You know, it's like just study hard and like, you know, but there's like so much that like goes on behind a scene that people don't really talk about. And I think the other second piece of worst advice that I got was the fact that you can do it all, which sounds really bad 'cause I think like when someone tells you like, "oh no, women can't have it all or whatever." You’re inclined to go like: "No, women can have it all." (Laughing) But I think that especially like when I first started learning to code, I was like, "oh my gosh I'll volunteer myself for everything. And, you know, it's like I'll do everything. And like, you know, like stay up all night and like work on all these side projects and stuff.

[00:39:36] SY: Yeah.

[00:39:36] JT: And I'm, I'm about to talk about like mental health, but it's like I think for me personally, that was my first time really dealing with burnout. And I think it led to a lot of listening to myself and constantly trying to tell myself like, "yeah, I can do it all." You know? It's like I'm okay. And if I can't do it all, it's because I'm weak and I'm gonna keep going and whatever. That led to a lot of like...

[00:39:56] SY: Definitely been there. Yep.

[00:39:56] JT: ...personal like mental health issues on my end. And it's just like really important to just like practice self care.

[00:40:03] SY: Number two, my first coding project was about...

[00:40:06] JT: My first coding project was for my Intro to Computer Science class, which I took a few semesters after I started college back when I was like "oh maybe I'll try this computer science thing." And it was a...

[00:40:17] SY: What made you try it?

[00:40:18] JT: Oh, I—so I went to a liberal arts school, and there were a lot of people around me who were ridiculously supportive of me starting computer—well, ok. I feel like there are two stories there. There's like why I decided to take it and why I decided to stay in it. Why I decided to take it was because I heard that the professors were very inclusive and very understanding. So I knew that there were a lot of people who have taken this class before who didn't come from a technical background. You know it's like we had like English majors and art majors take this class. And I was like, "oh, ok. You know? It's like you know it sounds like a class that's like accessible to people who...

[00:40:51] SY: Yeah.

[00:40:51] JT: ...don't have a technical background. Because the thing is that I took a computer—like a computer class like and this was like learning like conditional formatting and Microsoft Excel. I took a com—like a computer class when I was like 13, and I was really bad at it. And I was like, "oh my gosh, I'm really bad in computers. I'm never gonna do computers again." So when I was registering for this class, I was like, "oh, well maybe someone who is bad at computers like me can like maybe do ok in this class. And I mean it changed my life. But the reason why I stayed in the class was because there were a lot of people in that class who were just like really supportive. And I feel like I'm very privileged to have been able to start my tech journey or my coding journey in an environment like that. It's like being in a place where the professors were really like, like not only like said it, but also like acted. You know, like there's no such thing as a stupid question. And just like making sure that everything was clear and like having office hours like way into the nights just to make sure that like students had their questions...

[00:41:46] SY: Oh, that's awesome.

[00:41:47] JT: And I think the...

[00:41:48] SY: Yeah.

[00:41:48] JT: ...other really great thing was that even like my peers in the class were like really willing to support each other. So it's like I'd have people who have like, like classmates who already like finished a homework but they'd still come to office hours to try and support their peers and like help them debug and stuff even though it's like they don't have to be there. It's not their job, you know? But then it's like the following week they're struggling with the homework, and I'm able to help. And it's like just having that kind of community where we all just supported each other. This kind of like really got me through the class.

[00:42:13] SY: Yeah.

[00:42:13] JT: So that's why I decided to...

[00:42:14] SY: That's awesome.

[00:42:15] JT: Yeah, start my first...

[00:42:17] SY: That's great. 

[00:42:17] JT: ...computer science class. But the first project was a final project for that class, and it was like a choose-your-adventure game. It's kind of like you make—you know, like text comes up and you make choices. And then like somewhere in a back there's like a counter saying like "good option. You've chosen this five times. Bad option, you've chosen this like four times whatever. And then the outcome depends on what you've chosen.

[00:42:36] SY: Neat. What did you write it in?

[00:42:38] JT: It was in Python. 

[00:42:40] SY: Neat. Very cool. Number three. One thing I wish I knew when I first started to code is... 

[00:42:46] JT: There are a lot of very experienced people who will talk tech jargon to you and make you feel bad for not knowing. And one thing that really helped me with this was realizing that trying to think about like what I had control over in the moment when like people are like saying like, "oh my gosh, what do you mean you don't notice very specific JavaScript framework?" It's like what are you doing or whatever. It's like and you said you study computer science. And I'll be like, "oh my god." (Laughing) There are a lot of people who talked tech jargon, and I think one thing that was like really helpful for me to help me feel less overwhelmed was to think about like what I actually have control over. So it's like in a moment, that's like I, I can't help it that I don't know like that specific framework or I don't know Git or whatever, you know? It's like but the best thing I can do is just like calm myself down and then evaluate whether this is something I want to invest my time in and then just like ask questions and learn from there. And some people are gonna be supportive of it, and some people (Music) are not. And you wanna stay away from the people who are not.

[00:43:44] SY: Oh, I love that. Such great advice. Wonderful. Well thank you so much for being on the show and talking to us about astronauts and AR and coding and all kinds of really great things. You wanna say goodbye?

[00:43:56] JT: Oh, goodbye. Thank you so much for having me.

[00:43:58] SY: And that's the end of the episode. Let me know what you think. Tweet me @CodeNewbies or send me an email hello@codenewbie.org. Make sure to check out our local CodeNewbie meetup groups. We've got community coding sessions and awesome events each month. So if you're looking for real-life human coding interaction, look us up on meetup.com. For more info on the podcast, check out www.codenewbie.org/podcast. And join us for our weekly Twitter chats—we've got our Wednesday chats at 9PM EST and our weekly coding check-in every Sunday at 2 PM EST. Thanks for listening. See you next week.