One thing we always say about Virtual and Augmented Reality is that we’re on time to avoid the most horrific scenarios in terms of privacy and safety because mass adoption is not here yet.
Luckily, there are many great researchers and scientists that in the last years have started to dig into this matter.
In particular, Ibrahim “Abe” Baggili and his team have been the first ones to explore the novel cybersecurity attacks that can be conducted using the specific features of Virtual Reality, and they published a paper in 2019 illustrating how can a person be moved, disoriented, pushed against a wall, or blackmailed while using a common XR device. Abe is the founder of the University of New Haven’s Cyber Forensics Research and Education Group (UNHcFREG), and one of the original members of the XR Safety Initiative. So, there’s no one better than him to drive us in this exploration!
Oli: As many of you know, in every episode, we ask our guests to give us their definition of singularity. So what does singularity mean to you? If you have to kind of define it with your own words
Abe: That’s a good question. As a scientist, I understand that language can change over time. So anything that we call singularity now could actually change as we move forward. A lot of people talk about the singularity as when AI maybe will surpass human intelligence and be able to self-replicate, create, be more autonomous on its own, and things like that. When I think of singularity, that’s what I think of.
Kavya: As I said before, you and your team in 2019 published the very first set of novel attacks that can be conducted in virtual reality or immersive technologies. We can detail them out more, but first of all, what keeps these attacks together conceptually? Would you say it’s the attack surface or something else? Is there a sort of a fundamental in that covers them and ties them together?
Abe: Let me start with this thing called the “human joystick” attack. There’s kind of a story behind that. We actually figured it out by mistake. We were at a conference, and it was more of an artistic sort of conference where people were creating VR environments. And my student was playing, we tried to make that attacks. And what was interesting for us is that my student went and launched a script to change the environment, and we noticed the girls stepped to the right. We stumbled across this idea of a “human joystick” attack where you can change the center of the virtual environment and move people around. I definitely think it’s going to be an amalgamation of existing and new types of things. A lot of the attacks that we did were rooted in old exploits, like XSS (Cross-site scripting). So we leverage XSS to be able to get into the systems, but the impact might be different, and it might be relevant. I think this is the thing that we need to realize. Code is code, but now you’re in Virtual Reality, and maybe there are new types of attacks that could be created with different kinds of impact. People can start looking at these things and looking at the different technology-related building solutions that exist, like Unity. We need to help define it as somewhat of a subdomain so that we give it more merit and we get people to start thinking about the problem because otherwise, that’s never gonna happen. And I think that’s why labeling it as such is quite a crucial and imperative way for paving a path forward.
Oli: You talk about in your research documents, you talk about different immersive attacks, uh different categories, instead immersive attacks, and I’ve got a few here, so I’ve got chaperone disorientation, human joystick, and overlay. So imagine, like you have to explain this to the layman, which is, in fact, true because I’m kind of not very expert on this type of stuff, so I’d like to know a bit more about, you know, me and listeners or viewers how what that means in kind of easy terms.
Abe: An overlay attack is nothing but putting something in your VR glasses. I think of it as kind of the ransomware of your goggles, like we can put things there and you won’t be able to get rid of them and how annoying would it be if you have to wake up in the morning and go to work and you have to design something in AR and VR and all of a sudden you have this picture that says “pay me or else I’m not gonna go away.” That’s just one example of an overlay attack. The other way to use it obviously is if you really want to attack someone on a personal level, you start putting up pictures of their children on their glasses. And then I think of the psychological impact you might have on a person in that regard. That’s what an overlay attack is. The chaperone attack is really quite simple as well. If you understand VR, you know that typically there’s an environment, and there are detectors around to prevent you from hitting an actual wall. So, what if we could change that in the environment and take that away from you? You would be immersed in some game where you’re swinging your arm and jumping around, and all of a sudden, you don’t think you’re close to the wall because there’s no indication of that. And now you slam into the wall and hurt yourself physically. We’re no longer talking about just mental harm. We’re talking about physical harm, like being able to physically harm someone because of how the technology operates. When you’re in VR, you’re actually in physical space. And then there is the “human joystick” attack. It is when you move someone from one point to another point without their knowledge or consent. Imagine playing a game in VR and all of a sudden we start moving the center of the room virtually. And as a human being, you will compensate for that. We can move you in the direction that we’d like to move you in. From my perspective, that’s really interesting because imagine you want to snipe someone from the window, now you can get them closer to the window and snipe them. Disorientation is another one: if you’ve ever played with virtual reality, you know that it could get you sick if you stay in it quite a bit of time. And what if we could make you feel really sick? If you start moving around all all of the information in terms of the axes that you’re looking at all of a sudden, you’ll feel really disoriented and sick. I always tell “if your husband plays a lot of gaming, and you want to get rid of your husband, it’s the perfect time to do it, because they have the virtual reality headset on, they can’t see anything. A lot of times they have headphones on, strike them across the head.
Kavya: Do not do this at home. These are Abe’s research ideas.
Oli: That disorientation one was particularly interesting because I was kind of picturing this scene in the future. Maybe when some big public figures speaking in public in VR and they’re attacked with something like that while they’re doing a live speech. You know, it could be something quite powerful in terms of maybe not actually hurt them, but to create a blow to the reputation or something like that.
Abe: What if you could combine all of these attacks together? You can do some funky things. You can move someone to a staircase without their knowledge while they have their VR headset on, and then you can start disorienting them. They’d fall down the staircase. You can predict the exact location in the room that they’re in because of the technology that’s being used to track the people. So there’s a lot of combining things. The best kind of attack, in general, is when you combine multiple vectors, and that’s just how the bad actors think.
Oli: What’s the scariest thing you’ve seen when it comes to privacy and security in XR?
Abe: For me, the scariest thing is that we’re not giving it enough attention. Now that I have kids, there have been times where I’ve logged in to some of the VR applications, and you can hear little kids talking, and then you can also hear adults talking to them. Basically, you have a lot of people that might go into these sort of applications and start grooming kids, and it’s just another new avenue for some of these people to do some negative things. That’s the part I like the least, to be honest with you, other than just the security aspects. The scariest thing that we create from a technical perspective is we show that we’re able to create a self-replicating worm in a VR environment and be able to completely access the person’s computer without them having to do anything. The person would just have to use the VR application as it was intended, and then we would be able to download malware onto their system, control their chats, change their profile, change their avatar, and have complete access to them, and we could even stream their screen, their computer screen live to our command and control server. Those are the things, from a technical perspective, it’s really scary because that’s another thing, and then the other one is the man in the room attack. We showed like we’re able to do this thing called the man in the room attack in VR. Imagine having people in the room that you can’t see. It’s the same idea as having a peeping Tom in the room and without your knowledge and consent, so you might be in a VR environment on a date or with someone trying to be intimate, and all of a sudden, there’s a person right there, and you have no idea of that. This idea blows my mind.
Kavya: Have you seen any of these attacks out in the wild?
Abe: Honestly, it’s hard to say because you don’t know if anyone has done it, then it hasn’t been caught. The other aspect of that is that as a researcher, I always think about things before they happen. I can’t really tell you if anything has been in the wild now, but definitely grooming is happening. If you go into some of these VR environments, you can hear and see these things, so that’s something that exists. Another aspect is related to the platforms. Let’s take BigScreen as an example: one of the major issues was in the underlying technology that’s being used by BigScreen, which is essentially Unity. We told Unity about that, and they called it a feature and not a bug in their core code. So there’s a function called Open URL. And that function is being used liberally: whenever Unity coders are coding things in that function, you typically pass it to URL to open up. But it turns out that you can pass it other things. You can pass it the parameter of opening your command prompt on your computer. You can pass a lot of other parameters, like download this file, open this file, open this folder on your computer. So what I’m trying to say here is we don’t only want to blame vendors so to speak, but we also want to get down to the core programming issues in terms of the technologies and techniques that are being used to hit these issues at a lower level.
Kavya: I’m glad you mentioned grooming, because oftentimes when people talk about cybersecurity, they think about these cybersecurity-specific risks, but then there are these “real reality” risks that are also happening. When we discuss these attacks, we have to reflect on the consequences. You provided risk categories, and one of them is privacy. The amount of data captured by these devices and the quality of data that we can actually now obtain are pretty impressive. What tools do we have to manage this whole new level of reality capture from privacy, safety, cybersecurity, and also forensics perspective, which you really care about?
Abe: We were the first to actually publish papers on the forensic analysis of these systems. The idea with forensics is about the extraction of digital evidence from systems, things like when somebody opens or closes an application, you know what the user name is, who logged on at any given point in time, who communicated with whom. So we need to figure these things out, and it turns out there’s a lot of digital evidence we can extract from these things in terms of privacy. You asked what kind of tools. First of all, there aren’t many tools that focus specifically on XR when it comes to these things. Most of the tools I see are about collecting more analytics, while there are fewer tools about risk and privacy management. Actually, it’s kind of the opposite direction, where most of the tools are really about decreasing your privacy as opposed to increasing your privacy.
Oli: You’re the founder and executive director of the University of New Haven Cyber Forensics Research and Education Group. In the last few years, we’ve become more and more familiar with the cyber forensics domain. So, how do you plug XR into that discipline?
Abe: Cyber forensics is about the extraction of digital evidence and developing tools and techniques for the extraction of digital evidence from the system, both from memory as well as network. So, for instance, what data travels over the network, how we can figure out what that data is and who is communicating with whom. Those are the things that we look at. The interesting aspect about my field is that I care about privacy, but to do forensics, sometimes I have to invade someone’s privacy to figure out what happened. It’s a constant struggle for me to be in that situation and to find all that data. I truly believe that there’s a market for XR forensic stuff. So we have to really explore these things more and see what we could do with them and develop tools that are centric to these environments and to these systems.
Kavya: Could we somehow leverage XR spaces to do digital forensics, for instance, reconstructing things or events that may have happened? Are we there yet where we are conducting simulations?
Abe: The great thing about virtual reality is to be able to spin up physical environments for cheap. I look at it as the next iteration of virtualization. For the longest period of time, we wanted to virtualize computers, right? Now we have Virtual machines, we have systems clouds, all these things that are essentially virtualizing hardware for the end-user. So, if you want to spin up a server today, you click a button, and that’s it. The next iteration of that is how do we virtualize the actual meeting room or how do we virtualize a university campus? This is why VR is important because it decreases the cost and the barriers for people to be able to create these things. I don’t think the technology is fully mature yet to enable us to do it at scale and for it to be comfortable, but we’re getting there. We’ve already published a paper ourselves on the use of virtual reality in digital forensics education. We partnered up with Immersive VR Education, which is a great company from Ireland. We created an entire environment where you could learn digital forensics in virtual reality, where you would step in your investigator, you’re investing in a company called Big Thing, you’re on the scene, and you’re trying to find a hard drive or a USB stick, and we just teach the students how to document the crime scene. In my class, the first task that the students have to do is investigate a crime scene. So setting that up for 25-30 students is a very difficult and annoying thing. The idea for me was that doing this in VR is much cheaper and more cost-effective, and I don’t have to reset it every time. It worked quite well. The paper is available publicly for people that want to read it, and we made it freely available because of Covid-19, because this is something that could help in the education space.
Kavya: What was your general sentiment? Did you find it effective?
Abe: It’s effective in the sense of teaching concepts, but it’s probably not as effective in terms of it being less tactile. Our results show that they learned pretty much the same thing, which is great. But, you know, I’m a weird person because everything I do is in technology, but I also don’t believe in tech. I think there’s a lot of research that’s starting to point too much on technology. Too much automation is actually decreasing efficiency, leading us to become less efficient as a society. I think there’s gonna be a breaking point. I think VR is excellent and helps do a lot of great things. Still, we have to understand where the core uses might be in the future and how we can help people achieve certain things without decreasing efficiency and learning.
Oli: Technology isn’t just about digital and computers. Technology is anything that serves humanity. So maybe there is a form of technology that isn’t related directly to computers or making things do things for you instead. For instance, we know that people learn better if they have a more challenging time reading. We know that smaller fonts make people concentrate more. So maybe we should be setting ourselves to test a little bit more instead of getting people to do getting computers to do things for us all the time.
Abe: This entire Covid experience has really pushed digitization forward by like 10 years because people had to adjust very quickly. Everyone you talk to says this is such a positive, amazing thing. And I’m just there sitting down and thinking, “you’re full of crap,” most of the time. I’m sorry, guys, but that’s just how I feel about it. I mean, Zoom is excellent, and it helped serve its purpose, helped us digitize some of the things that we’re doing, but it’s not the solution. If we talk with students in university, they will tell you it’s just not the same experience. They don’t like it as much; they want to be in the classroom, be face to face, and I think there’s gonna be a breaking point for all of us. The whole idea of working from home is another issue. If I didn’t have kids, or my kids were growing up, if I had my home office and no one would bother me, it would be great. But working from home sucks most of the time. Some people love it, love the fact of spin out of bed, sit down and look at all the positive aspects. I’m like, great, but you’re not getting the actual job done most of the time. People will disagree with me and tell me they are a lot more productive, but they are not. They are just telling themselves that. If we continue moving in that direction, at some point, people are gonna revisit everything that we thought about. I’m a big technology advocate, but we need to start thinking about where the use cases will benefit humanity and not take us back 10 years in terms of how efficient we can be.
Oli: There’s a period when the technology is new where there’s like a honeymoon, and everything is good about it. Then there is a period where you develop some kind of immunity to it as a human society. So then people start to use it with more moderation and in the right places. We’ve seen this with TVs. There was a period when, in Europe, TV had been around for ages, so people would watch it, but not the whole time. There were still poorer countries who were so obsessed with coming over here and buying TVs and taking them back, and we’ll be like the big thing. Right? So that kind of made me think about that. Um, do you think there’s a period like a natural period where populations become obsessed with the new technology? Then that tends to balance out over time, or do you think we really do risk the future? We just become too dependent on it as a whole.
Abe: I can tell you this: buying things online is, especially in the United States, the thing to do today. Malls are closing, supermarkets are probably gonna close at some point. This has overall decreased the quality of purchasing, and it’s going to continue to increase our prices forward because people now believe they can do anything that they want with the click of a button. I don’t feel that we are taking the proper measures for us to stop and think about everything that we’re doing. Maybe it will happen in the future, and we can reconfigure and rethink, but that’s very difficult to do that when we become addicted. Another step is being able to order things online constantly and ordering food online too. I’m the biggest skeptical of these companies. I’m sure maybe some of you guys use them; they send you packaged items with the recipes. You get a recipe in a box, and now you’re going to cook these things. Are you kidding me? Go to the store, look up a recipe, and cook something. You don’t need someone to send you everything to your doorstep that is prepackaged, wasting a lot of money and all of these things to do it. I’m from the Middle East; I grew up in a household where my dad was constantly cooking, and I learned how to cook with him. I love cooking. It’s kind of my outlet for everything that I do. The last time I gave a presentation, it was actually a VR forensics presentation in California. And what struck me guys is you go to these coffee shops, and now there are robots making the coffee for you. The question is why. Because baristas that live there can’t afford the cost of living in that area, so now they have to find another way. Everything is about money at the end of the day, and sometimes we have to rethink that.
Oli: Let me play the devil’s advocate: does it make sense to teach cybersecurity? Sometimes it’s perceived as a practical discipline that doesn’t need formal training.
Cybersecurity is a big thing right now, and most places are starting to tell people you don’t need a college degree to get into cybersecurity. They tell you you just need to take a couple of one-week courses, and now go ahead and start finding a job in cyber, and you’re gonna do great. I’m the person standing out there, making everyone hate me, telling all these companies that they’re stupid. If you want someone to be a really good cybersecurity person, you have to teach them all the fundamentals; you have to understand things from the ground up. Now, if you want to reskill existing workers so you can move them to new things, that’s another problem. Google, for example, is boasting this, like “let’s just do this huge online education for tech talent.” There’s a lot of other companies that are doing that, and if you look at it, it’s to their advantage because if they can get people to get these things and hire them at like $90,000 a year, as opposed to hiring someone else at $120-130,000 a year, there’s a cost benefit ratio to the organizations that are doing that. I’m a first-generation college person, the first person in my family to get a college degree. Sure, there are some students and kids that didn’t have a college education and have been very successful, but that’s like 1% of the population, right? It’s not like the 100% or the 98 or the 96; it’s a tiny group of people that have been able to do something with their lives without a college education. I have to say this because I just have now and then the argument about “Bill Gates doesn’t have a college degree,” and I’m like, “Bill Gates was in Harvard!” He was already in Harvard, he was at the right place at the right time and definitely a smart guy, fantastic guy, done amazing things in his career. But you can’t just use that example. It’s not the same anyways.
Kavya: I totally agree with you. There is this misconception, primarily used by companies to manipulate you to think that this is a better education. Abe, not shocking news, but cybersecurity, as well as IT in general, is a white male-dominated field. First of all, have you ever felt marginalized in some ways in this domain? And do you think that this perception can be a barrier for women and minorities to even enter the industry?
Abe: I do a lot of work in terms of working with underrepresented groups and minorities. And you know, I have the GenCyber camp. It’s really been my push. Now, all of a sudden, the cybersecurity program has over 21% females. 3-4 years ago, it was like 0%. We’ve increased that amount significantly. In terms of me being feeling marginalized, I’m kind of a unique person. First of all, for whatever reason, a person from the Middle East in the United States is considered white. The second thing is most people, when I talk to them or meet them, assume I’m Muslim if they find out I’m Jordanian. But if they find out my name is Abe, they guess that I’m Jewish. If they see my last name, Baggili, many times they think I’m Italian. When I go to a Mexican restaurant, they start speaking to me in Spanish. So, I don’t know where I fit into the mold. Now I will say when I was in college, many students were getting internships in our digital forensics laboratory, and I kept trying to get one, which was unpaid even to work with, with a specific law enforcement entity there, and it wasn’t like federal level or anything like that. And finally, I remember going into my professor’s office asking him if I was not getting that internship because I am Arabic, and he told me he didn’t want to say that. Apart from that, I haven’t felt marginalized as much as others, probably because I just don’t fit any specific mold. But it is essential for us to change these things. Specifically, I mostly currently care about women in cybersecurity. I have two daughters, and I do everything in my power to make a change. At some point, 70% of my research lab were females. I’m hoping I could continue to change that.
