27 August 2020

Virtual Reality - A Future History?

 A few days ago I got together with a group of experts in 3D immersive learning and virtual reality and together we brainstormed what we saw as the major challenges facing 3D immersive learning/training and VR over the next decade or so.

The graphic below summarises our thoughts.

In the short-term (0-3 years) we identified the big challenges as being:

a) To make access easy and seamless. If people are going to use these environments they’ve got to be dead easy to use, and for organisations dead easy to manage. Oculus Quest is a good step forward (totally self-contained, automatic roomscale sensing), as is WebXR (no need to download to a headset or PC). But even within the experience, the “grammar” of how you navigate and use and interact with the space has got to be self-evident and common-sense. And no crashes or glitches or other “odd” happenings otherwise any sense of immersion is totally lost. And they need to integrate with your other digital presences - be that your desktop or work/social media accounts.

b) To make the applications desired. People and organisations have got to want to use this stuff. There has got to be user pull. For entertainment the experience has got to be worth all the hassle of setting up and clearing a space and totally isolating yourself from reality for a while, otherwise a film on Netflix or a game on Steam is going win out. For organisations the benefits of virtual learning and training have got to be clear and well understood. Yes there are lots of different case studies that show the benefits, but they aren’t well distributed or consolidated, and often aren’t too rigorous when comparing to BOTH the main alternatives (physical training and 2D eLearning).

In the medium-term (5-10 yrs) we identified two major challenges:

a) Mobility. Immersive learning needs to be available where and when people want to use it - it needs to be mobile. Yes an Oculus Quest is pretty mobile (I’ve even used it in my garden!), but in normal, non-COVID times, it's not feasible to use it on the bus or train into college, or sat in a cafe (locked away from the outside world with a purse or laptop ready to be stolen by your side), or sat on the sofa with half an eye on Love Island. Headset Mounted Display (HMD) VR needs to be complemented by mobile/tablet (and even laptop) based versions of the same experience. Yes there are advantages to HMDs (visceral immersion, scale, isolation), but there are also disadvantages (social isolation, nausea, convenience, cost). The user should decide HMD or non-HMD, not the software developer or trainer. 

b) Integration. We need to move away from walled gardens and towards standard based environments and applications. VR today is a bit like the pre-web Internet, different walled gardens, different access devices, multiple accounts. Users don’t want that - they want to hear about something and just access it with whatever device they have to hand, and with their long-standing personalised avatar. WebXR is helping here, and yes even the web still has many of these issues, and I know that asset wise we have broad portability between the platforms, but not in the scripting of the experience, or the management of the data associated with it. And does the “virtual world” approach of Second Life and Snowcrash offer a better model than the “app” approach of most current offerings. Many have talked about the Metaverse or Multiverse, being able to seamlessly (that word again) move from one virtual environment to another. There have been metaverse initiatives in the past - is it time for another one?

For the long term the group again identified 2 main issues:

a) Radical Interfaces. The VR HMD is a great step forward, but they are  still large, clunky, moderately uncomfortable for prolonged use, and not very portable. I’m pretty convinced that we need another big step change in HMDs before they become real consumer items where everyone one in they way that they currently (probably) have a tablet. What I have in mind is more like the holobands of Caprica than the Quest. Something that integrates VR, AR and MR, lets us readily see the physical world, tracks our hands, and, perhaps most important, manages to give us the “feeling” of locomotion, and perhaps the other senses. My guess is that this is as much a neurological interface as it is a visual one, and hence probably a decade or more out. 

b) Societal Change. VR is not just impacted by attitudes to it, but could also impact society itself. COVID has made us re-evaluate remote working and remote relationships. Popular media is full of stories based around virtualised people and places (Devs and Upload being just the latest examples). Even a decade ago virtual worlds were being used by hostile actors,  I doubt today’s environments are any different. How would a Caprica style virtual world, readily accessible by all, and with the capacity to do almost anything effect the way we all live and interact? Would it be for good or ill - and would it let us weather a second COVID that much better?

So there you are, 6 perspectives, 2 each for the short, medium and long term. You may not agree with all the details, but I hope that you can appreciate the general thrust of each, and each offers a timely call to action for the VR community.

Now scroll down a bit.

OK, I told a little white lie at the beginning there. The gathering of immersive learning experts wasn’t a few days ago, it was about 3,285 days ago at the ReLive (Research Into Learning in Virtual Environments) Conference. held at the Open University way back in 2011. Here's the original graphic - and you can find a fuller presentation I did later that years based upon it at https://www.slideshare.net/davidburden/virtual-worlds-a-future-history

But I think you’ll agree that the general vision and issues being raised back in 2011 differ little from what a similar analysis would yield in 2020 or even early in 2021 - ten years later! Some of the specifics might be different, and my commentary above reflects a contemporary take, but the big picture items are pretty much the same:

  • This stuff still isn’t seamless, although with Quest and WebXR we’re taking some great strides
  • The entertainment and business case is still struggling to be made. I know that Quests sold out early in lockdown, but I’ve also seen numerous reviews of technology to help with lockdown that haven’t even mentioned VR and immersive 3D.
  • We’ve actually made great strides in mobility if you consider non-HMD VR, I can now run avatar style experiences quite happily on my phone or tablet if they’re not too high-rez, and Quest again helps with instant set-up, but it’s still much of an either-or choice.
  • Integration seems further away than ever as VirBela, Immerse, AltSpaceVR, Sominium Space, Hubs etc all compete for users.
  • Radical interfaces is actually the one we achieved first, I was in SL in an Oculus DK1 in 2013, only 2 years after ReLive2011 - but as mentioned above there is still a long way to go for the ordinary consumer.
  • Societal change may be driven as much by COVID (and the fear of similar future outbreaks) and climate change, but VR is having far more of an impact on popular culture than it did a decade ago, and that triumvirate of VR capability, external pressures and cultural exemplars may well be driving change more quickly - although perhaps not a quickly as we thought back in 2011.

So I hope you’ll forgive my little deception, but I thought it might be a nice way to not only to illustrate how many things have stayed the same despite the apparent “improvements” in technology, but also highlights how much there is that current VR practioners can learn from the work on immersive environments that was being done a decade ago. For inspiration just check out the agenda and papers from ReLive11 (and the earlier ReLive08), still available on the OU website.

10 August 2020

Virtual Reality vs Immersive 3D - the Search for the Right Words!

As a company that been creating immersive experiences for over 15 years we find that the contemporary obsession with headset based virtual reality (HMD-VR) is often at risk of a) forgetting what valuable work has been done in the past in non-HMD immersive 3D environments and b) not highlighting to potential  clients that a lot of the benefits of "VR" can be obtained without an HMD, and not having the funds for, or access to (esp in COVID) HMDs does not need to stop a VR project in its tracks.

One problem is that we just don't have the right terminology, and what terminology we have is constantly changing.

"VR" has almost always been assumed to mean HMD-based experiences - using headsets like the Quest, Rift or Vive - or even their forerunners like the old Virtuality systems.

 But in that fallow period between Virtuality and Oculus DK1 3D virtual worlds such as Second Life, There.com and ActiveWorlds were enjoying a boom-time, and often found themselves labelled as "virtual reality".

One problem is that there seems to be no commonly accepted terms for the classic Second Life (or even Fortnite) experience, where you can freely roam a 3D environment but you have a 3rd (or sometime 1st) person avatar view of it. It's certainly not 2D. It's sort of 3D - but not as 3D as the stereoscopic experience using a VR-HMD. I've seen 2D/3D or "3D in 2D" but both are cumbersome. We sometimes refer to it as "first-person-shooter" style (but that doesn't go down well with some audiences), or "The Sims-like". 

There's also a qualitative difference between say a 3D CAD package where you're rotating a 3D model on screen (called an allocentric view) and the experience of running through Fortnite, Grand Theft Auto, or Second Life (called an egocentric view).  You feel "immersed" in the latter group, not just because of the egocentric view point but also because of the sense of agency and emotional engagement.

At a recent Engage event I went to I'd guess (from avatar hand positions) that about 50% of attendees were in VR-HMD and 50% using the immersive-3D desktop client. So should it be described as a VR or immersive 3D system? Our Trainingscapes is the same, we can have users on mobile, PC and VR-HMD devices all in-world, all interacting. And Second Life is often "dismissed" as not being "proper VR" - but when Oculus DK1 was around I went into SL in VR - see below - so did it stop being VR when they went from DK1 to DK2?

So if a system can support both - is it a 2D/3D system or a VR system? That is why we tend to refer to both the 2D/3D  approach and the VR-HMD approach as being "immersive 3D" - as long as you have a sense of agency and presence and the egocentric view. It's the experience and not the technology that counts.

And don't get me started on what "real" and "virtual" mean!

No wonder clients get confused if even we can't sort out what the right terms are, and its far too late for some de jure pronouncement. But perhaps we all could try and be a little bit more precise about what terms we do use, and whether they are just referring to the  means by which you access an experience (e.g. VR-HMD) or to the underlying experience itself (such as a virtual world or virtual training exercise).

In later posts I'll try and look more closely at the relative affordances of the 2D/3D approach (better name please!) vs the VR approach, what researchers experiences of virtual worlds can teach us about VR, and also how "virtual worlds" sit against other immersive 3D experiences.