7 September 2020

Aura Blogpost - "From Virtual Personas to a Digital Afterlife"


David has a blog post on "From Virtual Personas to a Digital Afterlife" on the blog of the new Aura website/service- "designed to help you prepare your memories, important information and connect with loved ones before you die", part of the burgeoning digital afterlife/memorial sector but also looking to open up the conversations ahead of time.

You can read the blog post in full at: https://www.aura.page/articles/from-virtual-personas-to-a-digital-afterlife/

It's also interesting to compare some of the ideas in the post with Channel 4's recent documentary on Peter: The Human Cyborg as there certainly seems to be some common ground worth exploring.

27 August 2020

Virtual Reality - A Future History?

 A few days ago I got together with a group of experts in 3D immersive learning and virtual reality and together we brainstormed what we saw as the major challenges facing 3D immersive learning/training and VR over the next decade or so.

The graphic below summarises our thoughts.

In the short-term (0-3 years) we identified the big challenges as being:

a) To make access easy and seamless. If people are going to use these environments they’ve got to be dead easy to use, and for organisations dead easy to manage. Oculus Quest is a good step forward (totally self-contained, automatic roomscale sensing), as is WebXR (no need to download to a headset or PC). But even within the experience, the “grammar” of how you navigate and use and interact with the space has got to be self-evident and common-sense. And no crashes or glitches or other “odd” happenings otherwise any sense of immersion is totally lost. And they need to integrate with your other digital presences - be that your desktop or work/social media accounts.

b) To make the applications desired. People and organisations have got to want to use this stuff. There has got to be user pull. For entertainment the experience has got to be worth all the hassle of setting up and clearing a space and totally isolating yourself from reality for a while, otherwise a film on Netflix or a game on Steam is going win out. For organisations the benefits of virtual learning and training have got to be clear and well understood. Yes there are lots of different case studies that show the benefits, but they aren’t well distributed or consolidated, and often aren’t too rigorous when comparing to BOTH the main alternatives (physical training and 2D eLearning).

In the medium-term (5-10 yrs) we identified two major challenges:

a) Mobility. Immersive learning needs to be available where and when people want to use it - it needs to be mobile. Yes an Oculus Quest is pretty mobile (I’ve even used it in my garden!), but in normal, non-COVID times, it's not feasible to use it on the bus or train into college, or sat in a cafe (locked away from the outside world with a purse or laptop ready to be stolen by your side), or sat on the sofa with half an eye on Love Island. Headset Mounted Display (HMD) VR needs to be complemented by mobile/tablet (and even laptop) based versions of the same experience. Yes there are advantages to HMDs (visceral immersion, scale, isolation), but there are also disadvantages (social isolation, nausea, convenience, cost). The user should decide HMD or non-HMD, not the software developer or trainer. 

b) Integration. We need to move away from walled gardens and towards standard based environments and applications. VR today is a bit like the pre-web Internet, different walled gardens, different access devices, multiple accounts. Users don’t want that - they want to hear about something and just access it with whatever device they have to hand, and with their long-standing personalised avatar. WebXR is helping here, and yes even the web still has many of these issues, and I know that asset wise we have broad portability between the platforms, but not in the scripting of the experience, or the management of the data associated with it. And does the “virtual world” approach of Second Life and Snowcrash offer a better model than the “app” approach of most current offerings. Many have talked about the Metaverse or Multiverse, being able to seamlessly (that word again) move from one virtual environment to another. There have been metaverse initiatives in the past - is it time for another one?

For the long term the group again identified 2 main issues:

a) Radical Interfaces. The VR HMD is a great step forward, but they are  still large, clunky, moderately uncomfortable for prolonged use, and not very portable. I’m pretty convinced that we need another big step change in HMDs before they become real consumer items where everyone one in they way that they currently (probably) have a tablet. What I have in mind is more like the holobands of Caprica than the Quest. Something that integrates VR, AR and MR, lets us readily see the physical world, tracks our hands, and, perhaps most important, manages to give us the “feeling” of locomotion, and perhaps the other senses. My guess is that this is as much a neurological interface as it is a visual one, and hence probably a decade or more out. 

b) Societal Change. VR is not just impacted by attitudes to it, but could also impact society itself. COVID has made us re-evaluate remote working and remote relationships. Popular media is full of stories based around virtualised people and places (Devs and Upload being just the latest examples). Even a decade ago virtual worlds were being used by hostile actors,  I doubt today’s environments are any different. How would a Caprica style virtual world, readily accessible by all, and with the capacity to do almost anything effect the way we all live and interact? Would it be for good or ill - and would it let us weather a second COVID that much better?

So there you are, 6 perspectives, 2 each for the short, medium and long term. You may not agree with all the details, but I hope that you can appreciate the general thrust of each, and each offers a timely call to action for the VR community.

Now scroll down a bit.

OK, I told a little white lie at the beginning there. The gathering of immersive learning experts wasn’t a few days ago, it was about 3,285 days ago at the ReLive (Research Into Learning in Virtual Environments) Conference. held at the Open University way back in 2011. Here's the original graphic - and you can find a fuller presentation I did later that years based upon it at https://www.slideshare.net/davidburden/virtual-worlds-a-future-history

But I think you’ll agree that the general vision and issues being raised back in 2011 differ little from what a similar analysis would yield in 2020 or even early in 2021 - ten years later! Some of the specifics might be different, and my commentary above reflects a contemporary take, but the big picture items are pretty much the same:

  • This stuff still isn’t seamless, although with Quest and WebXR we’re taking some great strides
  • The entertainment and business case is still struggling to be made. I know that Quests sold out early in lockdown, but I’ve also seen numerous reviews of technology to help with lockdown that haven’t even mentioned VR and immersive 3D.
  • We’ve actually made great strides in mobility if you consider non-HMD VR, I can now run avatar style experiences quite happily on my phone or tablet if they’re not too high-rez, and Quest again helps with instant set-up, but it’s still much of an either-or choice.
  • Integration seems further away than ever as VirBela, Immerse, AltSpaceVR, Sominium Space, Hubs etc all compete for users.
  • Radical interfaces is actually the one we achieved first, I was in SL in an Oculus DK1 in 2013, only 2 years after ReLive2011 - but as mentioned above there is still a long way to go for the ordinary consumer.
  • Societal change may be driven as much by COVID (and the fear of similar future outbreaks) and climate change, but VR is having far more of an impact on popular culture than it did a decade ago, and that triumvirate of VR capability, external pressures and cultural exemplars may well be driving change more quickly - although perhaps not a quickly as we thought back in 2011.

So I hope you’ll forgive my little deception, but I thought it might be a nice way to not only to illustrate how many things have stayed the same despite the apparent “improvements” in technology, but also highlights how much there is that current VR practioners can learn from the work on immersive environments that was being done a decade ago. For inspiration just check out the agenda and papers from ReLive11 (and the earlier ReLive08), still available on the OU website.

10 August 2020

Virtual Reality vs Immersive 3D - the Search for the Right Words!

As a company that been creating immersive experiences for over 15 years we find that the contemporary obsession with headset based virtual reality (HMD-VR) is often at risk of a) forgetting what valuable work has been done in the past in non-HMD immersive 3D environments and b) not highlighting to potential  clients that a lot of the benefits of "VR" can be obtained without an HMD, and not having the funds for, or access to (esp in COVID) HMDs does not need to stop a VR project in its tracks.

One problem is that we just don't have the right terminology, and what terminology we have is constantly changing.

"VR" has almost always been assumed to mean HMD-based experiences - using headsets like the Quest, Rift or Vive - or even their forerunners like the old Virtuality systems.

 But in that fallow period between Virtuality and Oculus DK1 3D virtual worlds such as Second Life, There.com and ActiveWorlds were enjoying a boom-time, and often found themselves labelled as "virtual reality".

One problem is that there seems to be no commonly accepted terms for the classic Second Life (or even Fortnite) experience, where you can freely roam a 3D environment but you have a 3rd (or sometime 1st) person avatar view of it. It's certainly not 2D. It's sort of 3D - but not as 3D as the stereoscopic experience using a VR-HMD. I've seen 2D/3D or "3D in 2D" but both are cumbersome. We sometimes refer to it as "first-person-shooter" style (but that doesn't go down well with some audiences), or "The Sims-like". 

There's also a qualitative difference between say a 3D CAD package where you're rotating a 3D model on screen (called an allocentric view) and the experience of running through Fortnite, Grand Theft Auto, or Second Life (called an egocentric view).  You feel "immersed" in the latter group, not just because of the egocentric view point but also because of the sense of agency and emotional engagement.

At a recent Engage event I went to I'd guess (from avatar hand positions) that about 50% of attendees were in VR-HMD and 50% using the immersive-3D desktop client. So should it be described as a VR or immersive 3D system? Our Trainingscapes is the same, we can have users on mobile, PC and VR-HMD devices all in-world, all interacting. And Second Life is often "dismissed" as not being "proper VR" - but when Oculus DK1 was around I went into SL in VR - see below - so did it stop being VR when they went from DK1 to DK2?

So if a system can support both - is it a 2D/3D system or a VR system? That is why we tend to refer to both the 2D/3D  approach and the VR-HMD approach as being "immersive 3D" - as long as you have a sense of agency and presence and the egocentric view. It's the experience and not the technology that counts.

And don't get me started on what "real" and "virtual" mean!

No wonder clients get confused if even we can't sort out what the right terms are, and its far too late for some de jure pronouncement. But perhaps we all could try and be a little bit more precise about what terms we do use, and whether they are just referring to the  means by which you access an experience (e.g. VR-HMD) or to the underlying experience itself (such as a virtual world or virtual training exercise).

In later posts I'll try and look more closely at the relative affordances of the 2D/3D approach (better name please!) vs the VR approach, what researchers experiences of virtual worlds can teach us about VR, and also how "virtual worlds" sit against other immersive 3D experiences.

30 July 2020

Garden VR

OK, why has it taken me 4 months of lockdown to realise that I've got the ideal room-scale VR space out in my garden! Having thought of the idea I did have some doubts about a) were there too few straight lines for it to manage the tracking, b) would rough grass flag as intruders in the scene and c) what happened if the dog walked through, but in the end it all worked swimmingly.

Wifi reaches about half way down - so that may be an issue, although I found it hard to draw out more  the first half the garden as a space. Oculus kept putting its "draw boundary" panel right where I was looking and walking and drawing didn't help - but I'll see if I can do better another time. I ended up with a space 15 paces by 7 - far bigger than the attic (and no slopey ceilings).

The image below shows a rough mapping of the space to the WebXR demo room - so I could walk about half of it (Oculus hides the warning barriers in photos - annoying in this case as I'd set myself up to show the exact extent!)

After that everything worked just as though I was indoors - apart from the occasional need to walk back closer to the recover the wifi. I certainly lost all sense of where I was in the garden and alignment, the soft grass didn't interfere with the immersion, and the slight slope up to the house end was a useful warning!

Not related to being in the garden I did notice that I felt more latency/unease with the 3D photospheres (and even more with the stereophotospheres) than with the true 3D spaces - where I felt none at all. Perhaps one reason why there were a lot of reports of unease with VR is a lot of people were having photosphere experiences - although admittedly true latency issues remain (but made worse by doing "crazy" things in VR - like rollercoasters - rather than just walking around in a room!

One experience which was heightened was the Mozilla vertigo experience - walking on ever smaller blocks over nothing. I suppose because I could move more in the garden I could better explore it and fully immerse myself in it - and it certainly made me check I could feel grass under my feet before I stepped - particularly when I just stepped off the blocks and into space.

Anyway all the space allowed me to have a good walk around the solar system model without using teleports and actually get the planets lined up for the first time! Even in the garden they are proving too big so need to at least halve the sizes!

13 July 2020

Further Adventures in WebXR - Playing with the Solar System

Having a bit more of a play with WebGL/WebXR and now have a nice draggable solar system! Could be a neat learning tool once finished to get the planets in the right order, too  look at their globes in more detail, and perhaps access further information about them. With World Space Day/Week going virtual might be time to set up a gallery of experiences for people to try that week. 

Need to sort a few more things first though - like rings for Saturn, a starscape backdrop, and change the highlight colours. Maybe also the option to check your order and give the you the right one. Also need to add a sun, and shrink the planets even further!

The more we play with WebGL/WebXR the more excited we are by it as a tactical solution, quickly creating small but powerful bespoke VR experiences that can be instantly accessed by anyone with a WebXR compatible VR headseat without any need for an install!

9 July 2020

Virtual Archaeology Review publishes Virtual Avebury Paper

The Virtual Archaeology Review has just published Professor Liz Falconer's paper on the Virtual Avebury project we did last year. The paper looks at the response to the VR experience by visitors to the National Trusts Avebury Visitor Centre - where two people at a time could collaboratively explore the Avebury site as was - i.e. without a village being built in the middle of it and all the missing stones replaced!

You can read the paper at: https://polipapers.upv.es/index.php/var/article/view/12924/12360

Key findings included:

  • More than 1200 members of the public experienced a 3D, fully immersive simulation of Avebury Henge, Wiltshire, UK over a nine-month period.
  • Patterns of use and familiarity with information technology (IT), and using mobile technologies for gaming were found that did not follow age and gender stereotypes.
  • There was little correlation between age, gender and IT familiarity with reactions to Virtual Avebury, suggesting that such simulations might have wide appeal for heritage site visitors.

Some of the key data are shown below:

Emotional Responses to Virtual Avebury

Experiences of Virtual Avebury

Responses to the Virtual Avebury Soundscape

Read the full paper at: https://polipapers.upv.es/index.php/var/article/view/12924/12360

6 July 2020

DadenU Day: WebXR

MozVR Hello WebXR Demo Room

For my DadenU Day I decided to get to grips with WebXR. WebXR is an new standard (well an evolution of WebVR) designed to enable web-based 3D/VR applications to detect and run on any connected VR or AR hardware, and to detect user input controls (both 6POS and hand controllers). This should mean that:

  • You can write and host VR applications natively on the web and launch then from a VR headsets built-in web browser
  • Not worry whether its Oculus, HTC or A.N.Other headset, both for display and for reading controllers
  • Have a 2D/3D view automatically available in the web browser for people without a VR HMD.
What WebXR does NOT do is actually build the scene, you use existing WebGL for that (essentially a 3D HTML standard, not to be confused with WebXR or WebVR!) through something like the Three.js or A-Frame frameworks.

To get a good sense of what web-delivered VR (via WebXR) can do I headed over to Mozilla's demo at https://blog.mozvr.com/hello-webxr/. This room has a bunch of different demos, and a couple of "doorways" to additional spaces with further demos. If you view on a 2D browser you just see the room, but can't navigate or interact (don't see why WebXR should pick up ASDW same way as its picks up a 6DOF controller). If you go to the page in your Oculus Quest (or other) browser you also see the same 3D scene in 2D. BUT it also offers you an "Enter VR" button, click this and your VR lobby and the 2D browser disappears and you are fully in the VR space as though you'd loaded a dedicated VR app. Awesome. In the space you can:

  • Play a virtual xylophone (2 sticks and sounds)
  • Spray virtual graffiti
  • Zoom in on some art
  • View 360 photospheres - lovely interface clicking on a small sphere that replaces the VR room with a full 360/720 photosphere. I'd always been dubious about mixing photospheres and full 3D models in the same app but his works well
  • View a stereoscopic 360 photosphere - so you can sense depth, pretty awesome
  • Enter a room to chase sound and animation effects
  • View a really nice photogrammetry statue which proves that web VR doesn't need to mean angular low-rez graphics 
MozVR Photogrammetry Demo

There's a really good "how we did it" post by the Mozilla team at: https://blog.mozvr.com/visualdev-hello-webxr/

Having seen just what you can do with WebXR the next step was to learn how its done. For that I went to the WebXR sample pages at https://immersive-web.github.io/webxr-samples/

Although thee are a lot simpler than the MozVR one, each shows how to do a particular task - such as user interaction, photospheres etc. You can also download the code and libraries for each from GitHub at https://github.com/immersive-web/webxr-samples.

Movement demo

Controller demo

The only downside of these seems to be that they use Cottontail - a small WebGL/WebXR library/framework purely developed for these demos and not recommended for general use - so adapting them to your own needs is not as simple as it would be if they were written in Three.js or A-Frame.

Keen to actually start making my own WebXR I started by copying the GitHUb repository to my own server and running the demo's up. Issue #1 was that any link from a web page to the WebXR page MUST use https, using http fails!

Starting simply I took the photosphere demo and replaced the image with one of my own. The image had worked fine on the photosphere display in Tabletop Simulator but refused to work in WebXR. Eventually I found that the image had to be in 2048x1024, higher resolutions (but same ratio) fail. Also the photosphere demo is for stereoscopic photospheres so you have to remove the " displayMode: 'stereoTopBottom'" parameter.

Hougoumont Farm at Waterloo in WebXR Photosphere

Next up was to try and add my own 3D object. I liked the block room in one of the demos and worked out how to remove their demo blocks form the middle, and to hide the stats screen. Nice empty room loaded up. Then I hit the bump that I usually write in Three.js or A-Frame and I could't just cut-and-past into their WebXR/Cottontail template. Then I ran out of time (it was Friday after all!)

I've now found a page of really basic Three.js WebXR demos at https://threejs.org/examples/?q=webxr so the aim for this week is to get those working and start on my own WebXR spaces.

It's obviously early days for WebXR, but given the MozVR demo this really could be a lovely download-free way of delivering both 2D/3D to ordinary browsers, and full VR to headsets without any downloads. Joy!