I/ITSEC 2022 in review

I was lucky enough to spend last week at I/ITSEC courtesy of Improbable where we were demonstrating our ASMSW Social Media Synthetic Wrap integrated with their Skyral system, and through that to VBS4, Montvieux's ASAIR and several other systems. Neat demo which drew a good crowd every time. Here are a couple of closer shots of the actual demo (our app top right in both).

In between the demos I had plenty of time to tour the hall and look at the other stands, although I couldn't get into any of the presentations being on an Exhibitor pass.

I must admit to being a bit underwhelmed by what I saw. Almost every stand had VR goggles, mostly high-end ones, or big screens, so this wasn't "accessible" kit, but being sold to defence organisations with big bucks. Despite some bias I think the Improbable offering was actually one of the more interesting and innovative, having a simulation that simultaneously linked the social media and other interactions of a ~250k population with the subjective experience of being in a riot that was driving and being driven by the larger social media response.

Anyway, here are my key takeaways form the show, and I've tried to group the key ones by themes.

Headsets

A good chance to play with the latest headsets from HTC and Varjo. With the Varjo XR-3 in particular  ($6495!) you could see the difference in fidelity against the Oculus Quest, and control panel on the training aircraft were crystal clear. With the HTC the posters on the IED trainer I was using were a bit more blurry, but probably better than the Quest. Neither as comfortable as the Quest.

Gloves

The HaptX gloves were on two stands. I only tried the older version. I hadn't appreciated that they were compressed air driven and needed a big floor mounted compressor (backpack in Mk 2) so not really very portable/domestic friendly. I also hadn't appreciated that since they were just air driven there was no hard-stop when touching a surface in the same way that old haptic stylus and controllers could be. The surgery demo was so-so, perhaps the gloves weren't well set-up, but the kids play farm was very good. When you "touched" something you felt enough feedback to say there was something there, but not enough to stop you pushing through. Holding your palm flat up and feeling raindrops under a cloud gave you a sense of the more delicate aspect of touch which, when combined with an animated fox prancing on your hand and the "feel" of her feet was very good and really heightened the immersion. 

The telling moment though was when I shut my eyes (in VR!)  and found that I could interact with the scene purely by touch, reaching out, picking up an object (closing my hands til feedback), moving the object (making sure I still had the pressure so I knew I hadn't dropped it), and then releasing it (confirmed by loss of pressure). The demonstrator said that in research they'd found that although we use our eyes for gross movements (e.g. put hand near coffee cup or aircraft control) we often then look away at something else and use touch to guide the final stages and then ,make the action. This neat demo proved that that was possible in VR.

I also tried out a more mechanical handset with control wires that gave a firmer touch response, but still not blocking, and just looked a bit Heath-Robinson at the moment.

Building Modelling

The closest to a "wow" moment was probably the companies showing how they could take oblique aerial/orbital photography of a city and then rather than do direct photogrammetry renders (which tend to have rounded corners and look very blurred at street level, so little use for tactical ground combat) were using AI to work out what sort of building it was and then working essentially with a library of building types to re-render it as a proper 3D model. Very neat. Presagis and Blackshark.ai were the stars here.

Avatars

The avatars on show in the apps range from pretty dreadful to OK, nothing really stunning. Here was a nice "life-size" avatar though, nicely animated and probably coming to (or already in) an NHS A&E department near you.

Non-Visual Modelling

Only one company I saw was really showing non-visual modelling, so for instance modelling RF and IR emissions in order to present more of a multi-spectral world - JRM technologies.

Mixed Reality

As well as the VR headsets, both of the main MR headsets were in evidence. Both were underwhelming. With the Hololens 2 I was able to look at an F16 (?) that was "parked" in the exhibition hall (and crushing several stands!). But the FOV really isn't much of an improvement on the Hololens Mk1 and it was impossible to get far enough back to see the whole plane in one go.

I had a good demo of the Magic Leap, showing an "Avatar" style 3D terrain model floating in space between us. If felt like a wider FOV, and being able to peer at the detail at ground level was really cool. BUT, the whole experience was very dark, so dark that I couldn't even see the hands of the demonstrator, so next to useless to work collaboratively pointing out different details. I assume its that they need either low ambient lighting (as in a hangar) or to filter the ambient light (as here) in order to get enough contrast with the projected image. The experience would have been a lot more effectively complete in VR.

Small Controllers

Lots and lots of pistols and automatic rifles instrumented to work as VR controllers, with room space tracking so you could practice your CQB drills. Queues were always too long to try out! There were also (appropriately) lots of medical mannequins around, some connected to VR, to allow you to practice the first aid afterwards.

Big Controllers

Of course the big draws at I/ITSEC were the "big" controllers: tanks, aircraft, helicopters, boats - you name it. Some used big/multiple screens and others used VR. Here are a few.

A "flappy bird" - reminiscent of an early UCL? VR experience

Other Cool Stuff

And some other things that caught my eye.

Best company name ever

US Army's augmented sandtable - always a great idea and demo

The robot-target, filling the A in LVCA.

Also the latest neuro-tech and physio-monitoring rig from Galea.

The Metaverse

There were about 17 companies (including Improbable) who listed or presented themselves as "metaverse" companies, and I'm sure that almost every salesperson would have been happy in discussion to call themselves the same. Quite honestly none of them were about the Metaverse as I understand it, and just as AI is being blurred beyond usefulness so too we were seeing metaverse being associated with almost anything that included a VR headset, and some which didn't even have that.

Conclusion

As I said at the top, overall underwhelmed. Just far too much "me too" when it came to VR-HMD based systems and not a lot of innovate thinking seems to be going on, and not a lot of democratisation (despite some "no code" vendors). Unlikely to ever go again, cost and time (but writing time by the pool was nice), but glad to have been and perhaps CES would give more of a future wow (but there again perhaps not).

Free Pedagogy for Virtual Reality Training and Education eBook Released

 

Last year David and Prof. Maggi Savin-Baden and Dr Victoria Mason-Robbie (both of the University of Worcester and regular Daden collaborators) produced a 56pp eBook for the Defence Science and Technology Laboratory (DSTL) of the UK's Ministry of Defence on the pedagogy of virtual reality. Whilst that version was approved for public release earlier this year we thought it might be nice to have a copy with non-military imagery and which removed the (few) military specific references. This new civilian version of "Pedagogy for Virtual Reality Training and Education" has now also been approved by MOD for public release and you can download a free copy of it right now!

If you've got any comments we'd love to hear them - just email info@daden.co.uk, and if we can help you think through VR and Metaverse issues more generally then again just drop us a line.

Effect of Non-Immersive Virtual Reality Simulation on Type 2 Diabetes Education for Nursing Students: A Randomised Controlled Trial

 

A nice paper that we co-authored with Heidi Singleton form Bournemouth University on her Diabetes project in VR has finally seen light of day. You can read it on open access at:

https://www.nursingsimulation.org/action/showPdf?pii=S1876-1399%2822%2900016-0

The abstract is:

Background: A virtual reality simulation was used to teach treatment of diabetic patients.

Methods: This study evaluated the impact of using virtual reality on short term knowledge of hypoglycaemia, via pairing of a randomised controlled trial, analysed via Partial Least Squares-Structural

Equation Modelling. The setting was two large lecture theatres based at campuses within the UK. Second year nursing students (n = 171) volunteered to take part in the study. Students were randomised into two groups, control (n = 88) and experimental (n = 83). The trial enabled comparison, via preand posttest surveys, of the simulation with normative teaching methods.

Results: VR was found to be significantly (p ≤ .001) better in terms of hypoglycaemia knowledge than normative methods. The method also enabled identification of the key point of action of the simulation, which evidenced that the “engagement to immersion” pathway was responsible for leading to higher knowledge scores in the experimental group.

Conclusion: This paper claims addition to knowledge about how the novel approach taken has the potential to deepen understanding of how virtual technologies can affect learning in nurse education.

https://doi.org/10.1016/j.ecns.2022.02.009.

Case Study - Social Worker Home Visits at the OU

 

Home visits are a key part of a social worker's life, but are notoriously hard to train for. Whilst roleplay can help the fact that the role-play is typically done in the classroom loses the immersion, and its only available when the course sets it up. Creating a home visit in VR eliminates both these issues - offering full immersion and 24/7 learning. 

Over the last year we've been helping researchers at the Open University build a home visit learning experience for their social work students, so that the students can work through a typical home visit (with support case notes and messages from colleagues) to try and work out what is going on.

The students get to talk to the mother, father and two of the children. Whilst we've used audio as well as text for the dialogues so that students can better judge emotional cues we've also implemented an "emotional barometer" alongside the chat to make plain text chat sections more able to convey the same sort of information.

You can read more about the project by downloading the case study.

Case Study - Virtual Field Trips

 

Though we'd refresh our case study of the Virtual Skiddaw project we did for the Open University a while back. The simulation modelled 100 sqm km (!) of the UK Lake District with 6 sites modelled at about ~1cm accuracy with photogrammetry. Students can pick up hand samples, look at virtual microscope slides, fly (!) around the environment and even lift up a cross-section of the rock!

The case study is downloadable here. 

There are also a couple of great videos - a narrated walkthrough by the OU's geology lead on the project.

And the OU's Geology and vLearning leads discussing the project.

The app is still in use with students at the OU.

Daden Video Presentation(s) at the GIANT Health Conference

 

David presented at the GIANT Health Conference on 2nd Dec, giving an overview of our increasing number of health related immersive 3D/VR projects. You can watch TWO versions of  the 20 min presentation and Q&A session by clicking on the images above due to some confusion in GIANT arrangements, the second has an interesting chat about future VR ecosystems at the end of the Q&A.

(Both videos should start about 10-15 secs before David's piece)

New Case Study - A VRLE For Midwifery Students

 

Over the last year or so we've delivered 3 simulations for use by midwifery students at Bournemouth University as part of a Virtual Reality Learning Environment (VRLE) project. The exercises cover:

• A Urinalysis appointment at a clinic with an expectant mother
• A home visit to do a routine post-birth baby & mother check and a check on safeguarding issues
• Dealing with a post-partum haemorrhage in a hospital ward

Student responses have included:

“It is a different and enjoyable way to learn"

“I think it is a good way of demonstrating what it is like to be in someone else’s shoes.”

"The VRLE is a really useful learning tool that I hope is continuously used throughout the duration of our course.”

You can view a video presentation on the exercises by BU project lead Denyse King on YouTube:

Detailed results will be published in Denyse's doctoral research but “the evidence gathered from 311

healthcare students demonstrates that VRLE have a statistically significant positive impact on: learning, confidence, clinical intuition, humanisation of healthcare and students’ perceived positive onward impact on service user safety.”

Our latest case study summarising the project is also now available - just click to download.

Case Study - In-Patient Diabetes Management

​With Bournemouth University we've been developing a training exercise for students to practice their response to a diabetes situation with an in-patient on the ward. 

Students could access the exercise with their from a normal PC, tablet or smartphone, or using a Google Cardboard VR headset.

The exercises was used for an A-B comparison with traditional eLearning and students on the 3D/VR exercise which found that the latter group performed "significantly better". 

Several research papers are coming, but our favourite student response was "“I really enjoyed it. Can we have more scenarios please?”!

You can read a bit more about the project and findings in our short case-study.

Trainingscapes Design Decisions

 

I thought it might be useful to write a bit about the key design decision which informed the development of Trainingscapes - and which going forward may give us some novel ways of deploying the system.

An early PIVOTE exercise in Second Life

Trainingscapes has its roots in PIVOTE. When we first started developing 3D immersive training in Second Life we rapidly realised that we didn't want to code a training system in Second Life (Linden Scripting Language is fun, but not that scalable). Looking around we found that many Med Schools used "virtual patients" - which could just be a pen-and-paper exercise, but more often a 2D eLearning exercise, and that there was an emerging XML based standard for exchanging these virtual patients called Medbiquitous Virtual Patients (MVP). We realised that we could use this as a basis of a new system - which we called PIVOTE. 

A few years later PIVOTE was succeeded by OOPAL - which streamlined the engine, and also added a 2D web-based layout designer (which then put props in their correct places in SL), and then when we built Fieldscapes/Trainingscapes which refined the engine further.

The key point in PIVOTE is that the objects which the user sees in the world (which we call props) are, of themselves, dumb. They are just a set of 3D polygons. What is important is their semantic value - what they represent. In PIVOTE the prop is linked to its semantic value by a simple prop ID. The propID is what is the exercise file, not the prop itself.

Within the exercise file we define how each prop responds to the standard user interactions - touch, collide, detect (we've also added sit and hear in the past), any rules which control this, and what actions the prop should do in response to the interaction. So the PIVOTE engine just receives from the exercise environment  message that a particular propID has experienced a particular interaction, and then sends back to the environment a set of actions for a variety of props (and the UI, and even the environment) to do in response. The PIVOTE engine looks after any state tracking.

The main Trainingscapes exercise/prop authoring panel

This design completely separates the "look" of an exercise from its logic. This is absolutely key to Trainingscapes, and in day-to-day use has a number of benefits:

• We can develop exercises with placeholders, and switch them out for the "final" prop just by changing the 3D model that's linked to the prop ID.
• We can use the same prop in multiple exercises, and its behaviour can be different in every exercise.

Original SL version of the PIVOTE flow - but just change SL for any 3D (or audio) environment

More significantly though it opens up up a whole bunch of opportunities:

• The PIVOTE "player" can sit at the end of a simple web API, it doesn't have to be embedded in the player (as we currently do in Trainingscapes so that it can be off line)
• If a 3D environment can provide a library of objects and a simple scripting language with web API calls then we can use PIVOTE to drive exercises in it. This is what we did with Second Life and OpenSim - perhaps we can start to do this in AltSpaceVR, VirBela/Frame, Mozilla Hubs etc.
• By the same measure we could create our own WebGL/WebXR/OpenXR environment and let users on the web play Trainingscapes exercises without downloading the Trainingscapes player.
• There is no reason why props should be visual, digital 3D objects. They could be sound objects, making exercises potentially playable by users with a virtual impairment - we've already done a prototype of an audio Trainingscapes player. 
• They could even be real-world objects - adding a tactile dimension!

Sounds like that should keep us busy for a good few DadenU days - and if you'd like to see how the PIVOTE engine could integrate into your platform just ask!

Time to Revisit 2D/3D?

The COVID epidemic is bringing a whole host of challenges to tutors and trainers in trying to deliver engaging teaching to students and staff – and in particular learning experiences which give the student a sense of context, place and shared experience. 

With on-site and classroom learning facing the greatest problems, many are naturally turning to remote teaching by Zoom (and similar), eLearning systems, or video. The diagram below briefly highlights key challenges with each of these - and we'll dive into more detail next week.

Whilst the uptake of headset-based virtual reality (HMD-VR) for both entertainment and training/education has been increasing in the last year or so (and Oculus sold out of the Quest headset early in the pandemic, and its new Quest2 had 5 times the pre-orders of the Quest1), the HMD-VR approach has its own challenges during COVID:

• Sharing headsets within a central location needs rigorous cleaning between use

• Posting/couriering headsets around remote students is costly and risky

• The cost of equipping every student with their own VR HMD is too expensive for most organisations/institutions.

And this is quiet apart from the challenges of the space to use them (although we’ve found the garden is good – at least in summer!) and the discomfort that some users feel.

So given the current situation shouldn't tutors and trainers be re-considering the 2D/3D approach to immersive learning – the Sims/Computer Game style where you operate in a 3D environment but the experience is delivered on an ordinary 2D screen – be that on a PC/Mac, tablet or even smartphone.

 

Such systems have the advantages that:

• Compared to Zoom you place students in the environment and they can learn when they want and a their own pace.
• Compared to eLearning they have more agency, being able to tackle tasks in a variety of orders and ways, and can even undertake collaborative activities.
• Compared to video the students again have more agency, more ability to explore the environment, and content can be changed on a regular basis for minimal or no cost.
• Compared to HMD-VR there are no headsets to buy, clean or distribute, and students can potentially use their own smartphones or tablets alongside their own laptops

Such 2D/3D approaches appear to have been forgotten and brushed aside in the race to the latest "shiny" VR experiences, despite the fact that whilst there is a lot of evidence comparing traditional eLearning to VR there is little that shows significant benefits from VR as against 2D/3D - and even some which shows a negative "benefit"!

So isn't it time to re-assess the 2D/3D approach (and perhaps find a better name for it!)?

And of course with Trainingscapes you can generate content for 2D/3D and VR at the same time, so if the situation changes and VR becomes more do'able (for some students, some of the time) then you'll be ready.

Live - Virtual - Constructive - Autonomous? - A Framework for Training Approaches

In military and defence training circles there is a very commonly used acronym called "LVC". It stands for Live, Virtual , Constructive:

• Live training: Real soldiers exercising against real soldiers on a training ground (and more broadly real soldiers practicing with real kit anywhere physical)
• Virtual training: Real soldiers exercising against real soldiers in a simulation, probably (but not necessarily) digital (and again more broadly real soldiers practicing with virtual kit anywhere non-physical, would also cover Fortnite!)
• Constructive training: Real soldiers exercising against so-called "computer generated forces" (CGF) with a simulator (think almost any traditional first person shooter)

When I first came across the model I struggled to remember what was what as so tried to put it into some sort of 2x2 matrix. The dimensions I eventually decided were:

• Whether the training environment was the real physical world, or a digital (or other) simulation of it, and
• Whether the opposition was being controlled by a human or a computer

One of the reasons I struggled was that nobody ever talked about the 4th space on the matrix - where you train in a physical environment but the opposition is computer controlled - which I've labelled Autonomous - hence LVCA. This is odd since this mode is not as far fetched as it sounds, fighter pilots have been practicing against UAVs for a while (although they may be remote-controller not autonomous), and there is some emerging work on mobile semi-autonomous robot targets (see https://www.youtube.com/watch?v=tlqMlPQpeAo)

So a complete matrix might look like this:

Now whilst emerging from the military and defence world it does seem to me that this LVCA gives  useful model for thinking about skills and process type training within the civilian world. Of course we need to think about "non-player characters" (actors? - who may represent clients, patients, customers or colleagues) instead of the "opposition", but I think the model holds up pretty well:

• Live training: Students learning with real people/kit – e.g. role-play, practical hands-on
• Virtual training: Students learning with real people via a simulator (e.g. Trainingscapes) (or maybe Zoom) (virtual role-play)
• Constructive training: Students learning by interacting with computer controlled NPCs in a simulation
• Autonomous training: Students learning by interacting with computer controlled physical entities (e.g. hi-spec mannequins)

As ever the key is that these approaches aren't competing, it's about finding the right blend of each given the subject, students, situation and budget in order to deliver the best possible training - and to help provide follow-up to keep that training fresh.

Hopefully this LVCA matrix will give you some fresh insights into the training you are trying to deliver, and perhaps open up some new ideas as to how it could be delivered for the benefit of your organisation and, of course, your students.

MS&T Feature Daden Thoughts on the impact of COVID19 on Virtual Training

Military Simulation And Training Magasine Editor Andy Fawkes has published a video drawing on the views of S&T industry leaders to discuss how the pandemic has accelerated existing digital trends and that this is the time to reimagine the management and delivery of simulation and training. Prompted by some of our posts here on the relative merits and experiences of 3D and VR immersive training, and the impact of COVID on the market, Andy includes some of our thoughts in the video.

You can view the video at: https://www.halldale.com/articles/17613-digital-trends-accelerated

MS&T Editor Andy Fawkes

Fire Extinguisher Exercise Walkthrough

 

We've just posted up a video walkthrough of our fire extinguisher exercise which shows you how different fire extinguisher types are applicable to different types of fires. We find its a great application to quickly get people immersed into 3D/VR immersive training and to "have a go" at the technology as fires and fire extinguishers are something we can all relate too. This is not intended as a fire training exercise, but rather an opportunity to see how 3D/VR immersive training can be applied to a topic.

The exercise was authored using our Trainingscapes authoring system, with a "Fire" widget to control the relationship between the fire types and extinguisher types.

Just get in touch if you'd like to talk through how this sort of immersive (and remote) learning could be applied in your organisation, particularly at a time when it remains challenging to do face-to-face training and teaching.

9 Business as Usual Uses for VR and AR in College Classrooms - Our take!

Saw an interesting looking article on "9 Amazing Uses for VR and AR in College Classrooms" on  Immersive Learning news the other day - although actually a retweet of a 2019 article. But reading it I was struck by how most of the uses they talk about are things that we've been doing for years.

So here's their Top 9 uses, and what we've done that's identical or close.

1) Grasping Concepts

When we built a virtual lab for the University of Leicester we also built 3D animations of what happens at  a molecular level. Students had found it hard to link the theory of a process with the mechanics of using the kit, and the combination of both really helped them to link and understand the two.

In another example a finance trainer we helped build for the University of Central Florida represented financial flows as tanks of water and piles of virtual money so as to better enable students to grasp more complex financial concepts.

2) Recreating Past Experiences for New Learners

Not one of ours but there was an awesome recreations of the WW1 trenches, augmented by the poetry of the war created by the University of Oxford back in the 2000s. We have though also used immersive 3D to recreate conversations between analysts and patients so that new learners can revisit these and actually sit in the virtual shoes of the analyst or patient.

3) Stagecraft for Theater Students

One of the first projects we got involved with was helping theatre educators at Coventry University make of us immersive 3D to teach stage craft and even create new cross-media pieces. There was also the wonderful Theatron project back in the 2000s that recreated a set of ancient theatres in order to better understand how they were used by staging virtual plays, and we did the Theatrebase project were we built Birmingham's Hippodrome Theatre and digitised a set of scenery from their archives to show how virtual environments could be used to both teach stagecraft but also to act as an interactive archives and to help plan and share stage sets between venues.

4) Virtual Reconstruction of History

With Bournemouth University and the National Trust we recreated Avebury Ring as part of an AHRC funded project and ran it for the summer at the visitors centre so that visitors could explore the Ring as it was 5000 years ago in VR - and without the village that has now been built in the middle of it!

5) Going on Space Walks

We've done the Apollo 11 Tranquility Base site 3 times now, in Second Life, Open Sim and now Trainingscapes. We've also done an exploration of the 67P comet and a whole Solar System explorer.

6) Reimagining the Future

                               

Back in 2010 we built the new Library of Birmingham virtually (hence VLOB) for Birmingham City Council so they could use it to plan the new building and to engage with the public and later subcontractors. The multi-user space even had a magic carpet ride!

7) Practicing Clinical Care

We have done almost a dozen immersive 3D exercises for health and care workers, ranging from paramedics and urinalysis to end of life pathway care and hospitalised diabetic patients.

8) Hands-on Railroading

OK, hands-up, we've never built a virtual railroad - but we have done equipment operation simulations on things ranging from air conditioners to jet engines!

9) Feeling the Impact of Decisions

In the article this is actually about team-work and collaboration within virtual spaces. Whilst we have had some "fun" builds - for instance virtual snowballs for Christmas parties we're also really interested in how to use these spaces to discuss issues and approaches through tools like walk-maps and 3D post-it notes. The classic though has got to be the fire demo where if you choose the wrong extinguisher then the fire blows up in your face - and as seen from the image above your body flinches away exactly as it would do in real life!

So there you are, 9 business as usual use cases for immersive 3D and VR as far as we're concerned!