Over the last several years, Virtual AND Augmented Reality have moved from being available exclusively in the military and aviation to the mainstream of professional development, as managers, instructors, coaches, and therapists have claimed increasing benefit from immersive experiences.

The healthcare industry is a large consumer of both VR and AR. They are used in patient care, provider education, and workforce development. This site is designed to provide you with a deeper dive into these technologies in healthcare.


The ideas that launched virtual and augmented reality date back to the 1800’s with the invention of photographic technology. The use of mirrors and light to capture images in time revolutionized how people thought about and looked at the environment around them. That technology and the desire to improve upon it led to the development of the View-Master in 1939, and the Sensorama (a pre-cursor to chromakey green screen in movies and television) in films in 1956.

By the 1960s, head mounted devices were being patented and introduced by several inventors wanting to be the first to give people a view of a virtual world. The term “virtual reality” (VR) was not widely used until the mid 1980s when inventor Jaron Lanier developed gear including goggles and gloves that the user would employ to experience a different type of reality. NASA was one of the first high profile organizations to use this new technology. The Virtual Interface Environment Workstation (VIEW) system combined a head-mounted device with gloves to enable simulated interaction in space or other environments.augmented-reality

Computing advances in the 1980s and 90s allowed augmented reality to become its own field of research and technology independent of VR. The term “augmented reality” (AR) appeared due to work done at Boeing that was designed to assist airplane factory workers in assembly by using googles that showed how wires were to be bundled. Since that time, AR has been widely used in other industries due to its ease of use and relatively low cost to implement  compared to VR. One of the most well-known examples is the National Football League’s use of a yellow line on our televisions to demonstrate to viewers the first down marker.

Today, as technology and computing are advancing and becoming more accessible to large numbers of people, VR and AR are being adopted in countless ways. Retailers use AR through apps on smartphones to enhance in store shopping experiences and educators at all levels are using both VR and AR in the classroom to bring learning to life. This is done through the use of Google Cardboard to “travel” to other places without leaving the classroom or AR flashcards that “come alive” when a smartphone or tablet are pointed at them. Soon, we may not know which “realities” are virtual and/or augmented.

VR and AR Timeline
Created by MH Walker 4/2018

Augmented (AR)

What is Augmented Reality?

“Augmented Reality or AR is is an enhanced version of reality where live direct or indirect views of physical, real-world environments are augmented with superimposed computer-generated images over a user’s view of the real world, thus enhancing one’s current perception of reality.” (http://www.realitytechnologies.com/augmented-reality)

Widely used examples include Pokemon GO! and Snapchat.

Current AR Trends in Healthcare

AR is currently one of the most exciting digital technologies available. It has the potential to change healthcare and everyday medicine for physicians and patients alike. Two aspects that make AR unique are the fact that it puts information into the users’ eyesight quickly and users do not lose touch with reality. These features, along with its relative ease of use and lower implementation cost than VR, give AR the ability to become a driving force in the future of healthcare.

close up of smart medical doctor working with smart phone and stethoscope on dark wooden desk with virtual icon diagramThe complexity of healthcare and medicine have made technology and innovation necessary in the industry. From using lasers in delicate brain surgery to adoption of electronic health records in independent primary care practices, technology and AR in particular, have limitless applications. Some of the current trends include:

  • AccuVein – a handheld device that allows healthcare professionals to see veins just under the surface of the patient’s skin, allowing for more accurate injections.
  • AED4EU – a smartphone app that uses GPS and overlays where to find the nearest automated external defibrillator (AED) in case of an emergency. Currently only available in Western Europe, but is rapidly expanding.
  • Cydar Medical – Uses 3D image overlays to visualize a patient’s anatomy during surgery, resulting in more accurate, safer procedures.
  • Microsoft HoloLens – There are many applications for HoloLens. Physicians can use it to give patients a better understanding of an upcoming procedure. The goggles could also be used for telemedicine, remote monitoring, training, and virtual care, among other things.

Augmented reality apps for all levels of healthcare can have rich functionality to aid professionals in their daily work and improve patient care. Installed on smartphones or smart glasses, AR apps can facilitate and improve doctor-patient communication, ensure accuracy of certain procedures, and help patients adjust to the new environment.

AR in Healthcare Education

AR snip
Click on the image above for the full infographic

Implementation Challenges

Although the future of AR seems to be limitless, there are still several skeptics. Several academics believe that there needs to be more research into learning theories that guide AR design. There also needs to be a cultural shift from the traditional styles of classroom learning to learning that is more technology and inquiry-based. Challenges for the future of AR include:

  • Hardware: Despite the emergence of smartphone and tablet apps that make AR more accessible, hardware, such as goggles or display boards are not widely available. These devices make the experience more immersive, but are too expensive for the average consumer and many small organizations. Beginning at $1,000 and going up, the price must go down in order for use to go up.
  • Content: Even if the cost decreases for the hardware mentioned above, there may not be enough content to support the devices. Lack of content or software is analogous to smartphones without apps.
  • Tech Overload: Technology has become such a large part of everyday life that many people complain about overdosing on tech. As people are encouraged to unplug or reduce screen time, AR has a possibility of becoming a causality of this fatigue.
  • Privacy and Safety Concerns: Recent data and privacy breaches at companies such as Facebook have raised concerns over consumer safety online. Massive regulation from the federal government has resulted and will continue to put the “squeeze” on AR innovators.


Virtual (VR)

What is Virtual Reality?

“VR is based on the diametrically opposite concept: It immerses real people inside a fictional digital, or virtual, space.”


Think PlayStation VR

Current VR Trends in Healthcare

Healthcare is one of the biggest adopters of virtual reality commonly used for surgery simulation, phobia treatment, robotic surgery and skills training. One of the advantages of this technology is that it allows healthcare professionals to learn new skills as well as refreshing existing ones in a safe environment. Plus it allows this without causing any danger to the patients.

The immersive qualities of VR make it a very good simulator, so it is excellent for ‘test-driving’ medical procedures. In surgery, for example, VR can be used to provide practitioners with incredibly high-definition 3D illustrations of specific organs or body parts, allowing them to ‘walk through’ or plan a procedure. This also has obvious benefits for surgical training.

Human simulation software – one example of this is the HumanSim system which enables doctors, nurses and other medical personnel to interact with others in an interactive environment. They engage in training scenarios in which they have to interact with a patient but within a 3D environment only. This is an immersive experience which measures the participant’s emotions via a series of sensors.

Virtual reality diagnostics – Virtual reality is often used as a diagnostic tool in that it enables doctors to arrive at a diagnosis in conjunction with other methods such as MRI scans. This removes the need for invasive procedures or surgery.

VR robotic surgery

Photo: University of Nebraska-Lincoln, Virtual Incision mini-robots conduct first known human surgery

Dmitry Oleynikov, a UNMC professor of surgery, operates a surgical robot as in the background Shane Farritor, a UNL engineering professor, adjusts the camera on the surgical subject in June 2015. The two developed the robot for minimally invasive surgeries. Farritor and Oleynikov’s startup company, Virtual Incision, announced March 1 the first use of its miniaturized robot in human surgery.

Virtual robotic surgery – A popular use of this technology is in robotic surgery. This is where surgery is performed by means of a robotic device – controlled by a human surgeon, which reduces time and risk of complications. Virtual reality has been also been used for training purposes and, in the field of remote telesurgery in which surgery is performed by the surgeon at a separate location to the patient.

The main feature of this system is force feedback as the surgeon needs to be able to gauge the amount of pressure to use when performing a delicate procedure.

But there is an issue of time delay or latency which is a serious concern as any delay – even a fraction of a second – can feel abnormal to the surgeon and interrupt the procedure. So there needs to be precise force feedback in place to prevent this.

Robotic surgery and other issues relating to virtual reality and medicine can be found in the virtual reality and healthcare section. This section contains a list of individual articles which discuss virtual reality in surgery etc.

Source: Virtual Reality Society 

VR in Education

Virtual Reality applications are being used by technological innovators in university-simulated clinical skills laboratories to support medical students’ technical competencies as they graduate into a digital healthcare environment. VR provides a safe innovation environment for healthcare solution developers to experiment with implementing technology to improve healthcare practice and faculty development. With the entry of consumer grade VR goggles, like: Hololens, HTC vive, Samsung Gear and even Google Cardboard, VR has hit a new wave of adoption and health care is starting to realize the opportunity VR offers in patient care.

The immersive experience that VR offers can provide learners a real opportunity to learn by doing, thereby increasing motivation and retention. It also provides a huge confidence boost to the learner, who is able to master skills in a virtual arena before applying these skills in real-world contexts. Immersive learning environments also allow learning from failures and encourage out-of-the-box thinking.

VR in education

Photos by Jennifer Jones, Georgia State University, CC BY-SA 2.0

Glenn Gunhouse, standing, takes a group of students, faculty, and staff on a virtual field trip of Rome using StreetView VR and the Oculus DK2.

Implications and Challenges

Currently, VR is making a comeback and trending in delivering immersive experiences to learning with many benefits in information retention and learner engagement; however, there are some questions about VR that we can’t possibly know the answer to just yet, so we’re forced to speculate and imagine.

  • As VR’s technology improves and develops, could the level of stimulation experienced in VR lead to confusion in the real world?
  • Since VR is physically risk-free, might we get too used to being able to survive large falls and end up having more accidents? Or,
  • Could virtual reality prove to be so compelling in the future that we end up choosing it over the real world?

For VR to be truly immersive, it must engage sensory organs including the use of haptic suits to create sensory engagement in the body: vibrations to emulate bumping into walls or getting hit by something; contraptions to emulate motion – wind on the face as if you’re flying. Ideally, there will be layers of sensory immersion added beyond eyes and ears.

The VR market is exploding with investment and set to grow exponentially, potentially changing our world, but, as with many technologies, the enthusiasm to embrace the new could mean we are not looking before we leap. And while the benefits are clear, there is little to no research around potential side effects of long exposure to VR.

Much like Augmented Reality, VR faces similar implementation challenges. According Chris Wiltz (2017), managing editor of Design News, says there are 5 major challenges VR will need to overcome:

  • Hardware: The hardware requirements for creating virtual environments include very large physical memories, multiple high-performance scalar processors high-bandwidth mass storage devices, and high-speed interface ports for various input and output peripherals. The National Research Council recommends for hardware development to remain largely a private-sector activity and for the federal government to only become involved if there are serious lags in development. Currently, appropriate VR  hardware exists but are too expensive for the average consumer.
  • Content: Even if you can overcome the price factor, both AR and VR lack compelling content. The number one challenge right now is that compelling content – content that’s habit forming from a consumer perspective. Think of touchscreens for example. At first touch seemed very gimmicky, but today if you touch a screen and it isn’t a touchscreen it feels like somebody broke it. It’s become habit forming and you count on it. So how do you that with VR?
  • Mobile: The problem with the cord is if a user is effectively locked into a box, restricted by the cord, it essentially defeats the purpose of having them immersed in a supposed 360-degree environment. The challenge of creating untethered, mobile VR is already being undertaken by a lot of companies – creating what’s called inside-out tracking, that allows VR headsets to track position without the use of external sensors.
  • More speed: VR and AR technologies, on all ends from collaborative enterprise to consumer entertainment, are already demanding more than ever from our computer hardware, and soon they’ll be doing it for our data communication speeds as well. The spectrum currently available for cellular communication is simply not adequate for the demands VR and AR will place on networks. Today’s 4G communication is based on sub-6 Ghz. That spectrum is not enough. You need to expand the spectrum to much higher bands.
  • Cybersecurity: As with any connected technology it’s only a matter of time before cybersecurity issues are raised. While there has yet to be any sort of high-profile hack or cyberattack conducted via VR, but anyone who follows cybersecurity should consider it only a matter of time given how little attention is being paid to the intersection of VR and cybersecurity right now. The same security measures companies have already applied to laptops and desktops will have to be applied to VR headsets and rigs. It could be another form of attack or all of sudden that expensive headset gets used somewhere else – that’s an important asset.

Source: DesignNews | 5 Major Challenges for VR to Overcome

Instructional Applications

Created by Guillermo J. Holguin 4/2018

HumanSim – Powered by  cutting-edge, real-time, human physiology engine, HumanSim facilitates self-paced learning and instructor facilitated team training via  browser based GO platform and mobile devices. HumanSim is a part of Advanced Learning Technologies leveraging simulation and digital games-based learning paradigms to accelerate learning, increase user proficiency, and reduce training costs.

Other HumanSim examples: