Virtual Reality (VR) content produced by Ghost Productions is interactive, immersive 3-D stereoscopic animation content delivered through a device worn on the head. This ground-breaking technology creates the convincing illusion of entry into a virtual new world. Some systems even allow the viewer to interact at will with this virtual space and manipulate encountered, virtual objects.
A number of applications have been developed over the past few years to deliver this immersive content. Perhaps the best-known tool is Oculus Rift — the result of a kick-starter program to develop affordable VR to the general public. Ghost Productions was invited to participate in content development well before the launch of the final product. We began adapting our surgical animation content for the Oculus Rift as soon as we received development kits I and II. Oculus Rift is presently owned by social media giant, Facebook.
We had previously designed various approaches to adapting our content for viewing on Oculus Rift, when Google released a competing technology: Cardboard. We felt that developing content for Google Cardboard deserved serious attention. In September 2015, approximately 10 months after its release, we were well positioned to develop Google Cardboard content.
Our latest endeavor is to elevate our animation to state-of-the-art, fully rendered medical VR. Our VR science content is currently rendered in a number of ways to achieve the 3-D, interactive qualities that characterize “real” VR. Headsets — like Gear VR (Samsung) — use content that is projected on a virtual cube instead of a sphere, as in Cardboard. This format shows great promise. Multinational conglomerate Samsung announced it will release a free Gear VR rig with every mobile phone sold in 2016, so we are focusing on producing content for this platform as well.
Fully rendered 3-D, omni-directional, stereoscopic content for Google Cardboard, Oculus Rift and Gear VR is produced in a number of ways: Among them, spherical, rectilinear and cubic rendering approaches are the most common. From a production standpoint, each has merits, yet presents challenges.
At Ghost Productions, we have developed production processes to maximize content presentation using any of these three formats. Accordingly, we are already well positioned to enable clients to take advantage of these compelling formats. This gives us the flexibility to produce stunning 3-D animation content that’s readily adaptable to the most popular VR projection systems. Among the most successful so far are Google Cardboard, Oculus Rift and Gear VR. Each requires special content formats.
Movie formats for VR can be broken down into two categories: spherical and cubic. Spherical formats are rendered by unwrapping a spherical projection into a rectangular format by “stretching” or expanding the image at the poles. It’s a bit like a Mercator map that shows Antarctica as a long, continuous landmass at the bottom.
Cubic formats render six square images that are projected separately onto a cube. The corners of the six images are stretched, so that when viewed from the exact center of the cube, the images seamlessly join to give the illusion that the content is being projected onto a sphere.
The movies are then formatted in a number of ways to prepare them for viewing on these systems. To produce content that is 3-D stereoscopic, two versions of each animation must be produced — one for each eye. These component movies are assembled into a single, seamless movie that sends the right and left channels to the appropriate positions for the VR system to deliver to the relevant eye for viewing.
Google Cardboard projections typically use a format called OVR (OVer, undeR). This format puts the right channel (eye) over, or on top of, the left channel (eye), creating a square movie that, when cut in half, contains the unwrapped spheric renders.
Content for Gear VR is assembled for cubic projection as a rectilinear image that assembles each square image in a long line. For stereoscopic content, this means each frame is a very long line of 12 squares. The cubic format assembles the square images into a stacked version of this render so that the back, top and bottom are in a line above the left, front and right. These in turn are stacked OVR for delivery to Cardboard VR players. The advantage of this format is that it is more easily compressed.
1) Full-Color Rendered Animation Movies (Spherical 3-D OVR)
For full-color rendered animation content, we utilize raytracing to create an omni-directional stereoscopic two-channel rendering. This image is an unwrapped, 3-D spherical image projected by Google Cardboard software onto a virtual sphere, centered on each eye. This format benefits from being more affordable and achievable within limited production timelines.
We are able to produce for this solution within a production schedule timeframe similar to that required to produce traditional linear 3-D animations. We render our content from a fixed point of view to minimize viewer disorientation.
2) Hardware-Driven Google Cardboard (Unity3d)
We can also produce VR content using a hardware-driven solution similar to a game engine. The engine we produce with is called Unity3d. We’re able to achieve big improvements in visual output clarity, because content is rendered using all the screen resolution available for any specific direction the viewer gazes at.
This sort of production is constantly being improved, but it is limited by the hardware capabilities of the device used to view the animated content. Care must be taken to create an animation that is optimized for mobile device playback, while offering the highest quality output possible. There are some limitations for image quality, which, in traditional animation, is normally processed for each frame.
By harnessing the power of this emerging technology, we’re able to produce animation content that’s more immersive and interactive. Accordingly, it’s also more effective at engaging our target audiences. The end result is higher levels of comprehension, which helps support our clients’ communication goals.
VR content also offers a significantly grander stage to support more effective messaging. Content is not limited to the typical paradigm of a rectangular frame; a window on a selected segment of representational reality. Rather, it essentially allows the viewer to step outside the window and see the complete landscape.
For example, medical VR audiences can look anywhere in a 360-degree spherical world, where additional content exists to support any communication goal. Immersive, interactive and engaging — these are the hallmarks of effective media. It’s what we strive for in all our 3-D animation productions. VR represents a natural technological evolution, which enables us to take our content to the next level, and beyond.
We partnered with a forward-thinking company, LDR Spine, to produce our first practical application for public release. Our work for LDR Spine puts the viewer into a fully rendered 3-D VR to demonstrate the procedure to implant a spinal-implant device. We were also able to show how a complicated tool is assembled from the viewer’s perspective, in 3-D. This helps our clients train more effectively by immersing the audience in the content. The viewer can see, in detail, how this tool is assembled, from a perspective that realistically renders the viewer’s experience of the activity.
Imagine a time before motion pictures existed. Still photography had been around for decades, but it suffered from technical deficits. Most subjects were necessarily static, as primitive film stock required long exposures, under intense light, to render a recognizable image. Then came motion pictures. According to reports at the time, audiences in Paris who viewed samples of some of the world’s first motion pictures gasped, jumped out of their seats, or even fainted. Despite its primitive image quality, this remarkable new technology evoked a new level of “reality” on a 2-D screen. A simple film of a train entering a station was initially perceived as thrilling, if not terrifying.
Today’s VR technology evokes some of those same feelings. Audiences have been known to reach out and attempt to touch objects that appear to be floating in front of their faces. With rapidly improving resolutions, rendering and haptic technologies, it’s now possible to view surgical procedures from multiple vantage points. Haptic devices, which can create the illusion of interacting physically with objects in the VR environment, are being pioneered to further increase the illusion of total immersion in a virtual world. This virtual reality medical technology is being developed to enhance surgeon training.
Some of the potential advantages of this technology are obvious: As noted above, we created a virtual experience that demonstrated a medical device implantation procedure. Medical virtual reality is also being used to train students in subjects such as cellular biology, which underscores another advantage of this technology. In such a virtual environment, students can experience the sensation of exploring a microscopic world, as if they’d been shrunken to the level of a macromolecule themselves.
Imagine poking the lipid bilayers of the endoplasmic reticulum or wandering through the coils of chromatin in a cell nucleus. Needless to say, this is not your grandfather’s dry cellular biology textbook, with simple VR medical illustrations. Rather, it opens a virtual, immersive world to explore. Learning and engagement are naturally enhanced many-fold when students have the opportunity to engage with unseen worlds in this remarkably intimate fashion.
The same is true of patients, doctors, administrators, CEOs, purchasing agents — anyone who might benefit from a better understanding of your medical device, procedure, service or concept. Comprehension skyrockets, while time spent explaining complex subjects decreases dramatically. Just as there will never be a return to old “standard-definition” television in an age of HDTV (and pending 4K television), it’s unlikely that simple 2-D renderings will continue to suffice for marketing purposes much longer in the highly competitive medical device arena.
Just as motion pictures have come a long way in a relatively short time since that initial clip of a fast-moving train shocked Parisian audiences, it’s all but certain that VR technology will continue to improve, innovate and amaze. As noted above, VR technology is being developed to include not only visual and aural stimuli, but tactile stimuli as well. Sophisticated surgical training programs around the world are already investing in this technology, and combining it with real-patient, 3-D scanning technologies to provide patient-specific models for pre-surgical planning that’s informed by the patient’s own idiosyncratic anatomical features.
Medicine appears poised to benefit from a happy convergence of recent and emerging 3-D technologies, including high-resolution 3-D imaging, increasingly sophisticated 3-D printing, and of course, 3-D VR. It’s a brave new world, and like most technological advances, it’s all but inevitable. The question that remains is this: Will you eventually play catch-up, or will you be in the vanguard of those who have embraced — and benefitted from — these new virtual reality science technologies?
We have never stop evolving and working to advance the sophistication of the VR imagery we include in medical applications. We are constantly adapting our 3-D animation content to make it more effective and engaging. With applications for VR in medicine, we are empowered to improve the effectiveness of our primary goal: To contribute to the betterment of human kind by improving the understanding of virtual reality medicine and science.
We invite you to contact us today, to learn more about the various ways Ghost Productions can work with you to produce exciting VR content that informs, educates, illuminates and amazes.