Virtual Reality (VR) v.s. Mixed Reality (MR)
Following the release of Virtual Reality (VR) products such as the HTC Vive and Oculus Rift, VR has already become the hottest topic of the year. This has also led to a wave of sales of high-end VR-Ready PCs. As a gaming hardware manufacturer, MSI recommends gamers use an Intel B150 or higher motherboard paired with an NVIDIA GeForce 1070 display card for the best VR experience, or just directly purchase the newest AEGIS TI desktop computer that already has VR Link capability.
We’ve listed all of our VR Ready products on this page for your reference. If your PC meets the necessary performance specs, you can refer to this previous article (MAINSTREAM VR GADGETS: GET READY FOR THE VR REVOLUTION) to figure out which VR gadgets you can consider buying. This other article (THE UPCOMING VR GAMES YOU SHOULD LOOK INTO) is good for learning about the latest games you can purchase. Also, We’ve released MSI Electric City, a VR experience game; those interested can download it from its exclusive webpage.
Now, let us talk about how to create mixed reality videos to create a virtual visual experience. Below are some videos illustrating the concept: (Below content is translated from The CY blog and agreed by the author.)
(HTC Vive SteamVR Promotional Video)
(Tilt Brush Promotional Video)
(Sound Stage Promotional Video)
If you already have a Vive and a VR Ready high-end PC, you will probably end up spending roughly US$900 buying everything listed in the MR Shopping List, excluding the video camera. You may be able to find these items at a cheaper price if you shop around for lower-priced distributors.
There are a few Unity games that support mixed reality video, such as Space Pirate Trainer, Job Simulator, Zenblade, Tilt Brush, and so on. Mixed reality recording mode will automatically open after a third controller has been connected and a special configuration document has been created. The third controller is used to position the camera to produce the virtual environment. The video feed of the user playing the game in actual reality is processed so that they background is removed. All of these images are compiled in real time by Open Broadcaster Software (OBS) to complete the mixed reality environment.
The purpose of a green screen studio is to allow for the background to be removed and replaced. You’ll need at least three pieces of green cloth, two for the walls and one for the ground. After setting up two racks to hang up two pieces of green cloth so that they overlap, you can lay down the final piece on the ground. For the best results, you could pull the hanging cloth taut and tape them to the racks so that they stay as smooth as possible. Even lighting within the studio set-up can ensure a better end result.
Capture cards or boxes are used to capture live video into OBS. It is normally a hassle to install a capture card, since you have to open up the computer case. I bought a GC550 LGX capture box because it connects directly to your PC with a USB cable (you don’t have to buy such an expensive model, look to my shopping list for other recommendations).
Copy the following text into a new Notepad file and save it as “externalcamera.cfg” (be sure to change the file extension to .cfg) or click here to directly download the file. Place this file into the directory where you want to open your MR game in (some supported games include Space Pirate Trainer, Job Simulator, Zenblade, and Tilt Brush).
The function of the third controller is to locate the position of the camera, so it needs to be placed in near the camera where it will remain completely untouched. Cameras will normally have a hot shoe mount, which can be used in conjunction with a clip to secure the controller, or you may choose to fasten it directly to the flash unit of your camera. In my trial, I used the Sony HDR-CX405’s flash unit hot shoe mount, added an L bracket and used an iPad clip (something I found while shopping around that was a perfect fit) to secure the controller on top of my camera.
The most optimal arrangement is to secure the controller to the camera horizontally, shown below, as this will make sync set up a lot easier.
(Photo Credit: ">Mixed Reality Trailers: Behind the Scenes)
The Unity plug-in will automatically establish the third controller as the one used to position the camera lens, so it is necessary to turn the controllers on in the following order:
The above directions ensure the controller used to position the camera is correctly detected as Controller 3.
We used Space Pirate Trainer in our example. After making sure externalcamera.cfg is within the game directory, launch your game and follow these instructions:
(The four screens are, respectively, Top left: foreground; Bottom left: Camera angle (Controller 3); Top right: Background; Bottom right: first-person viewpoint)
This example was conducted on a Full HD (1080p) monitor:
This is the last and also most troublesome step. The information within externalcamera.cfg includes the distance between the camera and Controller 3, the angle difference between the two, and the actual FOV quality of the camera. Perfect synchronization can be achieved by correctly adjusting the parameters within the document by following these steps:
(Positioning and direction of the camera and the controller’s preset position should be as illustrated above)
(This is a recording I made during my test run; if I spent more time fine-tuning, I would have able to produce better results)
Thanks to author The CY for the contribution.