The year, 2017 has become a remarkable year, this year brought continued growth and awareness on Augmented, Virtual reality, and Mixed Reality. Various questions lingering in the minds of newcomers, trying to understand the new fast growing digital phenomenon in the digital world.
Augmented and Virtual reality looks like similar features, but there is a lot of difference in their fundamentals. A digital environment that shuts out the real world is called Virtual reality.
Augmented reality places the real world in the physical environment around you.
Mixed reality is a hybrid of both the Virtual reality and Augmented reality.
What is Mixed Reality Video?
Making of mixed reality video is considered to be a practice of creating a video that looks like the Virtual Reality user is placed in the virtual world what they are seeing.
Mixed reality works on scanning the physical environment, creating a 3D map of the surroundings, so the device knows when and how to place the digital content realistically, it allows to interact using gestures.
The basic technique involved in mixed reality video is a green screen environment, a camera to shoot the footage of everything and a user experiencing the virtual reality world using VR headset.
Many people familiar with Augmented reality experiencing the graphics in real world, but the mixed reality is you can overlay real life over computer graphics.
Why Mixed Reality Video?
Mixed reality is the best way of converting virtual reality life into the 2D format. Translating the user experience on virtual reality world on small and big screens, it is difficult, but mixed reality proved it’s an easy task.
In today’s life, the virtual reality is still incredible to the majority of the people. Why mixed reality is, to show and experience the people what virtual reality user doing, screen it show in real life.
The current virtual reality videos, walking around and weaving in virtual reality worlds. The first head involved in mixed reality was a camera footage, the second head is a person who is experiencing the VR world.
Mixed reality is something like spectacular Cam from VR games with a green screen video of the player is the final results.
There are some fake videos on mixed reality, experiencing the 3D scanned model of a person’s face with an HTC Vive headset is taking reaction on digital world replaced with some face. Those are all face gaming technology software..
But now, Mixed reality, focusing on how the virtual reality world works in real life with a VR player in the video.
Some of the game publishers like Radial Games and Owlchemy labs done some extensive work on mixed reality and shared their view of the greater VR community.
How to Make a Mixed Reality Video
We all are familiar with the topic of mixed reality video, and we are enjoying the gameplay of mixed reality video as the real-time experience. Mixed Reality Video could be the most adventurous thing to do the mixed reality video creatively. Do you know that how to make a mixed reality video? Before making it just go through the below steps that give the clear idea.
Mixed Reality Video Research and Development
The main thing that we should know while implementing the mixed reality video is vive controllers which are the most important in the experience of mixed reality videos.
These vive controllers will act as media to interact human body with the objects in the virtual reality with help of sensors attached to it.
When we had decided to use a 3rd vive controller then it’s better to get the stream controller dongle which can be accessed wirelessly by installing the steam VR firmware or USB extension card can be used to operate the vive controllers.
To have a wireless function the steam VR firmware can be used.
Mixed Reality Video Camera Calibration
We have the entire equipment to make mixed reality video but how to merge the real controllers with virtual ones for that we need to calibrate.
The calibration tool helps in providing the secured file to use bulk applications.
When we use a 3rd vive controllers to make a mixed reality video then open the application of mixed reality camera calibration helper v0.1.
Here we can find position control, rotation control and field of view control then we must adjust the position of vive controllers by using position control.
After that move to rotation control which merges the VR controllers and real controllers by rotating.
Finally, find the field of view for the camera lens of vive controllers that allows VR controllers to match with the real vive controllers and then save.
This configuration can be used for every time you make mixed reality videos.
Mixed Reality Video Cameras
All DSLR cameras can be used with complete camera equipment to shoot the mixed reality videos.
Some of the cameras that are most preferable are Cannon, Red, Sony alpha a7S II with Movi-pro stabilizer.
We can pick up all cameras which give 4k resolution and support the vive controllers.
Mixed Reality Video Frame Rate
The 24 fps might be well suited to make mixed reality video which could give the experience as a movie and due to this, the camera stabilizer can be handled smoothly without any problem.
The other choice is we can go for choosing 60 fps when we need to record the live action moments with no other higher involvement of the player and the activity of the camera.
Including these two, go for opting 30 fps which are also the better way to capture the image.
Mixed Reality Video Slating
When we click the record option for many times that’s not stored in the personal computer.
If we want to club those actions then it might be very difficult to do that.
The solution for that is using the slate which could store the shots of the camera visually without being interrupted by the names of the documents.
When you want to modify the angles of the green screen then we can go for the best use of hotkey in the system which can be the best fit in the space of 180 degrees.
Mixed Reality Video Green Screen Stage
To get a perfect mixed reality video we can use green screen and green sike wall.
Otherwise, choose the studio which provides the full equipment including green screen lightening kit.
Before shooting just have a look that there are no shrinks on the green screen which spoils the environment of the game.
Then find whether we have two 500 watt photoflood light bulbs with collapsible softboxes and stand which gives the background lightening.
Just move the lights away from the edge of the screen which holds the track of the camera controllers, HMD and controllers.
This type of green screen will provide the gaming environment of good quality.
Mixed Reality Video Post-Production & Editing
After finishing the video just check the color of the background of a green screen and make any changes required for the best result of mixed reality video.
If we didn’t get a proper front facing view or background then use editing software to rectify the problem.
By using this editing software we can synchronize the audio that exactly matched with the video which is named as an audio-video test.
Here you can find the options of game view and camera view which can be used to visualize the gaming environment.
Alpha matte must be there to inform the editing software to keep the good one.
Mixed Reality Video Footage Calibration & Alignment
It might be very difficult to make the recordings of the camera and recordings of the gameplay.
Along with camera slate, the additional alignment is needed to get the perfect launch of the mixed reality video by synchronizing the footage with a particular frame.
Here we need to consider the steps that are involved in a normal recording of the video i.e. are roll gameplay, camera slate, hide controllers, roll camera and action.
As the controller calibrations are moving fast by adjusting their position then it should merge the both footage of the gameplay and camera even though they have different frame rates.
It should be aligned till we stop the process.
How To Live Stream Mixed Reality Video
Mixed Reality is the coolest way to show what exactly virtual reality environment feels like.Green screen environment makes it easy to remove the unwanted camera feed except your movements.
The body movements and virtual reality movements seamlessly combine two realities into one video.Virtual reality is rapidly growing on live streaming the media content. To stream a virtual reality video we need some tools and equipment needed. The process of streaming is quite complicated.
Require Equipments for Mixed Reality live streaming:
Stage Setup
Green Screen 6ft x 9ft
Green screen can capture the only body movements and removes unwanted camera feed on the environment. The wider your green screen , you can capture wider shorts, essential for the camera.
2 x lights on stands
Two box lights are essential because, when shooting at the green screen maximized it reduce the shadows show more even green, it is easy to remove the green background on footage.
Software Setup
Open Broadcast Software:
You need to download the mixed reality OBS software, it creates the screen into four quarter parts. They are Foreground is on the top left side, Foreground Alpha layer is the top right side , Background is bottom left side and First person’s View is on bottom right corner.
Virtual Controller drivers or 3rd Controller
These are required to assign the locations on both the Physical and Virtual cameras. One of the youtuber has created the mixed reality controller tool named as externalcamera.cfg to collaborating with virtual controller.
Hardware Setup
Sony Handycam
Sony Handycam for capturing the footage on the green screen environment. Used as camera to shoot what a game player playing in the virtual reality world and real time.
HDMI Capture Card
Capture card is installed in your PC it allows a room for the HDMI input/output of card which can opens a PCIe slot on your motherboard.
12ft HDMI Extension Card
The Handycam sends picture 12ft HDMI extension card to HDMI capture card which is mounted on your PC.
10ft USB Extension Cord
The camera gets power through USB extension card, this is plugged into USB charger instead of your PC. Then now from the capture card the picture display on OBS software.
Step by Step Process Of Streaming MIxed Reality Videos
First Install the capture card in your personal computer. You need to remove a piece of back from your PC to make a room for HDMI input/output on PC.
Follow the instructions to successfully install the Capture card and test the signal that receiving from the camera. Your capture card setup is ready, go to green screen.
Then HDMI input is connected to the PC and HDMI output is connected to the Camera.
To capture play area as much as you can through Handycam, there we mentioned two extensions like 12ft HDMI Extension cord and 10ft USB Extension cord.
Setup the Game Play where you have tested and uploaded through HTC Vive . This allows your deaf information to combine with VR graphics, free from the real life camcorder.
Open the settings menu on the game and open it to monitor , select toggle mixed reality Cameras and all zeros in Foreground Color key.
Next to setup Capture OBS software to combine the three layers in the game foreground, the game background and feed from the cam recorder.
After downloading the OBS software, click on Plus sign and select Window Capture.
Select fantastic contraptions can split the camera into four quadrant parts. You need to enter screen resolution in to 960p by 540p with 4K display.
Drag the corner of the layer to expand it complete screen. Next under sources select the Video Capture Device and name it as a camcorder or anything you like. Select the color, resolution, similarities and smoothness of the layers.
Select the foreground layer and set as “Game Capture” and unselect the “Capture any foreground application with Fantastic Contraption. Select dropdown layer and checked in “Apply Transparency” . Select a crop filter and set it to 960 on left and 540 at the bottom.
The OBS allows you to match up the VR gameplay and real screen, the streaming look fantastic.
Ways to Create a Mixed Reality Video
It is not an easy task to create Mixed reality videos. Here we have listed various ways to create a mixed reality video.
Mixed reality Videos With HTC Vive VR Games.
Creating Mixed Reality Videos without Capture Card.
Experimental Filming / Instance CG
Mixed Reality Video Capture Through YouTube Spaces Studio.
Mixed reality with Webcam without using Green Screen.
Creating Mixed Reality Video with Mixed reality App Uvforia 5.
Building Moveria VR Mixed Reality App with Cardboard and Unity.
Conclusion
As Mixed Reality is new and upcoming technology in digital life. Many of them don’t know what exactly the mixed reality means, How does it work, How to create a mixed reality, how to stream and how many ways you can create Mixed reality videos. This is an Ultimate Guide to Mixed reality Video.
