The developers behind open-source Oculus Rift drivers take issue with Nate Mitchell's claims.
By Tim Colwill on May 17, 2013 at 5:39 pm
Editor’s Note: After we ran our massive interview with Nate Mitchell, VP of Product behind the Oculus Rift VR headset (read part one here, and part two here) we were contacted by Neil Schneider from Meant To Be Seen 3D, the core developers behind the open source Vireio Perception drivers that many gamers have been using to turn games into VR-capable 3D experiences.
He asked us if he could issue a statement to clear the air around the issue of Vireio drivers vs. native Rift support, and we agreed. So without further ado, take it away Neil…
Let me begin by saying that I’m supportive of the Oculus VR SDK and Nate is 100% correct that a native VR game has competitive advantages over anything a VR or stereoscopic 3D driver can deliver. Oculus’ strategy is the correct one, and Vireio Perception’s role is to fill the need for games that don’t or won’t have native VR support. It’s not our intention to replace or substitute the benefits of true VR rendering and programming.
Nate’s remarks as they are written are misleading, however, and I would like the opportunity to answer his criticisms.
Andres Hernandez (Cybereality). Toy Gun!
First, Cybereality, whose real name is Andres Hernandez, is an MTBS forum moderator. He’s helped out with coding parts of the website and has contributed a lot of time to the community over several years. He still does. The same goes for Palmer Luckey, who also has roots as an MTBS moderator for years, and still maintains the role.
Long before the Rift was launched, Andres announced plans to release commercial stereoscopic 3D drivers for VR devices like the Oculus Rift. What made these drivers unique compared to the other products is that they supported head tracking which is a core feature of the Rift and is available through other means by off the shelf products. This isn’t a simple “mouse look” as Nate’s comments suggest. The game rolls and turns with your head movements; it does this by using a combination of mouse look and adjusting the projection matrix so that you get all three axes of rotation.
Head roll, deep 3D, table corner is protruding with Vireio Perception
Realizing that he was getting hired by Oculus VR, Andres decided to release his VR drivers as open source so he could put most of his energies into Oculus VR. Since that time, Vireio Perception went from one solitary programmer to three talented volunteer coders who are collaboratively working to build the next iteration of the software.
According to the article, Nate’s primary concern about the Vireio Perception drivers is latency, and that this latency is somehow disorienting for gamers. While I will be the first to tell you that Vireio Perception has a major problem (I mean MAJOR), it has little to do with latency.
Stereoscopic 3D, a core feature of VR and the Oculus Rift, requires that the left and right view appears on the screen at the same time. If the software is working correctly, you should see the left and right view, next frame, left and right view, next frame, etc. etc. Each frame has to show both views simultaneously.
Vireio Perception drivers are a software hack to get the same results from games that weren’t designed to do this. On the fly, the drivers take existing video game data from DirectX API calls, create a second camera view where none existed before, and pass on a left and right view to the display – in this case the Oculus Rift.
When Andres was working on the Vireio Perception code he got nearly everything correctly, but for whatever reason, the left and right views were out of sync. So even though things are happening faster than our eyes can easily perceive, the left and right views are not being updated at the same time in each frame. That was a piece of the puzzle he just didn’t have enough time to solve, and that’s how the drivers have been behaving right up to the current build.
This is one of those errors where you know something is wrong, but you can’t quite put your finger on it. Nate is calling it latency, but it’s really something very different. There is a happy ending to this story, and I will get to that shortly.
The biggest problem with Vireio Perception is the user interface (UI). The Oculus Rift is designed in such a way that the screen’s field of view goes beyond what your eyes can actually see. The majority of games place the HUD elements in the far corners of the screen which means that even though the UI is there, you can’t see it! This is a real problem when you are dependent on in-game menus or health and ammo indicators. There is also the issue of the game’s crosshair not being accurate in 3D mode because it wasn’t designed for that purpose.
Now the good news!
Convergence Portion of SHOCT Mechanism
First, the time differential problem between the left and right camera views is largely corrected. As long as users follow our settings guide and use the SHOCT tool (Schneider-Hicks Optical Calibration Tool), we think the next public build is going to be a lot more comfortable for extended gaming periods.
While Vireio Perception started as just supporting DirectX 9, we have been hard at work adding DirectX 10 and 11 support. We also think the crosshair problem will be easily solved with an add-on 3D crosshair or something similar, and we are experimenting with different methods to get existing game HUDs more VR friendly. This problem hasn’t been licked yet, but we have ideas.
I know this sounds crazy, but not one of our programmers have received their Oculus Rift yet! Imagine the improvements that will be made once they have the actual SDK in their hands!
So this begs the question: does native VR support have a real advantage over VR drivers like Vireio Perception?
Yes, and this is why. Vireio Perception makes it possible to track nearly all your head’s movements. It’s kind of like a mouse look, but it’s much more advanced. However, unless the game is designed to behave differently, your body and head movement are one and the same.
Native VR games let you look around and walk as two distinct movements just like real life. Depending on the game, there are additional hacks and modifications that can be found to replicate this behavior, but it’s not a simple driver fix. It’s also likely that there is a lower latency with native code as Nate describes, but this hasn’t been reported as an issue by excited Vireio Perception gamers with mid to high range PC gaming equipment.
I will finish by saying that I’m very proud of the Vireio Perception development team for their continued technical innovation and their positive problem solving attitude. When he got wind of Nate’s remarks, Philip Larkson’s first comment was “all we can do is keep improving it (Vireio Perception)”. With a Skype smiley, Chris Drain said “Doesn’t mean I don’t want to replay old stuff that’s never going to get first party support though!” As for John Hicks, he didn’t have to say anything. He’s just setting an example and is hard at work on the next wave of Vireio Perception.
I should point out that there are other products out there. There is a lot of excitement for Vireio Perception because it’s free and open source, but there is also a commercial package called VorpX in development, and with the help of NTHUSIM HMD, I think that established brands like Dynamic Digital Depth’s TriDef Ignition and Nvidia’s 3D Vision drivers will be able to repurpose their established 3D software for VR. At least, it’s something I’m hoping for. I don’t know if the NTHUSIM / 3D driver mix will have head tracking, so I’m looking forward to seeing what they cook up.
Palmer Luckey & Neil Schneider at CES 2013
Let me finish by saying that whether people love or hate VR drivers doesn’t matter. What matters is that VR is being taken very seriously by game developers and gamers alike for products in the not-too-distant future. I expect that there will be plenty of AAA titles and independent game releases with innovative VR support someday soon. In the meantime, MTBS’ community is just happy to help fill the large gap of games leading up to that point. After all, helping foster community driven ideas is what we do best.