Avatar

Is VR The future? I went to VRLA to find out (Gaming)

by Cody Miller @, Music of the Spheres - Never Forgot, Sunday, August 30, 2015, 05:00 (1922 days ago)
edited by Cody Miller, Sunday, August 30, 2015, 05:39

Today I went and tried out a ton of VR stuff at the VR convention here in LA, and played a bunch of VR games and watched a ton of VR movies. VR is absolutely the future in some areas, but a complete dead end in others.

Let's talk about FPS games first. Right now, and possibly forever these are a dead end with VR. The big problem is control. I played a game called World War Toons, in which you aim by moving your head. So, your head is acting like your right thumbstick. Imagine you are playing Halo or something, and you move the right stick all the way to the left. You start to turn left, and if you hold it there, you will keep turning. Return the stick to neutral, and you stop turning. But now imagine the stick is positional, so if you push it all the way to the left, you look 90 degrees left and stop. Push the stick halfway, you look 45 degrees, and when the stick returns to neutral you look forward again. This is how the game works, and you can probably see the problem. If I turn my head and look left, and I want to look left of that, I can't because my neck can't turn anymore if I'm sitting in a chair. You can't just use a swivel chair, because the head tracking works very poorly if you are facing the other way. Even if that somehow worked, you still can;t spin because you have the video cable going to your headset which will wrap around you and get tangled. You basically can never turn around, and to do so you have to just run in a big ass circle. It sucks.

After playing that I thought the solution would have been to have the game control like Halo or Destiny, with the left thumbstick moving, the right aiming, and the head tracking letting you look around at stuff you aren;t shooting. There were a few games like that, but these have an even more serious problem. Every time I used the right stick to turn, I immediately felt sick. The effect was instant. Basically any time my viewpoint was changing, and my head is stationary, I felt nauseous.

So with FPS games, you either have a game that's playable that you can't control, or a game that's controllable that you can't play. There was another game with a setup where there were 2 dozen cameras in a square, and you could simply turn your body and it tracked your position, as well as the position of your fake gun. This fixed both the problems above, but it still problematic because you can only walk about 10 feet, and nobody will have this setup in their home. So while the control was good, it's sucks to be boxed into the game world of 100 square feet.

Some things rock in VR. Racing games are phenomenal. I played a ton, and when Sony gets a Gran Turismo game running on their VR headset, it's going to be the killer app. It is just straight up better than playing it on a screen. I played a sim type game similar to Forza and GT, a Mario Kart clone, and a really hilarious game called SMS racer. The idea was that you were racing, but you'd get texts from your friends during the race, which you had to answer within 10 seconds or you lose. So you have to look down, text, all the while not hitting shit or spinning off the track. An amazingly awesome concept, and a blast, if not a bit gimicky.

Adventure games or other non twitch games are going to rock in VR as well. Adrift was a game that was essentially Gravity, where your space station had an accident and you have to rescue your crew and get out. Amazingly immersive and pretty awesome. Story driven games or games like Gone Home would work well in VR as well.

Surprisingly third person games worked okay as well. RTS works in VR, but doesn't really get the benefit the above genres get. Playing it on a 3D TV would be just as good.

I also checked out 360 degree video, where you basically take a ball of GoPros and shoot video all around. You can then play it back, and use the VR headset to look around. For storytelling it's pointless in my opinion since you can;t move around, and can only look from the camera's perspective. So you're essentially watching a movie with no edits, with important stuff you can miss because you aren't looking in the right direction. There's no real benefit. It has technical and commercial applications I think, as one of the demos had me looking around video of an apartment, which is a great way for realtors to show it off without you having to make the trip. Watching a concert is also okay I guess. But storytelling in VR is going to be dominated by video games, not live action film.

As for the tech itself, I felt it was pretty good except for two things. First, none of the units had a high enough resolution. The screen door effect is real, and it's a significant detraction from the experience. Each eye was 1080p (~2K), so realistically you're looking at 4K per eye to eliminate this. I don't know many systems that can render two 4K images at 60fps right now. Second, and this is minor, but none of the units completely filled my field of vision. I always had the black from the edges in my periphery, so the field of view was too narrow, and in a way felt like tunnel vision.

Occulus and Samsung were both great headsets. Google Cardboard sucked ass. There's no strap to hold it on your head (you need to use your hands), and the tracking is bad.

The other thing that needs addressing is positional audio. No games had it. I would turn my head, and the audio still came from the same place in the headphones.

TLDR;

FPS sucks in VR. Racing games rock in VR. VR live action video is not good for storytelling, but VR video games are.

thanks Cody

by Raflection, Sunday, August 30, 2015, 05:53 (1922 days ago) @ Cody Miller

Some fantastic field work here!

I agree that adventuring games with VR would be awesome.
Playing something like skyrim on VR combined with the Xbox cameras (seriously can't remember what this is called FML) would be awesome.

I can see the blacks in corners becoming very hazardous when playing on VR as you've effectively got 2 completely blind spots which other players would take advantage of.

It would be very frustrating being killed by a side swipe solely down to the fact you couldn't see them.

Avatar

Re: looking separate from where you're aiming

by kidtsunami @, Atlanta, GA, Sunday, August 30, 2015, 13:13 (1921 days ago) @ Cody Miller

After playing that I thought the solution would have been to have the game control like Halo or Destiny, with the left thumbstick moving, the right aiming, and the head tracking letting you look around at stuff you aren;t shooting. There were a few games like that, but these have an even more serious problem. Every time I used the right stick to turn, I immediately felt sick. The effect was instant. Basically any time my viewpoint was changing, and my head is stationary, I felt nauseous.

I've got a DK2 kit and also had an opportunity to play TF2 on a DK back in the day and found that looking separately from aiming worked just fine for me. It was actually awesome. I was able to put suppressing fire down a hallway while keeping an eye on what was happening in my periphery.

Whenever the problem is "doing X makes you sick" it's not a matter of not doing it in VR, it's a matter of resolving the issue causing that. In a lot of cases we've found that you to have tween/interpolate camera changes in a very fine tuned way to prevent that disconnect between your inner ear and what you're perceiving from causing you to blow chunks. Hopefully they can fix it for you.

Anyways, awesome write up, really appreciate it. I'm looking forward to more and more people getting their hands on VR.

Avatar

Is VR The future? I went to VRLA to find out

by red robber @, Crawfish Country, Sunday, August 30, 2015, 13:15 (1921 days ago) @ Cody Miller

Seems like it could be possible to develope a system for FPS down the road, just might have to re-engineer how we control/play. I think it would be hard to say it's a total dead end, we just need someone to imagine an alternate path to the destination.

I am surprised that operating an FPS with a standard controller causes issues but I have no experience with VR. I wonder if the "look" control (right thumbstick) was tied to the position of a weapon we can hold. This seems like a natural solution and a happy medium. You could track the gun and even make the gun a controller by putting buttons on the grip and forearm if gestures were either too complicated or unwieldy. The biggest issue I see with this is simply holding a weapon for extended period of time. Just holding a single pound for 15 minutes would be exhausting, but I suppose you'd be able to recognize gamers easily.

[image]

Avatar

IIRC, TF2 has quite a few different VR control schemes

by ZackDark @, Not behind you. NO! Don't look., Sunday, August 30, 2015, 16:55 (1921 days ago) @ red robber

Pretty much all the possibilities I can think of, actually. Maybe explore those and advance from there?

Avatar

I've been wondering if Highwire Games might dive into VR?

by Pyromancy @, Sunday, August 30, 2015, 13:16 (1921 days ago) @ Cody Miller

- No text -

Avatar

RC's uninformed opinions on VR

by RC ⌂, UK, Sunday, August 30, 2015, 23:10 (1921 days ago) @ Cody Miller
edited by RC, Sunday, August 30, 2015, 23:22

The kit required to make many excellent VR experiences will be too expensive and/or bulky for the vast majority of home users. Arcade-like installations will be able to achieve experiences that simply won't come to home users on a large scale because it would end up sucking with the price, weight and space concerns.

Note: I have not yet used any VR device and this is all based on what I've read, as well as my own imagination.

Let's talk about FPS games first. Right now, and possibly forever these are a dead end with VR. The big problem is control. I played a game called World War Toons, in which you aim by moving your head.

Yep, totally dumb idea. VR FPS games should (and I believe will) model themselves after a mech-style control scheme. More inertia and slower turning than current gamepad FPSes. One joystick for X/Y motion and turning. Second for fine-grain aim. Use your head for 'general direction of aim' and the joystick to finesse it. More intuitive that way, I think.

VR experiences may integrate more foot-pedals as well - which would be a natural bit of kit for car racing games, and could offer additional analogue input for other genres.

You can't just use a swivel chair, because the head tracking works very poorly if you are facing the other way.

HTC Vive has cameras embedded in the device. So should work equally well whatever direction you are facing. Or a set-up with cameras all-around as you mention later (as you said, not feasible in the short-medium term for home use).

even if that somehow worked, you still can;t spin because you have the video cable going to your headset which will wrap around you and get tangled.

There are such things as power and data connectors that work when they're spun in every direction. Ring rails? Not feasible for home, but wireless may be more of a thing with Gigabit-level WiFi (WiGig) and less data-heavy, low-latency video connections (next-gen, not currently existing).

After playing that I thought the solution would have been to have the game control like Halo or Destiny, with the left thumbstick moving, the right aiming, and the head tracking letting you look around at stuff you aren;t shooting. There were a few games like that, but these have an even more serious problem. Every time I used the right stick to turn, I immediately felt sick. The effect was instant. Basically any time my viewpoint was changing, and my head is stationary, I felt nauseous.

Apparently people have varying susceptibility to 'simulator sickness' (the corollary of car sickness). Moving the person's actual body helps, and as another poster mentioned, very high refresh rates where the image does not remain static compared to the expected motion blur/retinal smear apparently helps as well.

This fixed both the problems above, but it still problematic because you can only walk about 10 feet, and nobody will have this setup in their home. So while the control was good, it's sucks to be boxed into the game world of 100 square feet.

The whole game (which will be a more limited, focused and 'arcadey' experience) has to be designed with that limitation in mind. I've seen one demo where the players are on a hover-craft that flies through a desert canyon with various bad-guys popping up on their own craft. I think it'd turn out to be a really fun arcade experience (especially co-op) but, as you said, not something people would have in their own homes.

I also checked out 360 degree video, where you basically take a ball of GoPros and shoot video all around. You can then play it back, and use the VR headset to look around. For storytelling it's pointless in my opinion since you can;t move around, and can only look from the camera's perspective. So you're essentially watching a movie with no edits, with important stuff you can miss because you aren't looking in the right direction.

How do they direct audience attention in plays/theatre? Lighting, motion, sound. How do they edit? Scene changes.

VR video (cinema?) doesn't have to be 360 Degrees of Non-Stop Action, it can be any size, any shape, any orientation of focus, at any time, at any point around the viewer. In front, behind, above, below. Squares, circles, ovals, fades. The creator can have multiple areas of focus from disparate places at the same time. It's not limited to just a 16:9 rectangle, covering 50 degrees directly in front of the audience - it's anything up to a full sphere of a scene. Some movie-makers will never figure this out, but others will and create some awesome stuff, IMO.

Computer-generated Machinima, or possibly light-field data may allow viewers to be more 'within' the scene and at least lean around a scene, or move in a limited fashion. More experiences that blend movie and game with varying levels of interactivity are coming, I think.

Editing techniques will differ, but will still exist. If you have a scene where two characters are talking, it'd be more natural to simply place them both within the field of view and let the audience look from one to the other rather than cutting. If the audience misses clues, that's a part of their unique experience and already happens in cinema anyway. Or they can be led to noticing things with lighting, motion, sound & colour cues.

There's no real benefit.

It's different. I don't think it'll ever a be a replacement, however.

First, none of the units had a high enough resolution. The screen door effect is real, and it's a significant detraction from the experience. Each eye was 1080p (~2K), so realistically you're looking at 4K per eye to eliminate this. I don't know many systems that can render two 4K images at 60fps right now.

I think VR is where raster rendering may meet it's demise. However, alternatives are a few years out still, unfortunately. Look up 'frameless' and 'foveated' rendering. There is significant inter-ocular coherency (similarity between images presented to each eye), spatial coherency (similarity of pixels close to each other), and temporal coherency (between frames). Ray-traced Rendering could exploit these features just as video compression already does to reduce the rendering burden by orders of magnitude. But it needs graphics and display hardware support to get the full benefit. "Frames per second" will mean less than response time, and "resolution" will be more like a hardware maximum.

The first generation of games that use this will probably have a 'graphics' downgrade, but will be silky smooth and razor sharp. They'll probably have simpler art styles that focus more on shape and shadow than texture.

Second, and this is minor, but none of the units completely filled my field of vision. I always had the black from the edges in my periphery, so the field of view was too narrow, and in a way felt like tunnel vision.

Just down to optics of a flat screen that close to your face. Curved screens could do better horizontally. Vertically is much harder due to the geometry of faces.

The other thing that needs addressing is positional audio. No games had it. I would turn my head, and the audio still came from the same place in the headphones.

Audio is generally neglected :'( Which is a shame because binaural sound could add sooo much to VR games.


Anyway, I really want to actually try one of these devices some time. Really just jelly. Mega jelly.

Is VR The future? I went to VRLA to find out

by yakaman, Monday, August 31, 2015, 15:34 (1920 days ago) @ Cody Miller

Great read. Thank you.

As for the tech itself, I felt it was pretty good except for two things. First, none of the units had a high enough resolution. The screen door effect is real, and it's a significant detraction from the experience. Each eye was 1080p (~2K), so realistically you're looking at 4K per eye to eliminate this. I don't know many systems that can render two 4K images at 60fps right now. Second, and this is minor, but none of the units completely filled my field of vision. I always had the black from the edges in my periphery, so the field of view was too narrow, and in a way felt like tunnel vision.

Just give industry a few generations and a little commercial success, and maybe we'll see explosive progression here like we did for smart phones.

Is VR The future? I went to VRLA to find out

by marmot 1333 @, Monday, August 31, 2015, 16:04 (1920 days ago) @ yakaman

Just give industry a few generations and a little commercial success, and maybe we'll see explosive progression here like we did for smart phones.

To some extent... I doubt the sorority girl demographic will pick up on VR the way they did smart phones & facebook, though.

Avatar

funny

by kidtsunami @, Atlanta, GA, Monday, August 31, 2015, 16:32 (1920 days ago) @ marmot 1333

When I showed the DK2 to my family and in-laws, it was my mother-in-law (a sorority girl), who had the most excited reaction to it.

Avatar

Is VR The future? Eyes & Resolution.

by dogcow @, Hiding from Bob, in the vent core., Monday, August 31, 2015, 17:06 (1920 days ago) @ Cody Miller

As for the tech itself, I felt it was pretty good except for two things. First, none of the units had a high enough resolution. The screen door effect is real, and it's a significant detraction from the experience. Each eye was 1080p (~2K), so realistically you're looking at 4K per eye to eliminate this. I don't know many systems that can render two 4K images at 60fps right now. Second, and this is minor, but none of the units completely filled my field of vision. I always had the black from the edges in my periphery, so the field of view was too narrow, and in a way felt like tunnel vision.


The eye doesn't see in full resolution across its whole field of vision, only what you're looking directly at, anything in the periphery is in pretty low resolution. The computer scientist in me (who wants to optimize everything) wonders if VR makers could take advantage of that somehow. The initial problem I see is that the eye moves VERY quickly, and I doubt hardware could track & increase the resolution fast enough that you wouldn't experience "resolution pop-in". The other way about this is to train the person to move their head instead of their eyes, but doing so could detract from the immersion. We just need a matrix style jack in the back of our heads.

Avatar

Is VR The future? Eyes & Resolution.

by Cody Miller @, Music of the Spheres - Never Forgot, Monday, August 31, 2015, 17:16 (1920 days ago) @ dogcow

We just need a matrix style jack in the back of our heads.

Yeah, that's the real future of VR. As I said, it currently works great when your own body mimics what your virtual body would be doing. You're sitting down, hence racing game work so well. Adrift worked well too, since in your space suit, your body is stationary, and you move it using the MMU. When I exited the station and looked down at Earth for the first time… it was one of the most awesome things ever.

Avatar

Is VR The future? Eyes & Resolution.

by ZackDark @, Not behind you. NO! Don't look., Monday, August 31, 2015, 18:50 (1920 days ago) @ dogcow

We just need a matrix style jack in the back of our heads.

We're closer than the general public feels comfortable with, but still a few years away.

One day, dog, one day...

Is VR The future? Eyes & Resolution.

by electricpirate @, Monday, August 31, 2015, 23:26 (1920 days ago) @ dogcow


The eye doesn't see in full resolution across its whole field of vision, only what you're looking directly at, anything in the periphery is in pretty low resolution. The computer scientist in me (who wants to optimize everything) wonders if VR makers could take advantage of that somehow. The initial problem I see is that the eye moves VERY quickly, and I doubt hardware could track & increase the resolution fast enough that you wouldn't experience "resolution pop-in".

It's called Foveated rendering, and it's being worked on :).

https://en.wikipedia.org/wiki/Foveated_imaging


Here's an example in Unity that apparently improves performance notably.
https://www.youtube.com/watch?v=GKR8tM28NnQ

Avatar

Is VR The future? Eyes & Resolution.

by dogcow @, Hiding from Bob, in the vent core., Tuesday, September 01, 2015, 12:45 (1919 days ago) @ electricpirate


The eye doesn't see in full resolution across its whole field of vision, only what you're looking directly at, anything in the periphery is in pretty low resolution. The computer scientist in me (who wants to optimize everything) wonders if VR makers could take advantage of that somehow. The initial problem I see is that the eye moves VERY quickly, and I doubt hardware could track & increase the resolution fast enough that you wouldn't experience "resolution pop-in".


It's called Foveated rendering, and it's being worked on :).

https://en.wikipedia.org/wiki/Foveated_imaging


Here's an example in Unity that apparently improves performance notably.
https://www.youtube.com/watch?v=GKR8tM28NnQ

Cool. Foveated Rendering, now i'll know what term to follow this tech. Sounds like I was right regarding the eye-tracking. I would like to see an example where the higher resolution area is kept in the center of the video so I can just focus on it, or maybe have it move more slowly around the screen with an object in the center so I can track it, just to give you a better feel for how it would be if they were able to track your eye fast enough.

I'm glad to see my suspicions about tracking they eye being a major hurdle...

A quote from the guy who posted that video:

To become unnoticeable the eye tracking needs to be 300hz and the screen needs to be 120hz (check out the Microsoft Research paper from 2013). You get 5-6 times performance boost.
Thankfully there are other applications of foveated rendering where even a noticeable popping is fine compared to the alterative. Think real time raytracing.

As a young teen I learned how to raytrace & would wait DAYS for my poor LCII to render a simple scene. This was before I got a math coprocesor daughterboard which cut it down to somewhere between 12 & 24 hours. Many years later when Half-Life came out I was curious about using raytracing for Barney's shiny helmet and I wondered when they would have enough performance to get real-time raytracing into video games, even if it was just for reflections.

Avatar

Is VR The future? Eyes & Resolution.

by Cody Miller @, Music of the Spheres - Never Forgot, Tuesday, September 01, 2015, 15:31 (1919 days ago) @ dogcow

As a young teen I learned how to raytrace & would wait DAYS for my poor LCII to render a simple scene. This was before I got a math coprocesor daughterboard which cut it down to somewhere between 12 & 24 hours. Many years later when Half-Life came out I was curious about using raytracing for Barney's shiny helmet and I wondered when they would have enough performance to get real-time raytracing into video games, even if it was just for reflections.

Halo 2 WAS supposed to have real time reflections in the Master Chief's visor :-p

Back to the forum index
RSS Feed of thread