Virtual Reality development with Unreal Engine 4 4

This post serves as a guide for anyone looking to get into Virtual Reality (VR) development with Unreal Engine 4 (UE4). The topics I am covering are:

• Setting up UE4 for VR
• Best Practices for VR
• Blueprint
• C++
• Blueprint and C++
• Performance considerations

Before starting, I am using UE4.13.2 for the examples throughout this post. When you see this, the layout may not be the same as my examples.

Setting up UE4 for Virtual Reality

The first thing you want to do is to enable the VR plugins of UE4. To do this, go to Edit > Plugins > VR and enable the support for the head-mounted displays (HMDs) you will be developing for.

The next thing you want to do is enable instanced stereo for a nice performance boost in VR rendering. To do this go to Edit > Project Settings > Engine > Rendering > VR. This setting allows rendering of each frame for both the left and right eye at the same time.

Relaunch the engine once you enable both of the settings. Now in the editor, for the dropdown option of Play, there will be a VR Preview mode. Notice VR Preview is not available unless you have an HMD attached to your host device.

Unavailable VR preview

Best Practices for Virtual Reality

Note: Since VR is still new, best practices is always changing. By the time you read this post, some information may be outdated.

Teleportation for movement

One of the best ways for players to move now in Virtual Reality is by teleportation. This is because teleportation reduces the likelihood of player experiencing motion sickness. In Unreal Engine 4.13.2 there is a Blueprint VR template that has teleportation as a movement. I highly recommend you go check it out to see the possibility of teleportation for movement.

To understand why teleportation can be a good way to move around in VR you should try out a VR game call “World War Toons“. It is available for the PlayStation VR and offers movement with the controller’s thumbstick. Even though the speed was very slow (recommended for VR), I was still experiencing a bit of motion sickness. You can also do 90-degree turns without moving your head. Doing that in the game I found more comfortable than moving around with the thumbstick.

World Scale

Something that is vital to your VR project is the world scale. World scale is important for delivering the best user experience. The wrong scale can lead to all kind of sensory issues for the users and even result in simulation sickness.

Objects are best viewed in VR when they are in a range of 0.75 to 3.5 Meters from the player’s camera. Unreal Engine 4 has their own scale so you need to keep the distance valid with unreal unit. Here is how UE4 distance unit works: 1 unreal unit (UU) is equal to 1 centimeter (CM). Therefore, objects are best-viewed 75 UU and 350 UU away from the player’s camera. For reference:

• 1 CM = 1 UU
• 1 Meter = 100 UU
• 1 Kilometer = 100,000 UU

You can adjust the scale of your world by double clicking your map in the World Outliner and then modify the “World to Meters” value in the World Settings.

World scale setting for VR

For more best practices in VR, you can visit the Unreal Engine documentations.

Blueprint

Using Unreal Engine 4’s blueprint system for virtual reality development is simple. There are many predefined nodes that do the heavy lifting for you. All you have to do is link them together for what you need to do. Here is a non-exhaustive list of what UE4.13.2 has for HMD support:

Some HMD nodes in Unreal Engine 4.13.2

Requirements for VR support

The bare minimum for what you need for VR is a Camera Component. By default, your Camera Component is already set up for HMD support. Optionally, you can set up one or two Motion Controllers Components. For information on the Motion Controllers setup, you can visit the official documentations on how to Setup Motion Controllers.

VR template Blueprint Project

With Unreal Engine 4.13.2 and above, there is a VR blueprint template you can start your project with or reference. There is no C++ version of the template as of this post.

The VR template out of the box supports the Oculus Rift, HTC Vive, and PlayStation VR. There are two maps included in the template. One shows usage of teleport with the HMD and a gamepad controller. The other map shows usage of teleport with the HMD and motion controller. For more details on what the VR template has to offer, you can visit Tom Looman’s (one of the developers behind the template) post about the blueprint project.

When to use blueprint

Blueprint is great for rapid prototyping. Much of the functions you may need is already a blueprint node, which saves you a ton of time. Those blueprint nodes are also well tested and are likely to have no bugs. If performance is a requirement, it is better to move into a BP and C++ model for your project.

C++

For those of you that wants to use C++ in your VR project, you can start with Tom Looman’s VRFirstPerson template. In this template, the VR Character is defined in C++ with a Camera Component and two Motion Controller Components. From there you can choose to add extra functionalities in C++ or derive a Blueprint child character from the VR Character class.

To use the VR functions that UE4 has to offer in C++, you must include the “HeadMountedDisplay” Module in your ProjectName.Build.cs file. Example from the VRFirstPerson template:

VRFirstPerson.Build.cs

// Fill out your copyright notice in the Description page of Project Settings.

using UnrealBuildTool;

public class VRFirstPerson : ModuleRules
{ 
    public VRFirstPerson(TargetInfo Target) 
    {
        PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "InputCore" });

    /* VR Required Modules */
        PrivateDependencyModuleNames.AddRange(new string[] { "HeadMountedDisplay" });

        // Uncomment if you are using Slate UI
        // PrivateDependencyModuleNames.AddRange(new string[] { "Slate", "SlateCore" });

        // Uncomment if you are using online features
        // PrivateDependencyModuleNames.Add("OnlineSubsystem");
        // if ((Target.Platform == UnrealTargetPlatform.Win32) || (Target.Platform == UnrealTargetPlatform.Win64)) 
        // {
        //     if (UEBuildConfiguration.bCompileSteamOSS == true)
        //     {
        //         DynamicallyLoadedModuleNames.Add("OnlineSubsystemSteam");
        //     }
        // }
     }
}

To use the HMD and or motion controller components in your VR character class make sure to include the following header files:

• #include “HeadMountedDisplay.h”
• #include “MotionControllerComponent.h”

When to use C++

You should only go with a pure C++ model for your project if you are looking for the absolute highest amount of performance. Keep in mind developing with C++ will take substantially longer than with Blueprints. In addition, you probably will gain more performance from just optimizing your scene complexity than to convert your project into C++.

Blueprint and C++

To start a project that will contain Blueprint and C++, it is best to start as a C++ project. You can then write the base functions or classes in C++ and then extend the C++ class by deriving Blueprint child class from it.

When to use Blueprint and C++

This form of cross communication between C++ and Blueprint works best when you are working on a project as a team. For example, the programmer can write the base C++ classes or functions. Then pass them off to their teammates to continue building the classes or functions by Blueprint. This way of doing things, allow non-programmers to contribute to the project.

Advantages

Another reason you may want to use Blueprint and C++ in the same project is that sometimes there are no blueprint nodes available to access some lower level functions. For example, when I was working with Unreal Engine 4.11, there was no blueprint node to find out which HMD was being used. I had to write my own C++ function as a Blueprint callable node by using the low-level HMD API.

The main reason for using the Blueprint and C++ model is for efficiency. Writing the C++ equivalent of Blueprints can take many times longer than the Blueprint implementation took. However, when you implement the major parts in C++ and then handle the rest in Blueprint it can speed up the development time. Every Blueprint function call also takes longer than a C++ function, so with this model, you can balance between Blueprint and C++ functions.

Performance considerations

One of the most challenging aspects of VR development is meeting the spec requirements with our current level of hardware. When you are developing for the HTC Vive or Oculus Rift, your VR project must run smoothly at 90 fps (frames per second). You can have your VR project complete, but not meet the FPS requirement. So what can you do to address the problem?

The best thing to do would be to have performance in mind as you are developing your VR project. The next best thing you can do is to monitor your project’s performance and make changes that can boost performance without taking much from the VR experience. If you are going with this route, you should consider the development time for those modifications.

CPU profiling

Your VR project may be CPU or GPU bound. It is hard to tell which with the complexity of game engines nowadays. A fast way to start checking for the performance of your project is with the “stat unit” command. This command shows basic information such as the time it takes between frames, game level, drawing/rendering, and GPU processing time. To use this command you can either enter it in the console of the editor or at the beginning of your blueprint.

Invoking the stat unit command with Blueprint

Output of the stat unit command

For more in-depth CPU profiling techniques, you can visit Bob Tellez blog post about CPU Profiling with UE4.

GPU Profiling

In order to capture a single frame with GPU timings, you can use the command “profilegpu”. This command dumps accurate timings of the GPU. You can use the command in the editor console or in Blueprint.

Using profilegpu in Blueprint

Output from profilegpu command

A good place to learn about profiling your project is with the GPU profiling and Performance and Profiling documentations. Here is a great list of tips from developers part of FATED on optimizing VR experience.

Disable Heavy Post-Processors

There are certain expensive post-processes in Virtual Reality such as Ambient Occlusion. Others are an annoyance in Virtual Reality and even disrupt your VR experience. Such an example is Lens Flares. To disable post-processing in your project, go to Edit > Project > Settings > Rendering.

Reducing Scene Complexity

With current hardware, it is really difficult to stay at 90 fps. You may need to revisit your scene complexity such as dynamic shadows, polycount of meshes, and atmospheric particle effects. Lots of translucent surfaces and or particle effects can really hammer down your fps. To view the current shader complexity or overdraw press Alt+8 in your viewport. Here is what the colors mean:

• Green = good
• Red = bad
• White hot = extremely bad

Dynamic shadows and lights also have a huge impact on performance. Whenever possible, you should bake as much lighting as you can. Doing this will help keep the per-frame cost as low as possible.

Here is an excellent talk by Nick Donaldson and Nick Whiting about integrating Oculus Rift support into Unreal Engine 4. The talk has loads of great info about the basics of VR in general and ways to tackle GPU optimization for VR development.

I hope this was helpful to you and assist you in your VR development. If you found this post helpful, share it with others who may benefit from it.

If you have a question or feel that I missed something important, feel free to leave a comment. To stay in touch, follow me on Twitter. Happy developing!