Implementing a virtual cursor in Unity

Hi everyone!

This is a technical post about how I changed the control mechanism of our phone interface in Paralycid; a horror and adventure game. You can find the demo project here.

I assume that you have knowledge in Unity and its UGUI system.

First of all, I will give you some information about our game. It is a fact that flashlights, lanterns, torches etc. are used a lot in horror games. Instead of using a flashlight or a lantern, our female protagonist carries her most valueable thing with her: a phone with various features such as dialing, messaging, navigation, flashlight etc. It's like a real smartphone.

Phone in Paralycid

The phone has a very important place in the game as it is directly connected with the puzzles. Therefore, accessibility and ease of use is one of the most important things for us. For the "easy to use" part, we tried to control the phone with the arrow keys, "backspace" and "enter" keys on the keyboard. The player was using the arrow keys to navigate the menu, "backspace" to return and "enter" to select. However, this approach had a major disadvantage. The player had to switch from mouse to the keyboard whenever he/she wants to use the phone.

For those reasons, I've decided to try a virtual cursor for the phone and you can find the details of the process in this post. We are using Unity's UGUI for the phone interface. All phone UI is inside a canvas which has a Screen Space - Camera render mode. Then we use a render texture to render the whole canvas on a quad inside a 3D phone object. So what we want was, creating a virtual cursor on this quad that can interact with all UI elements like the default cursor. Tl;dr: Here is the phone UI with new virtual cursor:

To do this, let's create an empty scene first. Then create a Canvas with some elements (buttons, for example). Set Canvas's render mode to Screen Space - Camera. Create a new camera, rename it to "CanvasCamera" and assing it as the camera of the Canvas. I've placed the buttons inside a Grid to refer a problem you may encounter:

Simple canvas

After creating the canvas and its camera, create a new render texture asset and assing it as the Target Texture of the CanvasCamera. Then create a quad object and assign render texture to quad. After these step, your scene should look like the image below. You can also use the demo project.

Quad with a render texture on it

At this step, you won't be able to interact with the ui elements on the quad object. Let's find out why. If you click to the EventSystem object, you will see that there are two components attached to this object: EventSystem and StandaloneInputModule. These are two main components/scripts that are responsible for the interaction in a scene. The EventSystem is responsible for processing and handling of events. And StandaloneInputModule is responsible for listening to input events coming from the mouse, keyboard, and touch screen. For information about input modules please read here.

Event System

When we move the default OS cursor and hover the quad, default input module (StandaloneInputModule) thinks that we are literally hovering the quad, it has no idea about the Canvas and its elements. Therefore, we should create a custom cursor in the Canvas and a custom input module which knows the exact positions of the cursor and where the Canvas elements are. Luckily, Unity UI is open source. So, we can modify and use UI scripts easily. But be careful, when you navigate through source code, the default branch is 5.2. Make sure that you are at the same branch as your Unity version.

First step is finding a sprite for the cursor and creating a cursor image in canvas. Then we need to move this image (cursor) with the mouse movements. For that, I've written a small script in VirtualCursor.cs. You can use the script file from the demo project. Adding this script to the cursor image will let you move it with your mouse. Then disable Raycast Target option of the cursor and set anchor to the left top as its edge will interact with the Canvas elements:

Cursor anchor and raycast target property

Then we will create our custom input module and an input manager which will let us switch between standalone and virtual input modules. We will have two gameobjects with EventSystem and an input module components and input manager will manage them by setting only one of them active in scene at a time. You can check them in the demo project.

VirtualInputModule will share most of its code with StandaloneInputModule and it will also extend Unity's PointerInputModule. The difference will be using the position of our cursor image instead of using the position of default cursor while checking the cursor interactions. To do this, we need to download the StandaloneInputModule class at relevant branch from the Unity's bitbucket page (it can be found under this path: UI/UnityEngine.UI/EventSystem/InputModules/), rename it as VirtualInputModule and change all input.mouseposition with the position of our custom cursor. ``PointerInputModule.GetMousePointerEventData() method also uses input.mouseposition so you may want to override this method with your own implementation thet uses the custom cursor position.

After setting the input manager and the custom input module, you can test the scene. When you press "C" key, cursor switches between virtual and the default one. You can find working codes in the demo project. If you notice, highlights will be problematic:

I've set the highlight color to red, and pressed color to blue. If you notice, buttons are rarely blue although I rapidly click to them.

This is the result of locking cursor when enabling the virtual cursor. If you check the PointerInputModule, you can see lines like:

var targetGO = (Cursor.lockState == CursorLockMode.Locked ? null : pointerEvent.pointerCurrentRaycast.gameObject);

To fix this, we need to download PointerInputModule and alter these problematic lines, then use it as base class of the VirtualInputModule. Then the highlights will work properly.

And no, sadly, they will not.

Highlighting and pressing is working fine for the hovered buttons. But as you can notice, other buttons are also highlighted.

For a Canvas with a single button in it, it will work perfectly. But if you are using multiple interactable elements in the Canvas (like I used within a grid), highlighting problems will arise again. This is caused by the Navigation property of the buttons. If you set the Navigation to none for each button, cursor will work properly:

Working virtual cursor in Canvas

You can find the demo project here. See you next time!