Immediate UI vs Retained UI

Sublimination: Immediate UI vs Retained UI 

Graphics APIs can be generally separated into two styles, Immediate and Retained mode. They both have pros and cons about how they work and how they can be used in game development.  The software developer community has been mostly split between these two styles of writing graphics APIs. Retained mode used to be the most popular way of writing graphics APIs for software and was the primary method for game development. In the last couple years, developers have started to use more Immediate mode APIs.  Developers usually prefer one mode over the other and have specific reasons why. Over the course of this project, I have been experimenting with the uses of Immediate mode APIs and Retained mode APIs. I’ve used these styles to implement the UI system for our game to see the pros and cons and figure out why most developers prefer one over the other. Before I could decide which method I preferred, I had to understand what each style does and why it would be good for game development.

What is Immediate UI? 

Immediate mode UI or IMGUI for short, is the new, up and coming graphics API for games. Many developers like this new style of writing UI in their game because it’s easy and straightforward, for the most part. For IMGUI, all the data about a button is passed through a function that draws the button so if the button is pressed, the function will return true. Adding a button to an application would be as easy as adding an “if statement” and writing what the button will do in the “if statement”. If you do not want to render the button, all you need to do is not call the function. This means your system to render UI will be a big if/else statement to decide if the UI should or should not be drawn.

By using IMGUI for an application, the client holds and manages all the data required for the buttons. IMGUI UI would be procedural and be drawn frame by frame. The graphic’s library will not store a model between frames, this means every frame the vertex buffers must be rebuilt every frame. This is one of the drawbacks of IMGUI.  It can put a constant workload on your CPU and may slow the application. Another drawback of IMGUI is that it is not trivial to move widgets around real time in the application. The position and size of the widget is passed into the widget function and if a designer or artist wanted to move or resize the button, it would not be very easy for them. The developer would have to create a backend for the function parameters to have them changed real time.

Overall, IMGUI is a somewhat new and easy way of implementing UI. For a button, all you need to write is an “if statement” with what the button does inside. It’s very easy to implement and manage all the UI aspects. I recommend this style of API to solo developers or a small team of developers because the use of IMGUI is very code-based and not as dynamic as retained API. Some good IMGUI libraries are Dear ImGui and nulkear.

 

What is Retained IU?

 The other graphics API option is Retained mode UI or RMGUI.  This is an old tried and tested object-oriented graphics API for games. Many developers prefer this style because it aligns with all object-oriented engine structures and is easy to implement. With RMGUI, all the data associated with a button is kept with the widget object. When you want to create a button, you first create a widget object and set all its data, bind an on press function, size, position, etc. Then you add it to a list of objects to be drawn with the scene. The widgets are seen as game objects and are put into a list, then looped through, and drawn.

The data for retained mode is primarily held by the graphics library. When the application initializes and loads everything, it loads all the vertex buffers and model data into the graphics library. When the client wants to draw a new frame of the application, it only updates the objects that need to be updated. One drawback to this style of graphics API is that the widgets are involved in many systems of the application. They need to be initialized, manually set in position, added to a draw list, and added to an UI container. This means to delete a RMGUI widget, you will need to remove the widget from all these places instead of just deleting an “if statement” like you would in IMGUI.

In RMGUI it is easier to build a backend editor.  They are just objects so you can edit them just like game objects. This is helpful because all the data is in the same place and it is easier for a developer to create an editor for designers and artists to use.

RMGUI is very useful and easy to understand because most developers use an object-oriented system. For UI, the developers just need to create a new object for different widgets like, buttons, sliders, or checkboxes. I would recommend this for large development teams or developers that want an object-oriented UI system.

 

What Sublimination uses

For our project, we decided to use both IMGUI and RMGUI. IMGUI was used for the editor and RMGUI was used for the game’s UI. 

For the UI of the game, I decided to implement several different types of widgets. Most developed widgets that designers asked for were new UI things, like a health bar or a speed dial. Other than the special requested widgets, I implemented all the basic widgets that help the game function like button widgets, text widgets, and image widgets.  In this screenshot of our game, it shows the heads-up display of the player. Each part of the display is its own individual widget object. Doing the heads-up display as RMGUI was far easier than using IMGUI. It is easier to edit the positions and scale of the widgets. This helped the designers adjust the elements to match their layout of the heads-up display.

For the other menus in the game, I implemented border widgets that hold a list of other widgets. This makes it easy to open and close sub menus. I enable and disable different border widgets to show which menu the player is in. In this screenshot of the main menu, the main menu is a border widget with each of the buttons in that border widget’s list of objects. When the player goes to a submenu, for example the credits menu, it would disable the main menu border widget and enable the credits border widget. 

I decided to use the Dear ImGui library for the editor because it is important for the data shown to be up to date in every frame. This helps the developers see the data of the objects in real-time and they can change them in real-time. In the left panel of the editor shows all the managers. Each manager has a dropdown that shows all its objects in the manager’s container or data that is necessary to see or edit. In the screenshot of the editor, the game object manager is selected and shows all the objects in the current scene. Each object is clickable and then it’s data appears on the right panel. This is where IMGUI shines. All the values are changeable in real-time if the widgets values are not being set on update.  Now any designer or artist can start up the game and change any value of any widget or game object in real-time.

Summary

My team decided to implement both IMGUI and RMGUI in our game to do what each of these methods does best. We used RMGUI to make all our UI in the game, including buttons, images, text, health bars, etc. Then used IMGUI and the Dear ImGui library to create a real time editor, so that other developers, artists, and designers can edit and arrange those RMGUI widgets in real-time. This worked the best for our team and our workflow. Retained mode UI or Immediate mode UI both have pros and cons and could be used on their own or a team could use a hybrid of both, like our team did, in order to utilize the strengths of both methods. Hopefully, you learned about the pros and cons of these two different UI methods and that will help you decide how to best implement them into your projects.

 

Sources:

bkaradzic. “Why I Think Immediate Mode GUI Is Way to Go for GameDev Tools.” Gist, 2017, gist.github.com/bkaradzic/853fd21a15542e0ec96f7268150f1b62.

“Immediate Mode (Computer Graphics).” Wikipedia, Wikimedia Foundation, 18 Nov. 2020, en.wikipedia.org/wiki/Immediate_mode_(computer_graphics).

“Immediate Mode GUI.” Wikipedia, Wikimedia Foundation, 27 Sept. 2020, en.wikipedia.org/wiki/Immediate_mode_GUI.

“r/Opengl - Resources for Learning ‘Retained Mode’ GUI Programming with Graphics?” Reddit, 2017, www.reddit.com/r/opengl/comments/64wbuv/resources_for_learning_retained_mode_gui/.

Radich, Quinn. “Retained Mode Versus Immediate Mode - Win32 Apps.” Win32 Apps | Microsoft Docs, 31 May 2018, docs.microsoft.com/en-us/windows/win32/learnwin32/retained-mode-versus-immediate-mode.

“Retained Mode.” Wikipedia, Wikimedia Foundation, 6 Oct. 2020, en.wikipedia.org/wiki/Retained_mode.