Serialization/ Deserialization using Run time type information

Overview

Last time I went over the Entity Component System pattern and how it can be utilized to create a functional core game engine. I now want to expand on it and share a technique known as serialization.

So serialization may be something you've heard of before but never actually knew what it meant or what it entails. Serialization is simply a way that data can be translated to a storable format in order to be restored later (deserialization).
In terms of our game engine this can be utilized for a level editor in order to store game objects in a level so that the next time you open the game, the level is just as you saved it.
​
When serializing game objects you most likely want to store the objects components and their properties, but the problem is a game object has an array of base component pointers and we dont know what those components are or how to serialize them. A way to solve this is by creating a function for each component specifying how to serialize and deserialize that object but there is a simpler/lower maintenance approach. Reflection!

Run time reflection is the ability to examine and modify an object at runtime and C++ does not support this by default, so I will be using a 3rd party library specified later that has implemented it for us already.  Reflection allows us now to go through the game objects components to serialize their properties meaning we don't need to create a serialize/deserialze function for every component, instead we can just register our properties and let reflection do the work for us later on.

Libraries

​JSON is a common file format used for serialization and for this guide I will be using the JsonCpp library to easily create and parse json files that will store my serialized data.

​ RTTR (Run Time Type Reflection) library will be used for reflection as it has already implemented an "easy and intuitive way to use reflection in C++."

This guide will not serve as documentation to these libraries and the objective is not to teach how to use these libraries, but rather focus on the implementation and technique to easily serialize and deserialize game objects for various uses, one example bieng a level editor. If you choose to also use these libraries for your implementation the documentation can be found at these links :  RTTR  \\  JsonCpp

​Getting Started

​So the first thing we need to do is get the properties out of our game objects components. As stated before, we want to use reflection to grab these properties from the object. We also probably want to be serializing all the game objects in our game object array so we want to be going through each object, then going through that objects components, and serializing all the reflected properties of the object.

Once you add RTTR or however you choose to do reflection to your project, you will somehow need to register your components so that your engine knows they will need to be reflected. We will start off by modifying our BaseComponent class (changes in this guide will be highlighted in green):

​​As you can see our components will now have a name associated with them and must now be passed in when a component is created. This will be used later on when trying to access a derived components properties. The other addition is calling the RTTR_ENABLE() macro at the end of our BaseComponent. This macro allows RTTR to know to register this class. Nothing is bieng passed into it currently but in our derived components we will have to pass in the components base class (Base Component).

​Next up we will need to go through and change our components. As was just mentioned we will need to be registering our components with RTTR so the only real change to the components header file will be to add "RTTR_ENABLE(BaseComponent)" to the end of the class. Be careful with this macro, as everything after it will now be declared as private. If your class has private properties that need to be serialized be sure to add the
"RTTR_REGISTRATION_FRIEND" macro to the end of the class as well. Here is our updated TransformComponent that is now being registered with RTTR:

​​We also now want to actually register our components properties in the cpp file as well as change the constructor by adding the Components name in the initialization list. See following code on how to do so:

​And just like that our transform component is now able to be reflected at run time and all TransformComponent instances properties can be accessed and modified. 

​Accessing Components Reflected Properties

 Now that our reflection is set up, we want to be able to get all our reflected components properties and serialize them! To serialize a whole level we are going to want to make sure we get all objects in the level and add their components to those objects. We will do this with nested for loops iterating through our game objects, then through those objects components. From there we will use RTTR in order to get the current components properties to get their values. This technique requires that all property types, both built in types like ints or floats as well as user defined types such as Vector3, are handled appropriately. Here is an example of how to get all properties of each game object in the level; the resulting property value will be stored in a variable called "data":

​Serializing our data

​Now that we are able to access each property of every game object in the level, we want to store that data to a JSON file for later deserialization. Using JsonCPP we can create JSON objects and print them to a JSON file. We will need a bit more information such as the property name as well as the components name. The JSON objects will be nested with the largest element being the level itself. Below is the code for using JsonCpp to serialize as well as an example output for the JSON file, since the function is so big, for readability I have split the function into 2 separate images:

An example JSON file for our level

​Deserializing our data

 Now everything we have done so far has outputted this JSON file that contains all the information we need to reconstruct our level if we wanted to restart our game. We can once again use reflection in conjunction with JsonCpp to deserialize our JSON file start creating our game objects from these files. In my opinion the best place for this functionality is in the GameState Init function. Depending on the current level we want to open up that levels corresponding JSON file and parse it in order to remake that level. 

Before we move onto that though, we are going to need a way to create a way to create and attach a component on the fly given just the name of the component, a game object, and a Json Object; we will do that by creating a Component factory and adding a function called CreateComponent that will take these three things as parameters and handling all of this for us.

Here an example for the component factories CreateComponent function, I have once again split the image into 2:

​Now that we have this functionality, all we have to do is get the 3 parameters we need to pass to the function for each objects components that we read in from the JSON file and we are done!

We are going to want to open the JSON file and read all the data into a JSON object. Then go through the objects one by one to each of their components and call the function we just created. Here is the functionality that we want to add to our GameStates initialize function:

​And finally we have fully functioning serialization and deserialization for our levels and now we've eliminated any need for hard coded levels since we can create levels on the go. As long as you register any new component with RTTR it will be serialized and deserialized for every instance.

I know this was a lengthy subject, but I truly believe the time it spends to implement this functionality is way better than time spent writing a new serialize and deserialize function every single time you want to create a new component. This way designers/developers can easily make new components without having to worry about writing these functions every time and greatly increase development time.

If any of the material covered was confusing or complex and you have any questions you can feel free to contact me via any of my social links. I hope this helped and good luck!