Making A Free Unreal Engine Serialization Plugin

This summer, I released a serialization plugin called "Numbskull Serialization" to the Unreal Engine Marketplace.

I started working on this during development of Beyond Binary.

We felt it was necessary to create our own serialization system due to plugins being in the £60+ range (Rama Save Plugin) and Unreal not having built-in support for saving and loading entire actors.

The process of creating this plugin ended up being rather confusing, long-winded and difficult. However, I am slightly used to diving head first into problems.

I hope this blog post will help point newcomers in the right direction when it comes to serialization and plugin development.

Confusing Tutorials

When I started researching serialization, I struggled to find tutorials. And worse than that, the few tutorials I found would sometimes contradict each other and confuse me.

Before I go further, I should promote this tutorial as probably the best out there for understanding UObject serialization and serializing a large number of objects.

The first tutorial I read was Rama's "Read and Write Any Data" wiki post. He taught readers the << operator and how to save binary data to disk. However, the post steered readers (at least it did for me) into writing your own serialize methods.

In reality, doing this for a proper save system is incredibly inefficient and rather redundant. When reading Rune de Groot's tutorial, it's clear that marking properties as SaveGame and calling Serialize can save most of the data people need.

The only edge case is saving an actor's transform/position and their name and class when spawning actors from disk.

Other tutorials have followed Rama's wiki post and ended up with the same pitfalls.

The only other document, besides Rune's, is this answer from a Fortnite dev. It's short, concise and accurate and doesn't steer beginners into writing SaveLoadData methods for every object.

There are some okay learning materials like this forum post, but it's probably best to stay clear. While the writer eventually figures out a good serialization system, there are still bad ideas in the post that can be confusing if you don't know what's correct or incorrect.

Finally, I did find this answer which correctly saves a UObject. However, it's not quite the best solution and doesn't provide a solution for loading a UObject.

In short, I found two resources for learning a mechanic present in almost every game.

The Final Product

Here are extracts from the final plugin which recap the correct procedure for serializing data.

However, take this with a little grain of salt as it's unlikely to be a perfect solution.

UObject

void Serialize(TArray<uint8>& OutSerializedData, UObject* InObject)
{
    FMemoryWriter Writer(OutSerializedData, true);
    FObjectAndNameAsStringProxyArchive Archive(Writer, true);
    Writer.SetIsSaving(true);
    InObject->Serialize(Archive);
}

void ApplySerialization(const TArray<uint8>& SerializedData, UObject* InObject)
{    
    FMemoryReader ActorReader(SerializedData, true);
    FObjectAndNameAsStringProxyArchive Archive(ActorReader, true);
    ActorReader.SetIsLoading(true);

    InObject->Serialize(Archive);
}

Saving and Loading

void SaveBytesToDisk(const FString& InFileName, const TArray<uint8>& InBytes)
{    
    FFileHelper::SaveArrayToFile(InBytes, *InFileName);
}

void LoadBytesFromDisk(const FString& InFileName, TArray<uint8>& OutBytes)
{
    FFileHelper::LoadFileToArray(OutBytes, *InFileName);
}

AActor

void SaveActor(AActor* InActorToSave, FActorProxy& OutActorProxy)
{
    FActorProxy ActorProxy;    // FActorProxy is a struct included in Numbskull Serialization
    ActorProxy.ActorName = InActorToSave->GetFName();
    ActorProxy.ActorClass = InActorToSave->GetClass()->GetPathName();
    ActorProxy.ActorTransform = InActorToSave->GetTransform();

    Serialize(ActorProxy.ActorData, InActorToSave);
    OutActorProxy = ActorProxy;
}

void LoadActor(const UObject* WorldContextObject, const FActorProxy& InActorProxy, AActor*& OutLoadedActor)
{    
    UWorld* const World = GEngine->GetWorldFromContextObject(WorldContextObject, EGetWorldErrorMode::LogAndReturnNull);

    FActorSpawnParameters SpawnParams;
    SpawnParams.Name = InActorProxy.ActorName;
    SpawnParams.SpawnCollisionHandlingOverride = ESpawnActorCollisionHandlingMethod::AdjustIfPossibleButAlwaysSpawn;
    SpawnParams.OverrideLevel = World->PersistentLevel;
    UClass* SpawnClass = FindObject<UClass>(ANY_PACKAGE, *InActorProxy.ActorClass);

    AActor* SpawnedActor = World->SpawnActor(SpawnClass, &InActorProxy.ActorTransform, SpawnParams);

    ApplySerialization(InActorProxy.ActorData, SpawnedActor);

    OutLoadedActor = SpawnedActor;
}

Saving More Than Binary Data

In the AActor example, you can see the use of a struct for storing more information that just the binary data.

The Serialize method requires an array of uint8 so to convert the FActorProxy (or anything other struct) to an array, the struct must declare the << operator.

    friend FArchive& operator<<(FArchive& Ar, FActorProxy& ActorProxy)
    {
        Ar << ActorProxy.ActorClass;
        Ar << ActorProxy.ActorName;
        Ar << ActorProxy.ActorTransform;
        Ar << ActorProxy.ActorData;
        return Ar;
    }

And a method needs to be written to convert the struct into an array.

void SaveActorProxyToDisk(const FString& InFileName, FActorProxy InActorProxy)
{
    FBufferArchive BinaryData;
    BinaryData << InActorProxy;

    SaveBytesToDisk(InFileName, BinaryData);
}

I should be transparent and say that Rama's tutorial is helpful here. While it can throw off beginners trying to make their own serialization system, it still teaches an important section of serialization.

Releasing The Plugin

Once I wrote the plugin, I went through the publisher portal and started preparing it for release following the marketplace guidelines.

This process was rather straight forward. I went back and forth with the reviewers a couple of times but I had a huge problem with the certain wording of the guidelines.

However, this problem was just me not reading things properly.

Unreal states

Plugin folders must not contain unused folders or local folders (such as Binaries, Build, Intermediate or Saved)

I took this to mean "never include the Binaries folder". I also convinced myself that the plugin could be compiled on the client's end, stopping me from needing to build for Windows (when I'm on Mac).

Therefore, when testing the plugin in the engine, I would delete the Binaries folder. Unsurprisingly to some, the plugin wouldn't compile and didn't throw any errors.

I assumed there was some massive bug and took a long time debugging. This inevitably failed and I gave up for a while.

I recruited the help of another programmer at Numbskull but he too couldn't solve the problem.

I eventually had another crack at debugging and decided to see what was included in other code plugins. What I found was a Binaries folder full of library files and a complete Intermediate folder.

I quickly reread the guidelines and found my mistakes.

I built the plugin for Mac, Android and IOS, and got my friend to compile for Windows, submitted and was accepted the same day!

Audience Reception

Numbskull Serialization sold 1000 units in its first two days and 2000 in its first four.

I'm hesitant to say these 2000 sold units are actually being used; I suspect people have seen a free plugin and quickly nabbed it for future projects. However, I was expecting about 10 people to download it over a month so I'm rather blown away!

I definitely believe there's a market for the plugin. As far as I can tell, it is the only free serialization plugin on the marketplace and its code is available on GitHub.

My hope is that the plugin can act as an open-source alternative to the expensive full save systems. Or, just help people needing to implement serialization and want a good starting place.