Developer Update 12/2024

The Side 7 team is continuing its ongoing work to provide parity to the original experience provided by Gundam Evolution during live service. The below is a write-up of our ongoing efforts to improve the game’s backend infrastructure alongside the reintroduction of casuals. Below is a write-up of our infrastructure engineer, Moekumo (Discord: @moekumo) that highlights where we’ve been and we are going in terms of infrastructure clean-up and delivery of casuals.

The Side 7 private server project has unique challenges that very few similar projects need to solve.

We are not working with the original game code, instead we have to painstakingly reverse-engineer the native game assembly and use the network captures that Gamemaster took over a year ago from the client as reference. This can make even straightforward tasks challenging and time-consuming. Occasionally, our initial understanding turns out to be wrong and we need to backtrack and explore different avenues to eventually find the solution.

Games with private server support typically offer dedicated server builds, either included in the game or provided separately. These builds are specifically designed to run as servers. Players can then browse and select servers, similar to Source Engine games like Team Fortress 2, and/or connect directly to a server's IP or hostname, like in Minecraft.

In our case, none of that is available. Instead we have to use the regular client, the very same one all of you have and use to join us online, and convince it to run in “server mode” despite being designed to explicitly not support that scenario. We achieved this by carefully crafting a sequence of patches that are applied during the early loading process. However, it doesn’t end there, to make things worse, we only have network traffic captures from clients, not from the servers. We had to piece together how the server communicates with the game's online services by finding clues in the UE blueprints bytecode and by observing, with trial and error, how the game process would fail when sending requests to our incomplete service backend.

Eventually, with the foundation of a working game server, we had to figure out how to actually get players to connect to it. The quickest way to get this required replicating the custom match feature and creating a basic server allocator. The allocator’s job is to find an appropriate server for an incoming group of players and communicate the game details so the server can prepare itself to accept the players and run the match.

Running a fleet of game servers is very expensive. And, unlike a traditional modern live service game, where we have a budget set aside to have servers deployed in various regions to pair to matches (think Overwatch), our budget is $0 (or rather, in the negative). If we tried to host all the servers ourselves, we (the developers) would go broke very fast. As a result, we rely on crowdsourcing, where volunteers from the community provide their own infrastructure for us to run our "game servers" on (the same ones described above that require many hoops to operate). However this brings some additional, but necessary, complexity. This requires us to support and troubleshoot a number of different hosting scenarios and varying hardware configurations.

Matchmaking in itself is a hard problem to solve. For instance, with a pool of only 20 players, there are over 126,000 potential combinations for forming teams of 12. This number jumps over 120 billion with just 50 players. It's clear that exhaustively searching for the optimal match becomes near impossible and the matchmaker necessarily has to make compromises. For this, we can use the player skill range, region and also the time spent in queue to further restrict or relax the constraint.

Our initial objective was to spark some life back in the game as soon as possible. We had to prototype a lot of things very quickly, sometimes for the sole purpose of figuring out what the actual requirements for the game’s backend were. Evidently, many design decisions that we had to make with incomplete information turned out to be sub-optimal and, as we anticipated, are coming back to haunt us. This brings us to today with a significant cross-cutting change like adding matchmaking queues which is forcing us to move on from our early prototype and revise those areas with our mature understanding of the game client. In connection, we also need to make important changes to the infrastructure to make way for the new matchmaking components and to improve the general reliability of the system. This will also solidify the foundation for additional features and improvements going forward.

Below is diagram of the request flows for the major components we are planning for the improvements and Casuals implementation:

The Side 7 team will be providing more updates in the coming weeks.

If you are not yet a member of Side 7, you may join us over at https://discord.gg/side7