Where I left off

Last post the client packed up, loaded the arena map, and said JOIN. Then it sat on the loading screen forever, waiting for the server to hand it a character to control.

Handing it a character is the whole next job. But there’s a catch I’ve been circling for a few posts now: the exact format of those “here is a character” messages isn’t written down anywhere I can read. It lives in the game engine’s compiled C++ code, not in the script I decompiled back in Phase 2.

Every time I’ve hit that problem so far, the same trick got me through: send the client a guess, watch what it does next, and let its reaction tell me if I got it right. The client is a fussy, precise little oracle. Poke it, read the twitch.

This post is about that trick running out of road.

The client goes quiet

My plan was the usual one. The game engine this client is built on can log an enormous amount of detail about what it’s receiving over the network: every message, every field, every “I don’t understand this, throwing it away.” All I had to do was switch that logging on, point the client at my server, and read its diary.

The switch is in a config file. There’s a list of log categories the game is told to keep quiet, and two of them are exactly the network ones I wanted. I deleted those two lines. Ran it. Nothing. Same tiny fifteen-line log as before, with a note in it that literally says [1 lines suppressed].

Deleted more lines. Tried a second config file. Tried the “yes I really mean it” override syntax. Still nothing.

Here’s what’s going on. When a studio ships the final version of a game, they build it in a mode that strips almost all the logging out of the program entirely, for speed. The lines I was deleting from the config were instructions to a logging system that mostly isn’t in the shipped .exe anymore. There was never a diary to read. Auran also swapped in their own cut-down logger that only writes a handful of “analysis” lines per session, and that’s all I get.

I checked for a debug build of the client too, the kind with all the logging left in. The config file even names one, DEBUG-GOGame.exe. It’s not on the disc. Only the three retail renderers ship, and they’re all gagged the same way.

So the oracle still answers, but only in grunts now: it either leaves the loading screen or it doesn’t. It either crashes or it doesn’t. No running commentary. For the byte-level detail milestone 4 needs, that’s not enough to work with.

Opening the patient up

Which leaves the option I’d been saving for “much later”: stop poking the game from outside and read the engine’s actual code.

The client is about 15 megabytes of machine code. Machine code is what a compiler spits out at the end: pure numbers the processor runs directly, with all the helpful names and structure of the original source thrown away. You can’t just open it in a text editor and read it.

What you use instead is a disassembler, and the good free one is called Ghidra (built, weirdly, by the NSA, and then open-sourced). You feed it the .exe, it spends a while untangling the numbers back into something that reads almost like C code: ugly, no variable names worth anything, but followable if you’re patient. This is the first time this project has needed one. I installed it, and a Java runtime for it to run on, and that was its own little afternoon.

But 15 MB is a lot of haystack. I needed to know roughly where in it to look before Ghidra would be any use.

Finding the needles first

Even a stripped .exe keeps one kind of text: the error messages. Things like bunch header overflowed or trying to create channel of unknown type are baked in as literal strings so they can be printed if something goes wrong, and the code that would print them sits right next to the code that does the actual work I care about.

So the first pass was cheap and text-only: pull every leftover string in the binary that mentions networking, note its address, then scan the rest of the code for the spots that reference that address. Cluster those, and you’ve got a shortlist of “the interesting network code is somewhere around here.”

That turned 15 MB into about fifteen specific addresses. It also turned up a few freebies just from the strings:

  • A control-channel command I hadn’t seen the client send yet, HAVE GUID=... GEN=..., which is how the two sides agree on the list of game files they’re both working from.
  • Confirmation that the WELCOME message I guessed at in post 20 has exactly the shape I guessed.
  • The same handful of functions exist, byte for byte identical, in the low-spec version of the client. Which tells me they’re stock engine code, not something Auran customised, so anything I work out about them I can trust against public knowledge of the engine.

The read

Then Ghidra chewed on the full binary for about thirty-five minutes, I pointed it at those fifteen addresses, and it handed back the decompiled versions of the eleven functions that matter, plus the smaller functions they call. Around 300 KB of that almost-C. I did not read all of it line by line myself; I had a second AI pass go through the dump with the specific questions I needed answered, the same division of labour I’ve written about before, then I went through its findings against the raw code and the captures.

Here’s what came out of it.

We’ve been lucky

Back in post 19 I bit-decoded the message header by hand and labelled three of the bits “package map flag, then two spare bits.” I got that wrong. Those three bits are a small number that says what kind of channel this is: control, actor, file, voice. My server has been sending “control channel” for every message. That happens to be the right answer for the handshake, which is the control channel, so it worked. It would have been silently wrong for every single message in milestone 4, where the channels carry characters, not text. Found it before it cost me a week. I’ll take luck.

A whole job that doesn’t exist

I’d budgeted time to reverse-engineer how the client tells the server which messages it successfully received, so the server knows what to resend if something drops. It’s called an ack history, and in later versions of this engine it’s a fiddly little bitfield.

This build doesn’t have one. At all. The client says “I got message number 340” and that’s the entire report. If the server sent 338, 339, 340 and only hears about 340, it works out that 338 and 339 are missing by the gap. Simple, and one fewer thing for me to build. This also finally explains a couple of mystery bits I’d flagged and shelved two posts ago: they were just packet padding.

The one that made me laugh

The engine has a function for sending a growable list (an “array” that can change length) across the network. I went to read how it encodes the length and the elements.

It’s one line. It does nothing and returns. In this build, a growable list in networked game data just doesn’t sync, ever, and nothing warns you. Fixed-size lists work fine, and everything I need for milestone 4 uses those, so it doesn’t bite me. But it’s a hell of a thing to just find sitting there.

Objects point at each other by pipe number

This one actually shapes the next month of work. When the game needs to say “this player’s controller is that object over there,” it has to put “that object” into the message somehow. I expected an ID number.

It isn’t. For anything spawned during the match, the reference is the channel number: “that object is the thing coming down pipe number 4.” Which means an object has to already have its own pipe open before anything else is allowed to mention it. So I can’t just fling everything at the client in any order. It has to go: the match-info object first, then the player-info object, then the controller, then the actual character body, each one fully established before the next one is allowed to refer back to it.

Where this leaves me

For the first time since Phase 3 started, the actor channel isn’t a wall of unknowns. It’s a written spec: here’s the header, here’s how an object reference encodes, here’s how each type of value goes on the wire, here’s the order to open things in. research/phase3/re/wire-format.md in the private repo, if you’re the kind of person who’d want to read it.

Milestone 4 just turned from “figure out what the client wants” into “type it out.” Different, better problem.

Next

  • One decision to record first, its own short post: there are two routes to getting a character standing in that map, and now that I know what the client actually needs, I can finally pick one.
  • Then the build: teach the server to open a channel, name an object, and send a character down it in the right order.