More or less, nope. If you want your custom server game-mode to run, you need to download the files of said custom game-mode onto the computer of those joining the server. And said files need to be readable by the game engine, and the similar; as of such, they can’t be encrypted, and more or less the whole package needs to be downloaded. Even if they were Unreal-Engine specific files, it’d still be entirely possible to view their contents via a wide variety of external programs. More or less, the instant someone joined your server, you would have already made your gamemode available to the public. Short of a direct modification to the game encrypting the files downloaded, which would most likely obfuscate to an extreme degree the process of a host sending files to a client, increase load times to an extreme degree, and be entirely irrelevant as soon as the first program comes out to de-crypt the files.
Now, you might be thinking: “But wait a second… couldn’t you just DMCA any rip-offs of the code and be done with it?” Sure, you could, but that stops nothing. Anyone with a basic knowledge of C++ could just modify the files to a point where it’s unclear who actually created it, and even then; TTT is a popular gmod gamemode, and the format is pretty clear; if someone switched around the GUI elements for it, you could very well claim that it’s entirely original; after all, the idea’s been out there for a while.
Now, of course, if you were really determined, you could bypass these limits. There’s all sorts of things you could do to determine if someone had stolen your code; if you had enough time, it’d be possible to manually take down the servers by joining each and every one of them and scanning their downloaded files for any traces of your work. You could them DMCA them… but wait a second.
DMCA stands for Digital Millennium Copyright Act. Key word, copyright. Unless you’ve gone and filed a full copyright for your code, there is nothing that you can do, legally to prevent them from hosting a server with said code. So, more or less, the instant any copy of your code gets out there, your entire goal of making it a private gamemode is more or less undone, and permanently.
Edit: Turns out, I don’t know copyright law very well. Still; you can’t really takedown an owner of a server running stolen code without a law-suit, as the server host is in no way under steam’s jurisdiction.
And the entire time, I’ve not even mentioned the possibility of someone just making their own version of TTT. That is entirely possible. And think about it; what version would new server owners use? Even if your servers are less “greedy” and “cringy”, you can’t beat volume. Plus, someone could just host a server run on the same lines as yours, but using their own version of TTT. But there wouldn’t really be a limit to p2w or similar servers; more or less, your entire vision could become undone, and in quite a spectacular way.
So, this all sounds pretty bad. More likely than not, your original vision will collapse in on itself in any number of ways. Someone could steal the code, via decompiling what they’ve downloaded off the server, or via internal leaks, and host their own server with absolutely zero you could do against it; or, someone could just make their own version of TTT, drowning out your servers with the sheer volume of servers that would be able to host it.
There is, however, better alternative. Instead of just locking down the gamemode, and providing any number of motives for those that have the skills to carry them through, why not just make it open for public use, but include a disclaimer on how it would be run? Sure, not everyone would abide by it, but would having some measure of control over the TTT servers not be better than having strict control over a small handful of servers with the rest of the servers going entirely unregulated?