Game Server in 150 lines of Rust
Okay the title makes it sound like you’ll be writing game servers in 5 minutes. And you could skip straight to the source code if that is all you want. However, the objective of this article is to provide insight into the design decisions in the code, and possibly to help you conceptualize how a game server works in the general case. The article should also be useful if you are just learning the Rust programming language or its asynchronous libraries. In 150 lines it is not going to be a great game. In fact it can hardly be called a game. But the focus is on a scalable server infrastructure that can be expanded upon to support a game. In the code I use Rust’s asynchronous futures with the tokio runtime. If you haven’t looked into those but you do understand asynchronous programming, perhaps from NodeJS, you should be alright. Of course it’s very possible to create a server without these fancy features, but I believe it makes the code more readable and easier to reason about. Because these features are still new I’ll try to explain any core assumptions on futures and the runtime as I go.
The term “game server” is a little ambiguous. What I mean is a real-time server in which a client-player can establish a connection to subscribe to consistent world state updates at some frequency, and can send commands to the server to update their own entity as part of the world state. To clarify a little more, yes World of Warcraft, yes League of Legends, no Hearthstone.
Unlike in Hearthstone, the game world will continue churning whether or not any player makes a move. This distinguishes the program from a simple reactive service. The server needs a main loop to process and broadcast the game state at timed intervals. When does the server handle new player connections or player commands? In a single threaded program, it may be part of the loop.
fn main() {
// Process game state
for e in &entities {
e.update();
} // Handle new connections
...
// Process new player commands
... // Delay 1/60 of a second
...
}
Perhaps we have some big buffer that accumulates new player commands for the program to iterate through each cycle. But the frequency of the game loop is now coupled to the processing of client requests. A game server usually runs at a fixed rate of somewhere around 60 times a second. That is really slow to be processing a potentially large number of client connections.
I’ll point out that there is one benefit of this approach but I don’t want to linger on it because its beyond the scope of my intention in this article. That is that it is easy to guarantee that client updates are processed in the order they are received. This is important in a game engine, but there is less concern on a server because packets being sent over the internet can already arrive in different orders and speeds. But enough on that.
To scale this server, we’re going to make it multi-threaded. A simple intuition might be to divide the client message handling and the game loop into two concurrent threads. And that is a good idea. But we can go further, and we will because tokio makes it easy. With the designated threads approach, the client handling thread can still handle only as many messages as one thread can process. Instead we will use tokio’s thread_pool executor to load-balance tasks across a set of threads. Here’s the setup.
let runtime = runtime::Builder::new().build().unwrap();
let executor = runtime.executor();
let server = Server::bind("127.0.0.1:8080", &runtime.reactor()).unwrap();// Hashmap to store a sink value with an id key
let connections = Arc::new(RwLock::new(HashMap::new()));// Hashmap of id:entity pairs. This is basically the game state
let entities = Arc::new(RwLock::new(HashMap::new()));// Used to assign a unique id to each new player
let counter = Arc::new(RwLock::new(0));
If you are unfamiliar with the Arc and RwLock/Mutex data types, I recommend you read the Shared State chapter of the Rust book. Also my previous article is a good introduction to dealing with shared state in closures.
Now here’s the really cool part. On accepting a Websockets connection, the future returns a connection object that can be split into two objects commonly called a sink and a stream. This stream (confusingly is also a futures stream in that it is an iterator that returns futures) is invoked on new messages from the connection. In opposition, the sink is used to send messages down the connection to the client. By dividing the two functionalities — sending and receiving — we can split the program into two logical sections, receive handling and send handling.
let connection_handler = server.incoming()
.map_err(|InvalidConnection { error, .. }| error)
.for_each(move |(upgrade, addr)| {
let executor_inner = executor.clone(); // "accept()" completes the connection
let accept = upgrade.accept().and_then(move |(framed,_)| {
let (sink, stream) = framed.split(); // Increment the counter by first locking the RwLock
{
let mut c = counter.write().unwrap();
*c += 1;
}
let id = *counter_inner.read().unwrap(); // Store sink with a unique id
connections.write().unwrap().insert(id,sink); // Assign a new entity to the same id
entities.write().unwrap().insert(
id,
types::Entity{id, pos:0}
);
let c = *counter.read().unwrap(); // Spawn a stream for future messages from this client
let f = stream.for_each(move |msg| {
process_message(c, &msg, entities.clone());
Ok(())
}).map_err(|_| ()); executor_inner.spawn(f);
Ok(())
}).map_err(|_| ()); executor.spawn(accept);
Ok(())
})
.map_err(|_| ());
Connection_handler is a stream of futures that will be invoked each time a new client connects to the server. The closure in for_each(..) (line 3) is where the logic goes for what to do on each new connection.
First, the executor must be cloned (remember it is an Arc type) to be used in the inner closure “accept” (line 7). This is actually a pretty important concept, and if you look at the actual source code for this server you will see a lot more clonings. They tend to grow in number with the size of the code unfortunately. You may wonder why the “accept” closure needs to capture variables with the “move” command at all. The fact is that right now, futures take closures with static lifetimes. That is because it is hard to know when a future will finish executing, and it will most likely live longer than the scope it was defined in. Futures are still an early feature in Rust. I imagine in the “future” …hah… Rust will accept tighter lifetime bounds on future closures but for now the closure must take ownership of any variables it uses in order to be sure they aren’t dropped out of scope. That is why you see all future closures use “move”.
Notice beyond the clone on line 4, all the program logic is in the inner future “accept”. In execution, this means that as soon as a new connection is established, the connection_handler will spawn an “accept” task to process the new connection, which may run on another thread. As soon as the task is spawned, the connection_handler may return to listening for new connections. That’s efficient.
Inside the accept future, the unique id counter is incremented and a new entry in the connections hashmap is initialized to store the id:sink mapping. As well as the entity hashmap to store the id:entity mapping. You can create more efficient data structures to track your player state, but this is not so bad for 2 lines of code. Finally, the “f” stream is spawned to process each new message from the player. Even though the “accept” task will terminate, “f” will continue processing messages forever. This is the beauty of asynchronous programming to declare exactly how a program should respond on a given stimulus, but abstract the “when” to be determined by the internal runtime.
I decided not to show the “process_message” function here. It’s a few simple lines and you can see it in the source code. Essentially if the player sends the “left” command, their entity is moved one to the left and vice versa for “right”. Notice the function takes an entities clone in order to modify the state. Also notice that process_message doesn’t get its own future — the “f” stream blocks until it is finished. That is because we don’t want the possibility of a new player command executing before an old has a chance to.
Now for the second half of the program, the send_handler.
let send_handler = future::loop_fn((), move |_| {
let connections_inner = connections.clone();
let executor = executor.clone();
let entities_inner = entities.clone(); tokio::timer::Delay::new(Instant::now() + Duration::from_millis(100))
.map_err(|_| ())
.and_then(move |_| {
let mut conn = connections_inner.write().unwrap();
let ids = conn.iter()
.map(|(k,v)| { k.clone() }).collect::<Vec<_>>(); for id in ids.iter() {
let sink = conn.remove(id).unwrap(); // Meticulously serialize entity vector into json
let serial_entities = ... // Omitted let connections = connections_inner.clone();
let id = id.clone(); let f = sink
.send(OwnedMessage::Text(serial_entities))
.and_then(move |sink| {
// Entry goes back to the map
connections.write().unwrap()
.insert( id.clone(), sink );
Ok(())
})
.map_err(|_| ()); executor.spawn(f);
} // Strange way of saying loop forever
match true {
true => Ok(Loop::Continue(())),
false => Ok(Loop::Break(())),
}
})
});
loop_fn() is a tail-recursive future that makes it easy to loop futures indefinitely. Good for a game loop. Each loop, there is a 100ms delay before executing the core logic. The logic is a little strange here so I’ll first explain what is effectively happening, then explain how that is accomplished in Rust with its ownership rules. Functionally, all this does is iterate through the ids of each player, serialize the game state (all entities) into json, then send that json to the client of the corresponding id. An obvious optimization would be to just serialize once, not on each cycle of the loop. But I’ll leave that as an exercise.
The Rust borrow checker is a notorious adversary to newcomers of the language. The above code is an example of a scenario that could actually be improved to be more programmer friendly. At least syntactically. The issue here is that a hashmap will only return a reference to one of its values (the sink in this case). However we need full ownership of the sink in order to send a message to the player on it. To accomplish this, we must remove the entry from the hashmap (first line of the ids loop), and re-insert it when we’re done sending (inside the “f” future). That’s not so bad once you see how it works.
The previous issue leads into the next oddity. Why allocate a new vector of ids instead of just iterating over the connections hashmap? The reason is that an iterator borrows the self immutably, then inside the loop the “conn” is borrowed mutably when removing an entry from the hashmap. The easy way around this is to just clone the ids because they are cheap u32.
for id in conn.iter() { // Immutable borrow
conn.remove(id).unwrap(); // Mutable borrow not allowedFinally, you’ll notice that serial_entities is not defined. It’s just a conversion to json and it’s a little ugly because of an edge case I had to handle. The implementation can be seen in the source code but I thought it just subtracted from the important bits here.
To tie the program altogether we use a futures combinator; select(..). Select combines the connection_handler and send_handler futures streams into one future that completes when either of the two complete. Remember, the only instance in which one of those futures would complete is on a fatal error.
runtime
.block_on_all(connection_handler.select(send_handler))
.map_err(|_| println!("Error while running core loop"))
.unwrap();That’s just about everything. You may have noticed I didn’t define the Entity struct anywhere. Its a simple definition so I put it into the types.rs file which can be found in the source code. Also in the repository is an html5 canvas client to interface with this server. Try it out, bonus points if you play multiplayer with a friend.
If you’re wondering how to fix the stutter in the movement, the solution is not to increase the response time of the server. Instead, use a client-side LERP to smooth out the difference.