Multiplayer Updates and Protobuf Usage

Today I started the "no-going-back" multiplayer conversion for AstroMiner.  While I do have many backups and I could theoretically go back to day one, I am considering this the point of no return.

As I mentioned before, a lot of code has to be changed to allow multiplayer support.  I could hack it in and make it so it's not nearly as much effort, but that runs the risk of future hassles whenever I add new features.  There would be redundant code running on both client and server that wouldn't need to be run.  How do I know which results to trust?  Server or client?  What if my client disagrees with someone else's?  These are all questions that I asked myself when I decided to make this conversion.  If you want more detail, read my blog post from a few days ago that describes my model for multiplayer.

So today I started removing code from the client.  When the game launches, the server is also launched silently in the background.  The client no longer saves or loads maps.  That's all handled by the server now.  When you start a new map, you actually send a command to the server telling it "load this map" or "start a new map with this seed".  The server then does its business and sends another packet back telling you it's done.  At that point, the client says "give me all the updates around my position".  This was the tricky part.

Gathering the updates wasn't hard.  Building a container class for them to be transported wasn't hard.  What was hard is figuring out a solid way to transport them without taking so much space.  I toyed around with several serializers.  XMLSerializer was slow and took up a lot of space.  BinaryFormatter was fast, but it can't handle Vector2's.  I tried some custom libraries, such as the Sharp Serializer.  This is an awesome little library and would've worked, except it was giving me issues with Vector2's as well.  Vector2 is an XNA format and I can't modify it, so I needed something that would let me define my own custom serialization types.  Google's Protobuf came to the rescue.  More specifically, Marc Gravell's Mono Implementation.  This not only lets me define custom types, it is extremely fast when it serializes and extremely compact.  For the sake of internet searches, I'm going to post my code to serialize an XNA Vector2:

First, you want to declare a RuntimeTypeModel:

     private static RuntimeTypeModel serialize;  

Then in your initializer, you need to instantiate this model with all of your custom classes.  Mine looks like this:

     public netManager()
     {  
       serialize = RuntimeTypeModel.Create();  
       serialize.Add(typeof(Vector2), false).Add("X", "Y");  
       serialize.Add(typeof(chunkUpdate), true);  
       serialize.Add(typeof(change), true);  
       serialize.Add(typeof(block), true);  
       serialize.Add(typeof(Rectangle), true);  

By the way, I did all the coloring by hand.  You're welcome.

Ok so to explain that, I'm creating a model so Protobuf knows how to serialize my object.  For Vector2's, I'm telling it I only need the .X and .Y values.  For all my custom classes (chunkUpdate, change, block), I'm telling it to figure it out on its own.  It's also reading XNA's Rectangle on its own.  I'm willing to bet it can read Vector2 on its own, but I haven't tested it.

Anyway, once that's all fancy and done, I simple serialize like so:

 serialize.Serialize(fullPacketStream, cu);  

fullPacketStream is a MemoryStream and cu is my chunkUpdate object that I'm serializing.  I turn then fullPacketStream into a byte array and send it off on its journey to the client.  When the packet is received, I deserialize like so:

 serialize.Deserialize(chunkUpdateMemoryStream, chunkUpdateInput, typeof(chunkUpdate));  

I now have an intact chunkUpdate class that contains all of the updates for a given chunk of data.

I spent most of the evening learning about protobuf and getting this to work.  I think that by tomorrow night, I should be at the point where I can create a new map or load an existing one.  The map will be able to save itself to desk when the server is done with it and the player will be able to travel all throughout the asteroid field and have data being streamed to them on demand. (I just jumped from first person to third)

Anyway, no screenshots to show because it's all just code :[  I will hopefully have something tomorrow though when I get the full streaming set up.

Until then ~

Successful Multiplayer Test

Just wanted to report that I've had a successful early multiplayer test.  I logged into the server with my local machine.  I then used tethering through my phone to have my other computer logged into my server through my remote IP.  Lastly, a friend of mine logged in from his computer.

Lag was very minimal (I actually didn't notice any, but I'm local).  We flew around.  He deleted 2 blocks and I saw it on my screen.  Overall, a successful test :]

One thing I haven't addressed is world syncing.  He came into a world that was different than mine.  When the player comes onto a server, they'll need to be updated with all the changes/structures before they can start playing.

I'm currently on the 2nd milestone.  I've decided that generating the world is going to be left on the client side.  I think it would be too much work for the server to constantly generate asteroid locations around every single player connected.  Instead, I have several hashing techniques in mind so I can make sure that all the player worlds are similar.

Adding/deleting blocks has been completed.  I have the packet builder complete and I've been utilizing it.  I've decided that the floatingItems can stay on the client side.

Next thing I have on the list is to sync up the worlds between players.  After that, syncing up player inventory and passing in more player information.  You currently can see a player standing on a block and the block being slowly eaten, but you can't tell the player is drilling.  You can't see the thrust that the player is using.  All that needs to be sent to the server.

After all that, I'm going to actually add bullets before the next milestone.  That will leave adding structures as the only player interaction left.

Anyway, I'm progressing well :]  This is actually more fun than I thought it would be, heh.

Until next time ~

Multiplayer Milestones and A Summary of the Past Year

I started coding today by jumping straight into multiplayer.  I added support for structures, multiple players, AI and the environment when I realized I'm doing this all wrong.  I need to stop and rebuild from the ground up.

So I've come up with a plan.

I have my base server.  It currently compiles fine, but most of the sections where I'm checking for player collision are commented out because they currently only support 1 player.  I need to add multiple player support for those, but that's for later.  For now, I have my base server with all of the AstroMiner objects and classes built in and compiling.

Next on the list is the world.  Generating asteroids, for starters.  This will be done on the fly, like it currently is, and the player should be able to fly around in an infinite world, like they can currently do.  This also means I will be adding support for multiplayer player movement, light support (which I'm going to calculate client side) and some other things.

Once that milestone is passed, I will move onto adding/deleting blocks.  This will force me to build out a changesManager that can dynamically update from any source and report what changes it has so I can build an efficient packet to send.  We obviously don't want to send all the changes all the time.  We only want to send updates.  These kinds of updates are important, so they will require some validation on the client end telling the server that they received the update.  This will also require me to sync up the player's inventory (and now that I think about it, the floating items manager (the little blocks that drop when you eat a block) will have to be working here).

At that point, I should be able to have multiple players log into the same server, delete and change blocks and have them see eachother's updates.  That will be a major milestone because I will have collision detection for multiple players done, an efficient packet packing algorithm and a streaming server-side world that can save itself to the file system.

Next on the list is environment.  This will be game night/day sequence, gravity for any objects that register for it and water.  This one will be easy because it's pretty much already done, but nonetheless it's on the list to be reviewed and updated.

After that, structures.  I believe this will be one of the hardest to implement and that's why I've saved it for the very end.  Lots of objects check to see if the player is near them to activate.  I can do a foreach() on every logged-in player and check, but that seems horribly inefficient.  In order for this milestone to be complete, the player should be able craft an object (this will be validated client and server-side), place and remove it from the world.  The structures should all run their normal update() routines.

The next, and definitely hardest out of the bunch, will be the AI manager.  90% of the AI manager will have no problems and will be a seamless transition.  10% will be very complicated.  That last 10% is enemy movement and player collision detection.  There could be hundreds of enemies in the world and all of them need to check for player collision.  They also need to detect if they're near a player and if they are, go towards that player.  A foreach loop would also work here, but again - very inefficient.

So that's my multiplayer roadmap.  My goal is to have this done by the end of the month.  That's my goal because I'm going home to see my brother before he ships off to the Navy and I'd like to play multiplayer with him there before he goes.  I have a feeling that once I get started and figure out the fundamentals of how this server should be laid out, the rest is going to fly by quickly.

I still don't have a 100% solid idea of how I want the client to interpret the data.  I have an idea of how I want to efficiently build packets and how each player will have their own custom packet.  Sending it is already done.  How the client will interpret that packet and use it to update the world, though, is still a little foggy.

So anyway, that's my plan.  I feel like this is a whole new chapter in my game development saga.

- The First Chapter was just getting started.  Learning about Unity, 3d and the different engines/languages I can use.

- The Second Chapter was my Minecraft clone.  I learned about algorithms, heavy 3d triangle strips and efficient ways of rendering millions of cubes, player input and writing very efficient code.  Object pools and utilizing floats efficiently.  Using the Garbage Collector as little as possible.

- The Third Chapter was the beginnings of AstroMiner.  Creating a seamless 2d world.  Adding/deleting blocks.  Spritesheets.

- Fourth Chapter would be AI.  Implementing an effective, "thinking" enemy that can react to the player.  Making that enemy spawn in ways that make sense.

- Fifth Chapter would be crafting and structures.  Creating graphics that make sense.  Creating structures that take time to do their job.  Creating structures that pay attention to their surroundings and react accordingly.  Turrets that shoot enemy, doors that open and close and tractor beams that raise and lower the player.

- The most recent chapter, the Sixth, was all about fluids.  Creating an efficient fluid engine that let me have vast amounts of liquids and let them all update and react with the world around them without taking up tons of resources.

And now we're on the Seventh Chapter, multiplayer.  I've barely started this chapter and I've already learned a lot.  This might be one of the most challenging ones yet, mainly because I have so much of the fundamentals already done and there's so much that now needs to be changed.  I feel like I will definitely learn a lot though.  The nice thing is, at the end of the day, that's the point of this entire adventure.  To Learn.

So until next time my friends. ~

Learning 3d

First AstroMiner concept

First AstroMiner Build

Latest Screenshot

Lesson Learned

For my next game, the very first thing I will do will be multiplayer support.

The concept is simple.  When you launch in single player, the game is going to launch a local server and your client will connect to it.  You will be the only player on that server until you reach the multiplayer stage.  Then it will open up for others.

When you want to connect to another server, your client will simply point itself towards that server.

It sounds very simple and it would be, if I had worked like that from the beginning.  At this point of the game, I have so much code working with eachother that separating it is giving me a lot of issues.

I'll get it done - no doubt.  I just wish I would've thought of this from the beginning :]

MultiPlayer

That's a screenshot of the multiplayer server running.  It's using C#/Lidgren for networking.  Here's the first AstroMiner multiplayer screenshot:

There's a lot of work that needs to be done, but 2 players can log into the same server and watch eachother fly around.  This is just the tip of a massive iceberg though.

I need to make it so the worlds are always synced up.  The enemies need to be synced.  The time of day needs to be synced.  Structures.  Water.  Pretty much every single thing.  What I'm thinking of doing is having the game always launch with a local server that runs everything.  Even when you're in single player, you'll just be connecting to your own server and pulling information from it.  That way when it comes time for you to connect to another server, I just have to point the client somewhere else.

Anyway, Lidgren was a breeze to work with.  I had some minor issues at first but it was due to my own coding.  I'd definitely recommend it if you're thinking of adding multiplayer support to your own game.

I'm going to focus on that all week and probably for the rest of the month.  It's a lot of work, but I want to get it done before I do terraforming since that's going to be one of the most complicated parts of the game.