Why does Argus use sharding to build the infrastructure for blockchain games?
Why use sharding for blockchain games infrastructure in Argus?
Original Title: How I Learned to Stop Worrying & Love Execution Sharding
Video Link: https://www.youtube.com/watch?v=A0OXif6r-Qk
Speaker: Scott Sunarto (@smsunarto) on Research Day
Article Editor: Justin Zhao (@hiCaptainZ)
Hello everyone, I’m Scott (@smsunarto), the founder of Augus Labs (@ArgusLabs_). Today, I want to discuss a topic that we haven’t touched on for a while. With roll-ups becoming the mainstream of the times, our discussion of execution sharding is not as frequent as data sharding. So, let’s take another look at this somewhat overlooked topic – execution sharding.
- Encrypted ATM operator Bitcoin Depot goes public on NASDAQ
- PEOPLE On-chain Transaction Network Analysis and Valuation
- Conversation with Electric Capital Partner Ken Deeter: Investment P...
This will be a casual conversation. I know you’ve heard complex concepts all day, so I’ll try to make this discussion as practical as possible. I have prepared appropriate slide designs for this presentation.
For those who don’t know me, interestingly enough, I’m known as an anime girl on Twitter. I also missed my college graduation ceremony to come here, which made my parents very unhappy. Currently, I’m the founder of Argus Labs. We consider ourselves a gaming company, not an infrastructure or cryptocurrency company. One of my biggest frustrations is that everyone in the crypto gaming space wants to build tools, but no one wants to create content or applications. We need more applications that users can actually use.
Previously, my clever friends Brian Gu (@gubsheep) and Alan Luo (@alanluo_0) and I created Dark Forest (@darkforest_eth). Brian is now running 0xBlockingRC (@0xBlockingRC), he’s a lot smarter than me.
Today’s discussion will focus on execution sharding, but in a context where most people are not familiar with execution sharding. We usually discuss execution sharding in the context of Layer1, such as Ethereum sharding or Near sharding. But today, I want to change the context. Let’s think about what sharding would look like in a roll-up environment.
The basic question here is, why would a gaming company want to build its own roll-up, and what can we learn from World of Warcraft to design roll-ups? Additionally, we will explore how the design space of roll-ups far exceeds current reality.
To answer these questions, let’s go back to 2020 when the idea for Dark Forest was first conceived. We asked ourselves, what if we created a game where every game action is an on-chain transaction? At the time, this premise was absurd, and it still is for many people today. But it was an interesting hypothesis, so we built it, and Dark Forest was born.
Dark Forest is a completely Ethereum-based MMORTS game that is powered by ZK-Snarks. Back in 2020, ZK was not as popular as it is today because there were hardly any documents available. The only available documentation for Circom was Jordi Baylina’s (@jbaylina) Google Docs. Despite the challenges, we learned a lot from the process, and Dark Forest is a testament to that learning.
Dark Forest is a bigger experiment than we imagined. We have over 10,000 players who have spent trillions of gas, and the game is filled with chaos, with people stabbing each other in the back on the chain. What’s most attractive about Dark Forest and on-chain games is their platform nature. By having a full-chain game, you open the door to designing new behaviors, allowing people to build smart contracts that interact with the game, as well as alternative clients and game modes, such as Dark Forest Arena and GPU miners.
However, with great power comes great responsibility. When we launched Dark Forest on what is now known as the Gnosis Chain on xDai, we eventually filled the entire block space of the chain. This made the chain basically unusable for anything else, including DeFi, NFTs, or anything else on xDAI.
So now what? Are we at a dead end? Will full-chain games never become a reality? Or do we go back to making games with only small JPEG images on the chain and convince people that money grows on trees? The answer is, we make the software do the job. Many of us have a very rigid view of blockchains and roll-ups, as if there is not much room for improvement. But I disagree. We can experiment and find new possibilities.
We ask ourselves a question: What would a blockchain look like if we were to design it from scratch for games and games only? We need high throughput, so we need to scale reads and writes. Most blockchains are designed for heavy write usage. Transactions per second (TPS) is a metric people brag about, but in reality, reads are just as important. If you can’t read from the blockchain node, how do you know where the players are? This was actually the first bottleneck we discovered in our blockchain construction.
Dark Forest had a problem where full nodes were heavily used, causing I/O explosions, as we needed to read data from the state on the chain. This resulted in thousands of dollars in server costs, which were generously covered by the xDAI team. However, this is not ideal in the long run. We need high throughput, not just for writes per second, but also for reads, such as getting data from the blockchain itself.
We need a horizontally scalable blockchain to avoid the Noisy Neighbor problem. We don’t want a popular game to suddenly start having issues on the blockchain and stop all work. We also need flexibility and customizability so that we can modify the state machine to suit game design. This includes having a game loop so that it can execute itself, among other things.
Last but not least, and perhaps most importantly for those unfamiliar with online game architecture, we need a high tick rate. Ticks are the time atomic units in the game world. In the context of a blockchain, we have blocks as the time atomic units. In the game, we have ticks. This is almost similar when you build a full-chain game, where the tick of your blockchain or the speed of block generation equals the tick of the game itself.
So what we need is a blockchain that is high throughput, horizontally scalable, flexible and customizable, and with a high tick rate. Only such a design can meet the requirements of the blockchain designed from scratch for the game.
If you have a higher tick rate or more blocks per second, the game will feel more responsive. Conversely, if your tick rate is lower, the game will feel more sluggish. One key thing to remember is that if block generation is delayed, you will feel noticeable lag in the game. This is a bad experience. If you’ve ever dealt with angry players yelling at their computer because they lost the game, you know it’s an absolutely terrible situation.
Currently, our rollups have one block per second, which is equivalent to one tick. If we want to have cooler games, we need a higher tick rate. For example, Minecraft, a simple pixel art game, has 26 ticks per second. We still have a long way to go to build games that are as responsive as Minecraft.
One possible solution is to deploy our own rollup. Although it seems to solve the problem on the surface, it does not actually solve the root cause of the problem. For example, you will have higher write throughput, but not entirely at the level the game needs. Of course, if your game has a hundred players, this will be enough. But if you want to build games that require higher throughput, there will be very strict limitations due to the current I/O in the build.
There are no real performance gains in reading. You still need to rely on the indexer. You don’t really have horizontal scalability. If you try to launch a new rollup to horizontally scale your game, you will break your existing smart contract ecosystem. The market deployed by players will not work with other chains you launch for horizontal scaling of the game. This will cause many problems.
Finally, high tick rates and blocks per second are still a bit challenging, although we can try our best to push it, we may get two or three blocks per second, but this is really the farthest these blockchains can go because there are a lot of things like re-marshalling that are very dependent on compute cycles.
To solve this problem, we looked back to the early 21st century and late 1990s when online games like MMOs were just emerging. They had a concept called sharding. This is not a new concept; it has existed in the past. The word “sharding” we use in database architecture actually comes from a reference to Ultima Online. They were the first to use the word “sharding” to explain their different servers.
So how does sharding work in games? It’s not a one-size-fits-all solution. It’s a tool in the toolbox and how you adapt it to your game depends on the specifics. For example, the first sharding construct is one I like to call location-based sharding. A good mental model is to imagine a Cartesian coordinate system being divided into four quadrants, each with its own game shard. Every time you want to cross a shard, you send a communication to another shard that says “Hey, I want to move over there,” and you’re teleported to your shard, leaving behind your previous player body. By doing this, you distribute the workload of the server to multiple physical instances instead of forcing one server to do all the computation for the entire game world. The second construct is more popular now. It’s called multi-universe sharding, where you have multiple game instances that mirror each other. You can choose any shard you want to go to, and it defaults to load-balancing so that no server gets too crowded.
Now, the key question is, how do we bring this concept into the rollup? This is why we created the World Engine. The World Engine is our flagship infrastructure, essentially a opinionated sharding sorter designed for launch. Our design is different from many sharding sorters we’ve seen in previous discussions, and more suited to our needs. The direction we optimized for is: A, throughput, B, we want to make sure there are no locks blocking runtime to ensure tick rate and block time are as efficient as possible, so it defaults to synchronous, and we designed the sorter to be partially sorted, not forcing total sorting (where each transaction needs to occur after another).
The key components here are that we have two main things. We have an EVM-based shard, which is like a pure EVM chain where players can deploy smart contracts on it, combine with games, create markets with taxes, and so on. It’s like a normal chain, right? Like one block or one thing per second, just enough for you to do all your typical device and market stuff.
The secret ingredient here is that we also use a game shard, which is essentially a mini blockchain designed as a high-performance game server. We have a bring-your-own-implementation interface so you can customize this shard to your liking. You can build your own shard and inject it into the base shard. You just need to implement a standard interface, like the ABC interface in Cosmos that you are familiar with. You can basically integrate this into a similar specification and bring your own shard into the World Engine stack.
The key here is that we have a high tick rate, which we can’t currently achieve with the current shards. This is where I want to introduce Cardinal. Cardinal is the first game shard implementation of World Engine. It uses the Entity-Component-System (ECS) with a data-oriented architecture. This allows us to parallelize the game and increase the throughput of game calculations. It has a configurable tick rate of up to 20 ticks per second. For blockchain people here, that’s 20 blocks per second.
We can also geolocate it to reduce latency. For example, you may have a sorter in the United States, and then Asians have to wait for a 300 millisecond delay for the transaction to reach the sorter. This is a huge problem in games because 300 milliseconds is a long time. If you’re trying to play an FPS game with a 200 millisecond delay, you’re basically dead.
Another key point that is also important to us is that it is self-indexing. We no longer need external indexers. We don’t need these frameworks to cache game states. This also allows us to build more real-time games without delay issues because the indexer is still trying to catch up with the sorter block.
We also have a plugin system that allows people to parallelize ZK verification, etc. The best part, at least for me, is that you can write your code in Go. No more need to use Solidity to make your game work. If you’ve ever tried to build a blockchain game with Solidity, it’s a nightmare.
In summary, I would like to be candid here that many people hope their constructions can solve all problems, but we don’t. We believe our construction is useful for us, but it may not be suitable for your use case. It is unrealistic to assume that our construction can be suitable for everyone. For us, it meets our needs by providing high throughput, horizontal scalability, flexibility, and high tick rate, but it cannot cure cancer. If you need a DeFi protocol that requires synchronous composability, then this type of construction may not be suitable for you.
Overall, I sincerely believe in the concept of human-centric blockchain architecture. By designing around specific user roles and use cases, you can make better trade-offs instead of trying to solve everyone’s problems. The Renaissance is here, and everyone can design their own Roll-Ups to suit their specific needs instead of relying on generic solutions. I think we should embrace the Cambrian explosion. Don’t build roll-ups like one-size-fits-all layer one because it wasn’t built to solve the same problems at all. I personally look forward to seeing more people exploring more Roll-Up design spaces that are use-case-specific. What would a Roll-Up designed specifically for asset exchange look like? Would it be intent-based? What would a Roll-Up designed specifically for on-chain CLOBs (central limit order books) look like? Here, I’ll hand the mic over to MJ. Thank you for the invitation.
English Version:
https://captainz.xlog.app/Why-does-Argus-Build-FOC-Gaming-INFRA-Using-Sharding
