[AntiCheat] Game Security: Everything About Lag

@codewiz · July 06, 2012 · 10 min read

#0

If I had to pick the most sensitive issue in a successful game, it would be lag. It's one of the most disliked words among all gamers, and it's one of the biggest reasons for upgrading PCs. So, what exactly is this lag? As the word implies, it represents delay. We refer to all the delays occurring in the game as lag.

When it comes to the hottest game these days, Diablo 3 cannot be left out. Initially, due to congestion in the Asian servers, I cleared the normal mode in the North American servers and experienced the essence of Blizzard's craftsmanship. After watching the ending, I spent over 5 minutes watching the credits, something I had never done before. It was that impressive, and I was that curious about who made it. Funnily enough, I closed the credits after over 5 minutes because they weren't ending. A colleague of mine closed it after 15 minutes for the same reason. For reference, it takes about 18 minutes to watch all the ending credits. Classic Blizzard scale, haha~

When gamers encounter a screen like this due to lag, they feel their lid blow off.

When gamers encounter a screen like this due to lag, they feel their lid blow off.

Even such a blockbuster game could not escape lag issues. In fact, the lag in Diablo 3 is also on a Blizzard scale. This mainly refers to what we call server lag. Situations like using a skill and it activating 2 seconds later, or dying due to lag despite taking a healing potion at the right time after watching my health bar, or gold being picked up 2 seconds after I walk over it. Of course, this does not mean there is no client lag. In environments where the specifications are not sufficiently good, the game screen often freezes in situations where mobs suddenly appear en masse or where physical situations overlap. Not to mention the auction house lag. Such lag really halves the fun of the game by a factor of ten million. In fact, after being knocked down by champion mobs about three times at home due to such reasons, I lost the taste for it and am writing this article. Haha~

#1

By this point, you should have a sufficient understanding of what lag is. Next, let's look at the causes of such lag. Where does this damned lag come from, exactly?

The most common cause is literally due to network delay. Online games mostly proceed based on network connectivity. Naturally, there's a server/client structure, and there exists communication speed between them. Gamers commonly refer to this as ping. When this ping exceeds a certain level, it directly indicates that delay is occurring. If the delay speed exceeds a certain threshold, discrepancies occur. Situations like an attack motion being displayed on the game client, but the actual damage not being applied to the target, happen. If this gets worse, you might even start feeling nauseous unknowingly.

The quality of the network in such online games is not only affected by one's own internet connection. In games where gamers form a network with each other using the P2P method, one's experience is also affected by other gamers' network speeds. Just as a completely different world unfolds at the speed of light, when the ping gets high enough, new problems that the developers never predicted can occur. Some gamers exploit these bugs by using methods like tapping - disconnecting and reconnecting the LAN cable, nuking - sending infinite packets to a specific address, or lag hacking - deliberately inducing lag on their PC, to gain additional advantages.

Another factor causing lag exists in the game servers. The recent trend in online games is that the client serves merely as a viewer, and the game server models all the game states. Why is this? Obviously, to prevent hacking. In Diablo 3, for instance, you might be able to change the displayed value of gold with a simple manipulation, but you can't use that displayed data in the auction house. The actual data is all stored on the server and calculations are based on that data. As the server becomes more complex, any abnormalities in the server are directly transmitted as lag. If the game server I'm in has an abnormal state, causing a delay in modeling processing speed, I would directly feel all of that impact. Recently, Blizzard announced that they would prevent the frequent creation of rooms in short intervals in Diablo 3. Creating rooms frequently can cause abnormalities in the server, which in turn can harm other gamers connected to that server. Considering these points, there's nothing a gamer can do about this kind of lag. All we can do is suck our thumbs and wait for the game developer to stabilize the servers. It's the same principle as the current hopeless state of lag in Diablo 3.

The last reason causing lag, as all gamers know, lies in computer specifications. If the computer specifications are not fast enough to run the game, severe lag inevitably occurs. In this case, there's no solution other than an upgrade. Of course, one can gain some usability by lowering the graphic options and resolution of the game, but to truly enjoy a comfortable gaming experience, upgrading the computer is the only answer.

#2

Then, why do game companies leave such lag, which halves the fun of games, incurs the wrath of users, decreases concurrent connections, and hinders the generation of game revenue, unaddressed? Why do all gamers in every game complain about lag? If the cause is clear, why not just fix it? But solving this problem is not as easy as it sounds.

First, the issue of network lines. There's nothing a game company can do about this. In our country, the network lines are in very good condition, but in many places abroad, they are not. There are really bizarre places. Places with limited upload/download traffic, places where it's more like an intranet than the internet, and even places where updates patches cannot be properly received, requiring manual patching. It's not always great domestically either, with environments using multiple routers or where the lines themselves are faulty, or where the ping is unstable due to malware. There are various situations. However, gamers usually don't have enough technical knowledge to understand their internet conditions. They just say the game isn't working or the ping is spiking.

Next, server lag. This is an issue that game developers must resolve. It's also a problem that has a fairly clear cause. However, it's really not easy. This is because 99.98% of server lag issues occur when users flock in droves and play in ways the developers never predicted. It's difficult to resolve because the problems only occur in a live environment. Additionally, today's game servers are not just a single server, but are interwoven with various servers in an unimaginably complex structure. So, when lag occurs, it's as hard as finding a needle in a haystack to determine what part is causing the delay. The last reason this problem is taken to Andromeda is that once a game is commercialized, the things a game developer can change are extremely limited. In most cases, unless it's a critical issue, server developers can't do much about it. That's why they do CBTs and OBTs, but unfortunately, these days those are also very short. This seems to be why these issues are more highlighted.

Finally, let's look at client lag. Basically, once the size of client resources is set, the developer's ability to do something about it is greatly limited. Today's games are so large in scale and use so many external libraries. The era of optimizing with the developer's passion is a bit past. In fact, the reason users curse about client lag lies elsewhere – in the minimum and recommended specifications. Most game companies often announce the recommended specs as the minimum. Therefore, gamers think those are the real recommended specs and whine when they experience lag even though their PCs are better than the recommended specs. In reality, in most cases, it's better for your mental health to consider the recommended specs as the actual minimum.

Diablo 3 recommended specs. It's frustrating.<br><br>Our home PC is much better than that, but I still have to set all options to minimum to somewhat mitigate the lag.

Diablo 3 recommended specs. It's frustrating.

Our home PC is much better than that, but I still have to set all options to minimum to somewhat mitigate the lag.

Another reason for client lag lies in third-party products. External security products like XIGNCODE are among these. This is one of the reasons why developing game security products is difficult. Anyway, these external security products also have a cost to operate, so they affect lag and have a much bigger impact depending on the specifications. Unfortunately, there's practically nothing a game client developer can do about this. It's something for third-party product developers to handle.

Monitoring game forums, you can see many users criticizing the game developer's capabilities, talking about poor optimization. To add a few words with a heavy heart, most developers write code of almost similar performance. The performance difference between code written by a reasonably talented developer A and an average developer B is not that significant. Most dramatic performance improvements are created by changing the way we view the problem, rather than the developers' hard-working coding. And changing this perspective is often done by computer scientists rather than ordinary programmers. So let's not be too harsh on the developers, accusing them of poor optimization. They are always trying their best, writing the fastest code they know how.

#3

Up until now, I've mainly talked about how difficult it is to handle lag, but the real reason why solving this issue is as challenging as discovering the Higgs boson lies elsewhere. It's because almost all game developers, operators, QA staff, third-party product developers, and their QA teams don't understand lag. Most of these people do not comprehend the lag that gamers talk about. Why is that? Because they have different standards for perceiving time. Gamers are much more sensitive than we can imagine.

There was a time when a Blizzard developer, after coming to Korea, saw player Lim Yo-Hwan using Marines to kill Lurkers in Starcraft and said he never imagined such gameplay. It makes sense, as Lurkers are supposed to be strong against Marines. A player like Lim Yo-Hwan would feel whether Lurker tentacles are approaching faster or slower than usual, or how active they become after using Stimpacks. This is just an assumption, so let's not nitpick about it, haha~ Even among regular gamers, those who play well are extremely sensitive to such micro aspects. In FPS games, it’s about the speed of changing guns, bullet firing, zooming in; in MMO games, it’s about the speed of certain skills activating, or the speed at which their constructed structures become active. These are aspects that someone who isn’t good at the game wouldn’t notice. This is the difficult part because you can’t perceive these speed differences unless you play that well.

This is why QA teams often declare there's no lag issue after countless lag tests. It's because there's no player in the QA team who can perceive the lag a real user talks about. From the QA team's perspective, the speed of changing guns is sufficiently fast. It's a regrettable reality. To compensate for these aspects, third-party game-related companies like ours always ask during recruitment, “Do you play games well? Do you have a favorite online game?” In fact, we have a QA employee who is quite good at FPS games, and this employee demonstrates far superior capabilities in detecting lag and hacks in those games compared to other employees. If game companies or third-parties are serious about addressing the issue of lag, they need to hire gamers skilled enough to understand lag. Otherwise, testing repeatedly will be futile.

#4

When I was writing this article, Blade & Soul hadn't been released yet. Recently, Blade & Soul had its OBT and not long ago, it started its paid service. But even such a hot game like Blade & Soul couldn't escape lag issues. Whether you look at the forums or listen to the players, lag is a constant topic. Maybe lag is like a winner’s curse that comes with a successful game since it occurs where the crowd gathers.

Character: Super cool Blade & Soul~ Wow~

Character: Super cool Blade & Soul~ Wow~

@codewiz
Looking back, there were good days and bad days. I record all of my little everyday experiences and learnings here. Everything written here is from my personal perspective and opinion, and it has absolutely nothing to do with the organization I am a part of.
(C) 2001 YoungJin Shin, 0일째 운영 중