Название-Сеть в Играх 2006год_Язык-Eng
.pdf
154 Networking and Online Games: Understanding and Engineering Multiplayer Internet Games
Snapshot update messages
Local |
|
|
Local |
|
'thinking' |
Game |
Client |
||
every |
rendering |
|||
server |
|
|||
TICK |
|
|
||
|
|
|
Command update messages
Figure 10.1 Servers and clients ‘think’ and exchange information at potentially different rates
Figure 10.1 illustrates a useful conceptual model. A server typically ‘thinks’ in discrete time intervals (which we will refer to as ticks). A server must also transmit updates (or snapshots) of game state to every client on a regular basis. Similarly, the client is regularly making calculations (rendering game activity on screen) and sending updates (commands) to the server.
It is not essential that tick and snapshot rates are equivalent. Most FPS games provide a player-configurable mechanism for the client to request different snapshot rates that are better suited for their downstream network connection (towards the client). Clients can usually also set their command update rate (in packets per second) to suit their upstream network connection.
However, because an FPS server only does things at every tick:
•a client cannot request a snapshot rate faster than the tick rate.
•a client can request a snapshot rate slower than the tick rate, but the result is usually some multiple of the server’s tick rate.
An optimal snapshot rate is a trade-off between timeliness of game-state updates and network capacity towards the client. The product of snapshot rate and snapshot packet size is the message rate in bytes per second, which (naturally) must not exceed the available network capacity. If it does, even for brief instances, the client experiences additional jitter (and possibly packet loss).
Designers of online FPS games use various techniques to minimise the size of snapshot packets, such as:
•eliminating redundant or overly precise details in the state variables being sent to clients.
•only sending information about game-state changes within view of the client’s current player location.
•only sending the changes between one snapshot and the previous snapshot (sometimes referred to as delta compression).
Similarly, commands are kept very small in the client-to-server direction. They need to be sent frequently enough that a player’s actions are reflected into the virtual game
Online Game Traffic Patterns |
155 |
|
|
world accurately. They need to be small because upstream bandwidth is usually highly constrained in dial-up, DSL or cable-modem scenarios.
10.2.2 Controlling Snapshot and Command Rates
Let us take the example of Wolfenstein Enemy Territory (ET) [WET2005]. By default, ET servers tick 20 times per second (a tick every 50 ms) and their clients usually request 20 snapshots per second. If a client requests a higher snapshot rate, such as 30 or 40 per second, the server continues to send snapshots every tick – once every 50 ms. If a client requests 15 snapshots per second, the ET server rounds this up to the next multiple of tick intervals – 100 ms – which corresponds to 10 snapshots per second. The desired snapshot rate can be changed in-game using the client-side console command ‘snaps’. (This observation also applies to Quake III Arena [QUAKE3], the engine on which Wolfenstein ET was based.)
Half-life 2 (HL2) [HALFLIFE2004] behaves slightly differently. By default, the HL2 server ticks 66 times per second (once every 15 ms) [HL2NTWK]. As with ET, a HL2 client cannot request more snapshots per second than the HL2 server’s tick rate (in this case, 66). However, the situation is rather different when HL2 clients request less than 66 snapshots per second. Rather than simply rounding up to the next multiple of 15 ms, the server mixes transmission intervals to approximate the client’s requested snapshot rate over many packets.
For example, a request for 30 snapshots per second results in an uneven mix of packets sent at 30 ms and 45 ms intervals that averages out to 30 packets per second. A request for 50 snapshots per second results in an uneven mix of packets at 15 ms and 30 ms intervals, again averaging out to 50 packets per second. The snapshot rate can be changed in-game by modifying the client-side ‘cl updaterate’ variable.
Some games may also allow configurable server-side limits to override the client-side request. For example, setting HL2’s ‘sv minupdaterate’ server-side variable to 30 would impose a minimum rate of 30 snapshots per second on clients whose ‘cl updaterate’ was less than 30.
FPS games usually also allow capping of the downstream server-to-client traffic in bytes per second (not just packets per second). For example, HL2’s client-side ‘rate’ setting allows a client to specify the maximum rate at which it can accept traffic in bytes per second. It might seem redundant to have separate limits on rate (in bytes per second) and snapshots (in packets per second). However, the bytes per second limit protects your downstream link when the snapshot packets become large. If necessary, the snapshot packet/second rate will be temporarily reduced if the product of packet length and transmission frequency exceeds the bytes/second rate set by a client.
Command traffic in the client-to-server direction can also be constrained by the player. In Quake III Arena, the client-side variable ‘cl maxpackets’ specifies a cap on the number of command packets per second that the client will transmit to the server. This value dictates a lower bound on inter-packet arrival times from client to server.
In the following two sections, we will look in detail at the packet size distributions and inter-packet arrival distributions of some well-known FPS games.
156 Networking and Online Games: Understanding and Engineering Multiplayer Internet Games
10.3 Sub-second Packet-size Distributions
Not surprisingly, state information sent in server-to-client snapshots is far more varied and complex than the command updates sent from client to server. We will look at some examples from Quake II, Half-life, Quake III Arena, Halo 2, Wolfenstein ET and HL2.
Note that our focus here is on the ‘in-game’ distributions – packet sizes that are typically seen during the action parts of a game. The examples given here will not include packets seen during idle times and map changes (periods where interactive performance is less important).
Figure 10.2 and Figure 10.3 illustrate typical server-to-client and client-to-server packet-size distributions for Quake II over a Local Area Network (LAN). Figure 10.2 reveals that although individual playing styles can create minute differences in server-to- client packet-size distributions, the overall shape is fairly consistent. Figure 10.3 shows that client-to-server distributions are more clearly bounded regardless of playing style.
Figure 10.4 and Figure 10.5 illustrate similar characteristics for the original Halflife [HALFLIFE][LANG2003]. However, here we see an additional truth about server- to-client traffic – packet sizes depend on map type. In Figure 10.4, four players played on three different maps to show the impact on packet-size distribution. (This should not be surprising – map design influences the frequency with which players interact, and thus influences the amount of information carried by the average snapshot packet.) Over the same three maps (and with four players), Figure 10.5 reveals that client-to-server command packets are not influenced by the map type.
Server-to-client packet size distribution (Quake II)
2.5 |
|
|
|
|
|
|
Player 1 |
|
|
|
Player 2 |
2 |
|
|
|
1.5 |
|
|
|
% |
|
|
|
1 |
|
|
|
0.5 |
|
|
|
0 |
80 |
90 |
100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 |
60 70 |
|||
|
|
|
IP packet length (bytes) |
Figure 10.2 Quake II packet sizes: Server-to-client for two players during game play
Online Game Traffic Patterns |
157 |
|
|
Client-to-server packet size distribution (Quake II)
|
30 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Player 1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Player 2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Player 3 |
|
|
|
|
|
20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
% |
15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
0 |
56 |
57 |
58 |
59 |
60 |
61 |
62 |
63 |
64 |
65 |
66 |
67 |
68 |
69 |
70 |
71 |
72 |
73 |
74 |
75 |
|
55 |
||||||||||||||||||||
IP packet length (bytes)
Figure 10.3 Quake II packet sizes: Client-to-server for three players during game play
Server-to-client packet size distribution varies with map (Half-life)
|
3 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2.5 |
|
|
|
|
|
|
|
|
|
|
|
Xflight |
|
|
|
|
|
|
|
|
|
|
|
|
|
ChilDM |
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
Odyssey |
|
|
|
|
2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
% |
1.5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
0.5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
0 |
25 |
50 |
75 |
100 |
125 |
150 |
175 |
200 |
225 |
250 |
275 |
300 |
325 |
350 |
|
0 |
||||||||||||||
|
|
|
|
|
|
IP packet length (bytes) |
|
|
|
|
|
||||
Figure 10.4 Half-life packet sizes: Server-to-client over three different maps
158 Networking and Online Games: Understanding and Engineering Multiplayer Internet Games
Client-to-server packet size distribution per map (Half-life)
9 |
|
|
|
|
8 |
|
|
|
Xflight |
|
|
|
|
ChilDM |
7 |
|
|
|
Odyssey |
|
|
|
|
|
6 |
|
|
|
|
5 |
|
|
|
|
% |
|
|
|
|
4 |
|
|
|
|
3 |
|
|
|
|
2 |
|
|
|
|
1 |
|
|
|
|
0 |
60 |
70 |
80 |
90 |
50 |
||||
|
|
IP packet length (bytes) |
|
|
Figure 10.5 Half-life packet sizes: Client-to-server over three different maps
Quake III Arena shows similar client-to-server packet-size distribution curves to Quake II and Half-life in Figure 10.6 (two separate client curves are shown, essentially the same distributions). An additional truth about server-to-client packet-size distributions is illustrated in Figure 10.7 – the snapshots carry more information on average as the number of players goes up. (This example is taken from [LANG2004], where the same map was played multiple times with different numbers of players). Wolfenstein ET client-to-server traffic shows a similar packet-size distribution to that of Quake III Arena.
In general, snapshot size distributions are relatively constant over time for a given map type and number of players. Unfortunately, it is not immediately evident how to predict the likely packet-size distribution of a given map without previous measurements of actual game-play traffic.
A similar set of results emerge from the Xbox game Halo 2 [HALO2004]. Figure 10.8 and Figure 10.9 show the server-to-client and client-to-server packet-size distributions respectively, measured on a LAN when playing in ‘System Link’ mode [ZANDER2005a]. As expected, the server-to-client distributions increase and broaden out as the number of players increases. However, unlike the earlier PC-based games, a single ‘client’ (Xbox console) may have one to four players on it. This results in four possible client-to-server distributions – similarly shaped but with progressively larger packets to carry additional player action messages to the server. Halo 2’s client-to-server packets only appear in certain discrete sizes, always in multiples of 4 bytes and spaced 8 bytes apart. (The lines in Figure 10.9 visually associate data points rather than imply a continuity of packet sizes between data points.)
Figure 10.10 and Figure 10.11 show the server-to-client and client-to-server packet-size distributions seen with HL2. Since server-to-client size distribution generally varies with
Online Game Traffic Patterns |
159 |
|
|
Quake III Arena client-to-server packet size distribution
|
12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Client 1 |
|
|
|
|
|
|
|
|
Client 2 |
|
|
10 |
|
|
|
|
|
|
|
|
8 |
|
|
|
|
|
|
|
% |
6 |
|
|
|
|
|
|
|
|
4 |
|
|
|
|
|
|
|
|
2 |
|
|
|
|
|
|
|
|
0 |
65 |
70 |
75 |
80 |
85 |
90 |
95 |
|
60 |
|||||||
|
|
|
|
IP packet size (bytes) |
|
|
|
|
Figure 10.6 Quake III Arena client-to-server packet-size distribution
Server-to-client packet size distribution versus number of players (Quake III Arena)
|
3 |
|
|
|
|
|
|
|
|
|
|
|
|
|
2.5 |
|
|
|
|
|
|
|
2 players |
|
|
|
|
|
|
|
|
|
|
|
|
|
4 players |
|
|
|
|
|
|
|
|
|
|
|
|
|
5 players |
|
|
|
|
|
|
|
|
|
|
|
|
|
6 players |
|
|
|
|
|
2 |
|
|
|
|
|
|
|
8 players |
|
|
|
|
% |
1.5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
0.5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
0 |
|
|
|
|
|
|
|
|
|
|
|
|
|
50 |
75 |
100 |
125 |
150 |
175 |
200 |
225 |
250 |
275 |
300 |
325 |
350 |
IP packet length (bytes)
Figure 10.7 Quake III Arena’s server-to-client packet distributions versus number of players
160 Networking and Online Games: Understanding and Engineering Multiplayer Internet Games
Halo 2 server-to-client packet size distribution versus number of players in game
22 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
20 |
|
|
|
|
|
|
|
|
|
|
|
3 players |
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
5 players |
|
|
||
18 |
|
|
|
|
|
|
|
|
|
|
|
7 players |
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
9 players |
|
|
||
16 |
|
|
|
|
|
|
|
|
|
|
|
11 players |
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
12 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
% |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
8 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
0 |
80 |
100 |
120 |
140 |
160 |
180 |
200 220 |
240 |
260 |
280 |
300 |
320 |
340 |
360 |
380 |
400 |
60 |
||||||||||||||||
Packet size (bytes)
Figure 10.8 Halo 2 server-to-client packet-size distribution (IP packets over LAN)
Halo 2 client-to-server packet size distribution versus number of players on client
|
60 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 player |
|
|
|
50 |
|
|
|
|
|
|
|
|
2 players |
|
|
|
|
|
|
|
|
|
|
|
3 players |
|
||
|
|
|
|
|
|
|
|
|
|
4 players |
|
|
|
40 |
|
|
|
|
|
|
|
|
|
|
|
% |
30 |
|
|
|
|
|
|
|
|
|
|
|
|
20 |
|
|
|
|
|
|
|
|
|
|
|
|
10 |
|
|
|
|
|
|
|
|
|
|
|
|
0 |
60 |
70 |
80 |
90 |
100 |
110 |
120 |
130 |
140 |
150 |
160 |
|
50 |
|||||||||||
Packet size (bytes)
Figure 10.9 Halo 2 client-to-server packet-size distribution (IP packets over LAN)
Online Game Traffic Patterns |
161 |
|
|
Packet length distributions: Half-life 2 server snapshots
0.7 |
|
|
|
|
|
|
|
|
0.6 |
|
|
|
|
|
Client 1 |
|
|
|
|
|
|
|
|
Client 2 |
|
|
0.5 |
|
|
|
|
|
|
|
|
0.4 |
|
|
|
|
|
|
|
|
% |
|
|
|
|
|
|
|
|
0.3 |
|
|
|
|
|
|
|
|
0.2 |
|
|
|
|
|
|
|
|
0.1 |
|
|
|
|
|
|
|
|
0 |
100 |
200 |
300 |
400 |
500 |
600 |
700 |
800 |
0 |
||||||||
IP packet length (bytes)
Figure 10.10 Half-life 2 server-to-client packet-size distributions of two clients on the same server
Packet length distributions: Half-life 2 client commands to server
|
8 |
|
|
|
|
|
|
|
|
|
|
7 |
|
|
|
|
|
|
|
Client 1 |
|
|
|
|
|
|
|
|
|
|
Client 2 |
|
|
6 |
|
|
|
|
|
|
|
|
|
|
5 |
|
|
|
|
|
|
|
|
|
% |
4 |
|
|
|
|
|
|
|
|
|
|
3 |
|
|
|
|
|
|
|
|
|
|
2 |
|
|
|
|
|
|
|
|
|
|
1 |
|
|
|
|
|
|
|
|
|
|
0 |
60 |
70 |
80 |
90 |
100 |
110 |
120 |
130 |
140 |
|
50 |
IP packet length (bytes)
Figure 10.11 Half-life 2 client-to-server packet-size distributions of two clients on the same server
162 Networking and Online Games: Understanding and Engineering Multiplayer Internet Games
the number of players and map layout, Figure 10.10 should be treated as indicative of what to expect, rather than setting specific boundaries on possible packet sizes.
10.4 Sub-Second Inter-Packet Arrival Times
Inter-packet arrival times reflect the actual, packet-by-packet burstiness of traffic between a game server and its clients. From server to client, the distribution of inter-packet intervals depends on the server’s snapshot update algorithm. From client to server, the patterns are more complex, being the result of multiple unsynchronised clients sending player action messages continuously (but somewhat irregularly) to the server.
Once every tick, a game server transmits a back-to-back burst of one or more snapshot packets towards its clients (Figure 10.12). The precise number of packets sent per each tick depends on the snapshot update strategy employed by the server.
Burstiness is highest at the server, gradually dropping as packets branch out along network paths towards each client (Figure 10.13). Inter-packet interval histograms of the aggregate snapshot traffic at a server will show one peak at some multiple of the server’s tick interval and another peak down under 1 ms (from snapshots sent back to back during the same tick interval).
Time t 1 |
Time t 2 |
||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Packet 1 |
|
Packet 2 |
|
Packet 3 |
|
Packet 1 |
|
Packet 2 |
|
Packet 3 |
|
|
|
Snapshot interval |
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
Figure 10.12 Servers transmit one or more back-to-back packets every tick to meet snapshot rate limits
|
|
|
|
|
|
Client 3 |
|
P1 |
P2 P3 |
P1 |
P2 P3 |
|
|
P3 |
P3 |
|
|
|
|
||||
Snapshot interval |
|
P1 |
P3 |
P1 |
P3 |
|
|
|
|
|
|
|
|
||
|
|
|
Router 1 |
|
|
Router 2 |
|
Game |
P2 |
|
P2 |
|
P1 |
P1 |
|
server |
|
|
|
||||
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
Client 2 |
|
|
Client 1 |
|
|
||
|
|
|
|
|
|
||
Figure 10.13 Updates become less bursty closer to clients
Online Game Traffic Patterns |
|
163 |
||
|
|
|
||
|
Table 10.1 Default FPS snapshot and tick rates |
|||
|
|
|
|
|
|
Game type |
Snapshot |
Tick rate |
|
|
|
rate |
(per second) |
|
|
|
(per second) |
|
|
|
|
|
|
|
|
Quake II |
10 |
? |
|
|
Half-life |
16.67 |
? |
|
|
Quake III Arena |
20 |
20 |
|
|
Wolfenstein |
20 |
20 |
|
|
Enemy Territory |
|
|
|
|
Half-life 2 |
20 |
66 |
|
|
Halo 2 |
25 |
? |
|
|
|
|
|
|
For a game such as Quake III Arena or Wolfenstein ET, if there are n players on your server, all requesting 20 snapshots per second, then (100 × (n − 1)/n) percent of the measured inter-packet intervals will be in the sub-millisecond region. The same applies to HL2 if the clients have all requested 33 or 66 snapshots per second. The inter-packet interval distribution becomes more complex to predict when concurrently connected HL2 clients request different snapshot rates.
Table 10.1 shows the default snapshot rates for a number of well-known FPS games along with the server’s tick rate (where known).
Snapshot intervals do not generally vary with map choice or number of players. However, game server tick timing can fluctuate by a few milliseconds due to central processing unit (CPU) load, resulting in observable jitter of snapshot transmissions.
Client-to-server intervals of the PC-based FPS games are more difficult to characterise.
•Client transmissions towards the server are uncorrelated (so inter-packet intervals of the aggregate traffic close to the server are spread widely).
•Many games have user-configurable rates (and thus minimum inter-packet interval) at which they send updates from client to server.
•Actual distributions seem to depend strongly on each player’s game-play behaviour, the client settings and their system’s technical capabilities (e.g. CPU speed and graphics card hardware acceleration) [LANG2003, LANG2004].
Most published empirical |
results suggest intervals clustered |
between 10 ms and |
30–50 ms. (Although this is |
not always so – Halo 2’s XBox |
System Link clients |
transmit their updates at regular 40 ms intervals, same as the server-to-client inter-packet interval.) [ZANDER2005a]
One final note. When interpreting inter-packet interval statistics, be aware that you may sometimes find multiple clients hiding behind a single IP address. Two or more people playing on your Internet-based server from their home LAN will usually be connecting through a home ‘gateway’ implementing Network Address Translation (NAT). Their user datagram protocol (UDP) packets will have the same home IP address. It is important to always use both the port number and IP address to differentiate remote clients. (You may also be able to confirm multiple distinct client machines behind the NAT box by the unique ‘IP ID’ field patterns in packets from each client [BELLOVIN2002].)
