Multiclient Servers

We have now seen how a TCP server is initialized and performs data transfer with a single client. Although this suffices for a very simple, two-player game, real-world game servers need to cope with more complex scenarios. At least eight players should be supported, and many massively multiplayer games can handle thousands in parallel. But our server can only carry out one task at a time. If, after accept(), we decide to take care of the client, we will lose the ability to handle any more incoming connections.

Clearly, we need a way of keeping an eye on the incoming connection queue while we perform data transfers with the already connected players. At least two ways exist to perform this task. We could take a concurrent approach and create several processes running in parallel from our core server, each one taking care of different tasks. Under this assumption, we would have N+1 processes (where N is the number of connected users). N processes would handle one connected socket each while the extra process would be the one running the server, and thus permanently waiting for new connections at an accept() call. Alternatively, we could take an iterative approach, and by making sure we don't get blocked at any connected socket, check all the communication endpoints into a loop. The following sections describe these two approaches.

Concurrent, Connection-Oriented Servers

In this section, we will focus on concurrent (thus, multiprocess) TCP servers. The strategy we will use is to spawn a child process for each accepted connection, so the parent can always keep waiting for new connections.

For those not familiar with multiprocess programming, it usually revolves around the fork() UNIX system call. This is a parameterless call that, upon execution, duplicates the execution environment, so there are two copies of the program in execution. Fork() returns an integer that has the peculiarity of being different for the two newly created processes. The child process will receive a 0, whereas the parent will receive a nonnegative integer that is the process identifier of the child process. Fork() is available only in UNIX and Linux operating systems, which are the operating systems on which many servers are built. For Windows-based servers, a similar technique would use the CreateThread call.

As a sample of the fork() call's behavior, take a look at the apparently innocent code that follows:

#include <stdio.h>

void main()
{
printf("I am a simple program\n");
int id=fork();
printf("My id is: %d\n",id);
}

This program produces the following output:

I am a simple program
My id is: 0
My id is: 37

Intrigued? Well, remember that fork() divides the program in two. So, the first printf() is executed once, and the second printf() is executed by the two programs, producing a different output in each one. Now that we know how the fork() call operates, the following algorithm should be easy to understand:

Master 1: Create a socket, and bind it to the IP and port

Master 2: Put it in passive mode with listen

Master 3: Wait for a new connection with accept

Master 4: When a new connection arrives, fork, creating Slave

Master 5: Go to step 3

1. Slave 1: Enter a send-recv loop with the new connection

2. Slave 2: Once the connection terminates, exit

So we have a master process that is permanently blocked in the accept call, spawning child processes to handle the connected users. You can see this method in action in Figure 10.2. We need to make sure the number of processes remains in control (for example, limited to the number of players), but other than that, the algorithm is relatively straightforward. Here is the commented source code for the preceding example:

// create a TCP socket
int sock= socket(AF_INET, SOCK_STREAM,IPPROTO_TCP);

// fill the address info to accept connections on any IP on port "port"
struct sockaddr_in adr;

adr.sin_family=AF_INET;
adr.sin_port = htons(port);
adr.sin_addr.s_addr=INADDR_ANY;
ZeroMemory(adr.sin_zero,8);

// bind socket to address
bind(sock, (struct sockaddr *) &adr, sizeof(adr));

// put the socket in passive mode, and reserve 2 additional
connection slots
listen(sock,2);

// loop infinitely for new connections
while (1)
   {
   struct sockaddr_in connectionaddr;
   int caddrsize;
   // we have a new connection
   int newsock=accept(sock,&connectionaddr, &caddrsize);

   // spawn a child process
   switch (fork())
      {
      case 0:      // child, I handle the connection
         communicate(newsock,connectionaddr);
         exit(0);
      case –1:
         // error in fork
         exit(-1);
      default:
         close (newsock);
         break;
      }
   }

Some error handling code as well as the communicate() function have been eliminated for the sake of clarity. Notice how the parent must close the newsock socket descriptor because fork duplicates variables, which means we have an open descriptor that we are not going to use. If we do not close sockets handled by the child process, we might run out of descriptors.

Iterative, Connection-Oriented Servers

Now, imagine that we want to implement the same behavior without using the fork() call, so the whole server is run in a single process. This is easy for just two peers: Put the server in accept mode and wait until a new connection arrives. But how can we do this with many users? As soon as the first user is accepted and we are exchanging messages with him, we can no longer execute an accept call because it would stop the whole server while we wait for a new connection that can or cannot take place. Thus, we need a way to keep a socket open for incoming connection requests while we continue working with already connected clients. This is achieved with the select call, which allows us to check many sockets at once. The syntax follows:

int select(int nfds, fd_set *read, fd_set *write, fd_set *except, struct timeval 
*timeout);

The call returns the number of sockets for which activity was detected. Then, the parameters are used to pass the sockets we want to check. To do so, the socket's API provides a structure called fd_set, which represents a set of sockets. The name fd_set, incidentally, comes from "file descriptor set," because in UNIX systems (where sockets were first devised), a socket is similar to a regular file descriptor. So, we can add sockets to an fd_set, delete them, and check for activity on such a set with the select call. Thus, the following three parameters for select are a set with those sockets we want to read from, a second set with sockets to write to, and a third, generally unused set with exceptions. The last parameter is just a timeout so select does not wait forever. Because the syntax is pretty obscure, here is a full select() example:

int msgsock;
char buf[1024];
fd_set ready;
int sock;
int maxsocks;

listen(sock, 5);        // 5 positions in the queue
FD_ZERO(&ready);
FD_SET(sock, &ready);
maxsocks=sock;
while (1)               // server loops forever
   {
   struct timeval to;
   to.tv_sec = 5;
   to.tv_usec = 0;
   select(maxsocks, &ready, 0, 0, &to);

         if (FD_ISSET(sock, &ready))
      {
      msgsock = accept(sock, (struct sockaddr *)0, (int *)0);
      bzero(buf, sizeof(buf));
      read(msgsock, buf, 1024);
      printf("-->%s\n", buf);
      close(msgsock);
      maxsocks++;
      }
   }

I'll comment on the code a bit. Basically, we want to check those sockets we have already been connected through while keeping an eye on the initial socket, so we detect new connections. To do so, after listen(), we initialize an empty set of socket descriptors. We clear the array with FD_ZERO, which is just a macro. Then, we initialize the entry occupied by our own socket to specify that it's the only socket available, and we are awaiting connections through it. After we do this, we enter the infinite loop where we call select, passing the maximum number of sockets to be tested. We also pass the ready array so the call fills it with data referring to those sockets that have data available. Obviously, here we can find two situations: data available in the base socket, which means we have a new incoming connection that must be accepted, or data in another socket, which is already communicating with us. If we're receiving a connection request, the entry corresponding to the initial socket will be activated. Then, we check for that event with the FD_ISSET call and, if needed, accept the incoming connection and read the string the other peer is sending out. This way we are effectively handling many clients with just one initial socket and opening new ones as needed using select. A complex communication protocol should then be implemented on top of this skeleton, but this simple example should suffice to understand how select works.