add chapter heartbeat, organise files

21 files changed, 101 insertions, 47 deletions

diff --git a/src/gameoverview.tex b/src/appendices/gameoverview.tex
index 8da2418..8da2418 100755
--- a/src/gameoverview.tex
+++ b/src/appendices/gameoverview.tex
diff --git a/src/protocolreference.tex b/src/appendices/protocolreference.tex
index 6345b1e..6345b1e 100755
--- a/src/protocolreference.tex
+++ b/src/appendices/protocolreference.tex
diff --git a/src/usermanual.tex b/src/appendices/usermanual.tex
index 9fd0cfb..9fd0cfb 100755
--- a/src/usermanual.tex
+++ b/src/appendices/usermanual.tex
diff --git a/src/background.tex b/src/body/background.tex
index 5121190..a7d0e3b 100755
--- a/src/background.tex
+++ b/src/body/background.tex
@@ -11,7 +11,7 @@ From the poker analogy, we continue to multiple games. In the local bar, several
 \section{Game interactions}
 To illustrate the more technical interactions between dealers, barkeep and player, we create three roles: {\bf gameserver}, {\bf masterserver} and {\bf client}\footnote{We refer to \emph{gameservers} and \emph{masterservers} instead of \emph{game-} and \emph{master} servers. The latter is correct English, but the former improves readability and avoids confusion about what type of server we explicitly try to describe.} in figure \ref{fig:totaloverview}. When a gameserver is initialised, it repeatedly sends a signal to the masterserver (1). We refer to these signals as \emph{heartbeats}, as the masterserver listens for these signals to determine if the gameserver is still active, similar to how we listen for heartbeats in the human body to determine if somebody is still alive. Game clients make a request to the masterserver to obtain a list of all servers that sent heartbeats and the masterserver provides this list (2). The client then inquires at all of these gameservers for their information (or \emph{status}) so that the user can make a choice which gameserver to join (3).
 
-\begin{figure}[h]
+\begin{figure}[ht]
   \centering
   \includegraphics[width=\textwidth]{img/general-overview}
   \caption{General interactions between gameserver, masterserver and client.}\label{fig:totaloverview}
@@ -23,4 +23,15 @@ All interactions between gameserver, masterserver and client occur over the inte
 \section{Networking}
 In the next chapters we address every interaction between gameserver, masterserver and client, but first we describe what we consider to be some required general knowledge about network communication and protocol. Since there is a variety of literature available for in-depth knowledge, we limit ourselves to the essential information that is necessary to understand the following chapters.\\
 
-There are multiple methods to send information over the internet between peers. The two most important methods are \emph{UDP} and \emph{TCP} communication. Imagine that there is a speaker in front of a large audience. In order to get to know the audience, the speaker could ask everyone to shout their names. Everyone would shout their names one by one to the speaker and hope that the speaker would hear them properly. The \emph{user datagram protocol} (UDP) is similar to this situation: a gameserver sends a heartbeat to the masterserver, but there is no guarantee that information is correctly received or in the correct order. Another method would be for the speaker to ask the audience to walk up to him one by one, shake hands and introduce themselves before sitting down again. The \emph{Transfer Control Protocol}  handles information in the same way: a client connects with the masterserver, identifies itself and retrieves the serverlist. As one could imagine, the information is conveyed correctly, but at the expense of a much larger duration of the interaction. The choice for either of these protocols in the game interactions is defined by the circumstances: some interactions require speed rather than precision, some interactions require precision over speed.
+There are multiple methods to send information over the internet between peers. The two most important methods are \emph{UDP} and \emph{TCP} communication. Imagine that there is a speaker in front of a large audience. In order to get to know the audience, the speaker could ask everyone to shout their names. Everyone would shout their names one by one to the speaker and hope that the speaker would hear them properly. The \emph{user datagram protocol} (UDP) is similar to this situation: a gameserver sends a heartbeat to the masterserver, but there is no guarantee that information is correctly received or in the correct order.\\
+
+The other method would be for the speaker to ask the audience to walk up to him one by one, shake hands and introduce themselves before sitting down again. The \emph{Transfer Control Protocol}  handles information in the same way: a client connects with the masterserver, identifies itself and retrieves the serverlist. As one could imagine, the information is conveyed correctly, but at the expense of a much larger duration of the interaction. The choice for either of these protocols in the game interactions is defined by the circumstances: some interactions require speed rather than precision, some interactions require precision over speed.
+
+\section{Protocols}
+Any form of communication, whether it is on the street, in the shop or digitally between two or more computers, follows a certain communication structure. During any meeting we often shake hands \footnote{This was before the global nCovid-19 pandemic.} and introduce ourselves before getting to business. This is called a \emph{protocol}. A protocol is a set of agreements that all communicating parties follow in order to convey information.
+
+Many of the interactions that we discuss in this document follow the \emph{GameSpy} protocol. This protocol was, as the name suggests, developed by GameSpy Industries\cite{gs0} and contains a series of steps to perform the interactions as described in figure \ref{fig:totaloverview}. The formatting of these interactions are defined in the protocol and described in the next chapter as well.
+
+Over the years, there have been iterations and expansions of this protocol, as well as completely new and different protocols by different third parties. The 333networks masterserver that we describe in the following chapters uses the GameSpy protocol as foundation and expands from there where necessary.
+
+
diff --git a/src/credits.tex b/src/body/credits.tex
index bb0d91c..bb0d91c 100755
--- a/src/credits.tex
+++ b/src/body/credits.tex
diff --git a/src/body/heartbeat.tex b/src/body/heartbeat.tex
new file mode 100755
index 0000000..ad365fb
--- /dev/null
+++ b/src/body/heartbeat.tex
@@ -0,0 +1,50 @@
+\chapter{Heartbeat}
+\label{chap:heartbeat}
+
+In this chapter, we discuss what information a heartbeat contains and how it is processed by the masterserver.
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=\textwidth]{img/overview-hb}
+  \caption{Focus on interaction between gameserver and masterserver}\label{fig:overviewhb}
+\end{figure}
+
+\section{Exchange}
+When a gameserver is first initialised, it sends a signal to the masterserver as seen in figure \ref{fig:overviewhb}. This message contains the gameservers IP-address, the port on which the hosted game is active and an acronym or shorthand name for the game title.
+
+The masterserver receives this message or \emph{heartbeat} and processes this information. As any user or automated software can send these messages, the masterserver will attempt to authenticate the game title by challenging the gameserver with a code phrase. Both the gameserver and masterserver have the necessary algorithm to determine the expected response. The gameserver returns this response and is either rejected as an invalid sender, or acknowledged as a legitimate gameserver and added to the masterserver's list of gameservers. 
+
+Heartbeats are sent repeatedly to indicate that a server is still active. The name ``heartbeat'' is especially similar to how we listen for heartbeats in the human body to determine if somebody is still alive. When the gameserver stops sending heartbeats, the masterserver no longer considers the gameserver active and removes it from the list.
+
+This exchange is visualised in a diagram in figure \ref{fig:heartbeat} where the vertical axis represents passing time and the horizontal interactions correspond to the exchange discussed above.
+
+\begin{figure}[ht]
+  \centering
+  \includegraphics[width=\textwidth]{img/heartbeat-exchange}
+  \caption{Interaction between gameserver and masterserver}\label{fig:heartbeat}
+\end{figure}
+
+\section{Secure challenge}
+To authenticate the validity of gameserver, masterserver and client, the GameSpy protocol introduced the \emph{secure}-challenge. In the case of a heartbeat, the masterserver attempts to authenticate the gameserver by sending it a 6-character \emph{secure} word. Both the masterserver and gameserver have a 6-character \emph{cipher} that is associated with the unique game title and perform a series of mathematical operations on the combination of the secure word and cipher. The result is a unique 8-character \emph{validate} word. If the masterserver and gameserver both arrive at the same word, the gameserver validates as legitimate and is added to the list of online gameservers.
+
+Both the secure word and cipher may consist of any uppercase or lowercase character and any number. The validate word is a calculated response and can contain any uppercase or lowercase character and any number, but also certain special characters such as (back)slashes, underscores and dashes. Later versions of the algorithm also allow to specify an encryption type, which results in a different validate word. The algorithm to determine the validate word consists of a series of array operations and is purely mathematical\cite{lam}.
+
+\section{Formatting}
+Many of the games published between 1995 and 2005 follow the GameSpy protocol\cite{gs0} and communicate heartbeats over the user datagram protocol (UDP). Sending data over UDP is fast at the risk of losing packets, corrupted packets or the information arriving out of order. Heartbeats contain only little quantities of information and are sent every few minutes, which minimises the risk and effect of data not arriving properly.
+
+\lstinputlisting[caption={Example of the GameSpy protocol message formatting.}, stepnumber=1, label={ls:gs0prot}]{lst/gs0protocol.txt}
+
+The GameSpy protocol formats the messages through a backslash. In example \ref{ls:gs0prot}, the first line specifies the general format where there is a keyword, followed by a value, both prefixed by a backslash. A simple heartbeat can be represented by the second line of the example: the {\tt heartbeat} key indicates that the next value is the network port through which the gameserver can be queried and the {\tt gamename} key indicates that the next value is an acronym or shorthand for the game title. An example of how the secure challenge and response would appear is seen at lines 3 and 4.
+
+The minimum requirement for a heartbeat is an empty UDP packet. This conveys the IP-address of the gameserver. A masterserver specifically developed to support this single game title can automatically generate the default network port and game title. However, this means that the gameserver can not be serviced on another network port and that the masterserver is limited to serving a client list \emph{only} for this specific game title. 
+
+In order to support different network ports, a heartbeat must contain a network port (as seen in example \ref{ls:gs0prot}, line 0). A masterserver that supports multiple game titles also requires an additional key {\tt gamename}. The gamename is, as mentioned before, a unique identifier for the game title and allows the masterserver to identify which game is being serviced and authenticated\footnote{The game title and \emph{gamename} both refer to the original publications from game developers. As per the protocol, we use the word ``gamename'' to refer to these titles from now on. This term returns in chapter \ref{chap:serverlist} and \ref{chap:status}}.
+
+Additional information is often incorporated in the heartbeat as well. Some developers allow the indication of a new session through the {\tt statechanged} keyword, that could be used to indicate to the masterserver that a new session or map has been started. An overview of the protocol is added in appendix \ref{app:protref}.
+
+\section{Practical execution}
+In the past fifteen years, we have seen many different ways of formatting a heartbeat. It is worth noting that many game developers deviate from the specified protocol. Some games like \emph{Vietcong}, published by \emph{Gathering of Developers} in 2003, do not specify a {\tt gamename} in their heartbeat, which does not make it possible to support the game on a masterserver for multiple gamenames. 
+
+Another game, \emph{Jetfighter IV: Fortress America} published by \emph{TalonSoft} in 2000, was originally developed with a single masterserver in mind, but did not follow protocol. The gameserver specifies the gamename \emph{jetfighter4} whereas the client specifies the gamename \emph{Jet Fighter IV} for the same game.
+
+These inconsistencies and/or deviations from the original protocol introduce unexpected obstacles in the development process of a replacement masterserver.
diff --git a/src/implementation.tex b/src/body/implementation.tex
index 871f813..871f813 100755
--- a/src/implementation.tex
+++ b/src/body/implementation.tex
diff --git a/src/introduction.tex b/src/body/introduction.tex
index 8511e40..153195e 100755
--- a/src/introduction.tex
+++ b/src/body/introduction.tex
@@ -11,15 +11,15 @@ Since the early years of classic and modern (electronic) games, people have felt
 \section{Masterserver}
 Since 1995, a large number of game titles were released with the capability to play online together. Online games provided the need for the development of networking protocols that made it possible to play together on different systems. Some games required a direct connection with each other (peer-to-peer) whereas other games utilised a server-client hierarchy. In order to find each other on the vastness of the world wide web, technologies were developed to connect peers, game servers and clients together. One of the technologies that was developed for this purpose was a \emph{masterserver}. This technology provides an infrastructure for game servers and game players to interact and play together.\\
 
-Game developers would initially develop their own multiplayer protocol and masterserver for their game title. With the rise of GameSpy Industries in 1995, game developers had the possibility to choose for proprietary software and third party services to provide the multiplayer aspects, matchmaking and masterserver support for their games. With the rise of broadband internet and reliable masterserver services available, it was no longer necessary to develop their own protocols and maintain their own masterserver hardware. Many game developers followed and dependant on GameSpy's services or discontinued their own masterserver infrastructure in favour of the existing services.
+Game developers would initially develop their own multiplayer protocol and masterserver for their game title. With the rise of \emph{GameSpy Industries} in 1995, game developers had the possibility to choose for proprietary software and third party services to provide the multiplayer aspects, matchmaking and masterserver support for their games. With the rise of broadband internet and reliable masterserver services available, it was no longer necessary to develop their own protocols and maintain their own masterserver hardware. Many game developers followed and dependant on \emph{GameSpy}'s services or discontinued their own masterserver infrastructure in favour of the existing services.
 
 \section{Loss of GameSpy}
-In the autumn of 2008, all masterserver services for two of the titles, \emph{Unreal} and \emph{Unreal Tournament 1999} (Epic Megagames Inc$^{\textcopyright}$) went down at the same time. For three days, thousands of players, were frustrated that not a single online server showed up in the server browser\cite{bu2008,ou2008}. In the spring of 2010, the masterserver infrastructure became even more fragile when the rebranded \emph{Epic Games} permanently shut down their masterserver. Effectively, thousands of gamers now depended on a single remaining GameSpy masterserver. In December 2013, GameSpy Industries was bought by GLU Technologies, leading to the shutdown of all GameSpy services on May 31, 2014\cite{nutt2014, stapleton2013}.\\
+In the autumn of 2008, all masterserver services for two of the titles, \emph{Unreal} and \emph{Unreal Tournament 1999} published by \emph{Epic Megagames Inc\textsuperscript{\textcopyright}} went down at the same time. For three days, thousands of players, were frustrated that not a single online server showed up in the server browser\cite{bu2008,ou2008}. In the spring of 2010, the masterserver infrastructure became even more fragile when the rebranded \emph{Epic Games} permanently shut down their masterserver. Effectively, thousands of gamers now depended on a single remaining GameSpy masterserver. In December 2013, \emph{GameSpy Industries} was bought by GLU Technologies, leading to the shutdown of all \emph{GameSpy} services on May 31, 2014\cite{nutt2014, stapleton2013}.\\
 
-Game developers and publishers who had relied on GameSpy to provide online multiplayer support for two decades, were left empty-handed. People who bought expensive game titles were no longer able to play online in the era where connecting with each other is essential. In response to the shutdown, some game developers like Epic Games returned to an in-house solution. Many other multiplayer titles and their communities had no such luck.
+Game developers and publishers who had relied on \emph{GameSpy} to provide online multiplayer support for two decades, were left empty-handed. People who bought expensive game titles were no longer able to play online in the era where connecting with each other is essential. In response to the shutdown, some game developers like \emph{Epic Games} returned to an in-house solution. Many other multiplayer titles and their communities had no such luck.
 
 \section{Filling the void}
-In the early years of this century, a number of Unreal Tournament players and their clans worked together to display (interactive) statistics of their game servers. Through \emph{php}-scripts it was possible to obtain server information and display this on websites for one or more individual servers. With a small group of people, we started \emph{333networks} as a statistics website and private game server host for Unreal Tournament 1999. With more understanding of \emph{server queries} and \emph{network infrastructure}, 333networks started experimenting with replicating properties of the masterserver to obtain more server information for display. This lead to the development of an intrinsic masterserver that could list all cooperating Unreal Tournament servers on the website. Due to various successes, 333networks continued with the development of a fully functional masterserver with integrated website, which was completed mere months before GameSpy shut down in 2014. This document contains all the knowledge that was accumulated in the process of developing and expanding the 333networks version of the masterserver.
+In the early years of this century, a number of Unreal Tournament players and their clans worked together to display (interactive) statistics of their game servers. Through \emph{php}-scripts it was possible to obtain server information and display this on websites for one or more individual servers. With a small group of people, we started \emph{333networks} as a statistics website and private game server host for Unreal Tournament 1999. With more understanding of \emph{server queries} and \emph{network infrastructure}, 333networks started experimenting with replicating properties of the masterserver to obtain more server information for display. This lead to the development of an intrinsic masterserver that could list all cooperating Unreal Tournament servers on the website. Due to various successes, 333networks continued with the development of a fully functional masterserver with integrated website, which was completed mere months before \emph{GameSpy} shut down in 2014. This document contains all the knowledge that was accumulated in the process of developing and expanding the 333networks version of the masterserver.
 
 \section{Content overview}
 TODO: in this section we describe what information can be found where. This is not a repetition of the table of contents, but a contextual, descriptive narration of the document itself to justify the chosen order of information. Save this for later.
diff --git a/src/revisions.tex b/src/body/revisions.tex
index 3364e57..3364e57 100755
--- a/src/revisions.tex
+++ b/src/body/revisions.tex
diff --git a/src/serverlist.tex b/src/body/serverlist.tex
index d7c0a64..d7c0a64 100755
--- a/src/serverlist.tex
+++ b/src/body/serverlist.tex
diff --git a/src/status.tex b/src/body/status.tex
index be0f264..be0f264 100755
--- a/src/status.tex
+++ b/src/body/status.tex
diff --git a/src/website.tex b/src/body/website.tex
index af9f86d..af9f86d 100755
--- a/src/website.tex
+++ b/src/body/website.tex
diff --git a/src/doc.tex b/src/doc.tex
index 339f0b1..57847e9 100755
--- a/src/doc.tex
+++ b/src/doc.tex
@@ -1,31 +1,31 @@
 \documentclass[10pt, a4paper]{report}
-\input{headers}
+\input{layout/headers}
 \begin{document}
 \pagenumbering{gobble}
-\input{titlepage}
-\input{contact}
+\input{layout/titlepage}
+\input{layout/contact}
 \pagenumbering{roman}
 \setcounter{page}{1}
-\input{abstract}
-\input{revisions}
+\input{layout/abstract}
+\input{body/revisions}
 \setcounter{tocdepth}{2}
 \tableofcontents
-\input{introduction}
-\input{background}
-\input{heartbeat}
-\input{serverlist}
-\input{status}
-\input{implementation}
-\input{website}
+\input{body/introduction}
+\input{body/background}
+\input{body/heartbeat}
+\input{body/serverlist}
+\input{body/status}
+\input{body/implementation}
+\input{body/website}
 \clearpage
 \phantomsection
 \addcontentsline{toc}{chapter}{Bibliography}
 \bibliographystyle{unsrt}
-\bibliography{references}
-\input{credits}
+\bibliography{layout/references}
+\input{body/credits}
 \begin{appendices}
-\input{usermanual}
-\input{protocolreference}
-\input{gameoverview}
+\input{appendices/usermanual}
+\input{appendices/protocolreference}
+\input{appendices/gameoverview}
 \end{appendices}
 \end{document}
diff --git a/src/heartbeat.tex b/src/heartbeat.tex
deleted file mode 100755
index a106950..0000000
--- a/src/heartbeat.tex
+++ /dev/null
@@ -1,23 +0,0 @@
-\chapter{Heartbeat}
-\label{chap:heartbeat}
-
-repeat introduction of heartbeat from chap:background, figure with focus on gameserver-masterserver interaction.
-
-\begin{figure}[h]
-  \centering
-  \includegraphics[width=\textwidth]{img/overview-hb}
-  \caption{Focus on interaction between gameserver and masterserver}\label{fig:overviewhb}
-\end{figure}
-
-explanation of purpose, UDP information which it contains and/or is meant to convey.
-\\
-security challenge (secure/validate)
-\\
-ports that are opened, listening\\
-\\
-
-\begin{figure}[H]
-  \centering
-  \includegraphics[width=\textwidth]{img/uplink-exchange}
-  \caption{Interaction between gameserver and masterserver}\label{fig:uplink}
-\end{figure}
diff --git a/src/img/uplink-exchange.png b/src/img/heartbeat-exchange.png
index 9936a3d..9936a3d 100644
--- a/src/img/uplink-exchange.png
+++ b/src/img/heartbeat-exchange.png
diff --git a/src/abstract.tex b/src/layout/abstract.tex
index b390cb9..b390cb9 100755
--- a/src/abstract.tex
+++ b/src/layout/abstract.tex
diff --git a/src/contact.tex b/src/layout/contact.tex
index ee0dcf0..0099e23 100755
--- a/src/contact.tex
+++ b/src/layout/contact.tex
@@ -7,7 +7,7 @@
 {\bf Title}             & : & An introduction to the masterserver.\\
 {\bf Subtitle}          & : & Reference for heartbeats, serverlists, status and implementation.\\
 \\
-{\bf Revision}          & : & 0.2\\
+{\bf Revision}          & : & 0.3\\
 {\bf Date}              & : & \today\\
 \\
 {\bf Author}            & : & Darkelarious\\
diff --git a/src/headers.tex b/src/layout/headers.tex
index 73271d1..73271d1 100755
--- a/src/headers.tex
+++ b/src/layout/headers.tex
diff --git a/src/references.bib b/src/layout/references.bib
index d170e44..c97ca81 100755
--- a/src/references.bib
+++ b/src/layout/references.bib
@@ -1,3 +1,10 @@
+@misc{gs0,
+    author = "{GameSpy Industries}",
+    title = "{The GameSpy Open Architecture for Free Internet Gaming Developer Specification}",
+    edition = "{2.2}",
+    year = {1999}
+}
+
 @misc{oldunreal1,
   author = {Brandon ``GreenMarine'' Reinhart},
   title = {All you ever wanted to know about the Master Uplink settings in Unreal - IpServer Package for Unreal v1.0},
@@ -8,6 +15,11 @@
   publisher = {Epic Games}
 }
 
+@misc{lam,
+    title = "{GameSpy Masterserver Algorithm 0.3.3}",
+    author = "{Luigi Auriemma}",
+    howpublished = "{\url{http://aluigi.altervista.org/papers.htm#gsmsalg}}"
+}
 
 @misc{bu2008,
     title = "{Lamespy master server down}",
diff --git a/src/titlepage.tex b/src/layout/titlepage.tex
index 266f4d3..266f4d3 100755
--- a/src/titlepage.tex
+++ b/src/layout/titlepage.tex
diff --git a/src/lst/gs0protocol.txt b/src/lst/gs0protocol.txt
new file mode 100644
index 0000000..f7ed940
--- /dev/null
+++ b/src/lst/gs0protocol.txt
@@ -0,0 +1,4 @@
+\key\\value
+\heartbeat\7778\gamename\ut
+\secure\wookie\enctype\0
+\validate\2/TYFMRc