Jarate Chop!

Visualising 70,000 Rounds of Smurfy TF2

| Comments

A few weeks ago we introduced SmurfPowder™, a system that assigns players to team based on rankings derived via TrueSkill. The goal of this system is to pick balanced teams so that games will be challenging for all. Specifically, we wanted to address the issue where our previous (random) assignment system would sometimes produce hugely unbalanced teams, resulting in one team steamrolling the other.

gsan and I spent several days building, testing & deploying SmurfPowder™ and it is running well but how do we know if it's actually improving things?

We need a way to identify bad games so that we can see if the proportion of bad games decreases over time.

From experience, a couple of examples would be:

  • Blu failing to capture point A on Badwater, or failing stage 1 on Goldrush
  • Blu literally steamrolling Red on Badwater or Frontier by pushing the cart from start to finish without pause

We could try to write down every bad game scenario but this is tedious, error prone, and limited by our imagination and personal experience.

We have the logs from every round played since Smurfy Fortress began so we decided to do some log crunching and use visual tools to help us identify features that define bad games. A link to the raw data is provided at the end of this post.

Initial Exploration

Although we had some ideas of the sorts of features we were looking for we weren't entirely sure so I went looking for a visual tool that would make experimentation quick & easy. I had heard good things about Tableau so downloaded a trial.

Tableau makes it very easy to visualise data. You simply drag & drop dimensions and measures, and with a few clicks you can try out different charts to see what patterns emerge.

The two examples listed earlier led us to consider Round Duration and Kill Difference as possibly significant features of bad games. The intuition is that unusually short durations probably indicate a Blu steamroll. For longer durations it's not clear whether one team was overly dominant so for longer durations we need to take into account other metrics such as the Kill Difference.

Initially we plotted Average Round Duration vs Time, and Average Kill Difference vs Time but these graphs were uninspiring and rightfully so. Both Round Duration and Kill Difference vary greatly by map so averaging across all maps produces a meaningless statistic. Fortunately Tableau makes it easy to dissect data by any dimension so we were quickly able to replot the same data by map/stage, as well as experiment with varying time windows (i.e month, week, day etc).

Let's start with Round Duration. Clickity, click, here we have Round Duration vs Winner for Payload maps summarised by week.

Note: I recommend opening these graphs in a new tab as we will provide some commentary below so you'll want to be able to flick between this post and the related graph.

Round Duration vs Winner for Payload maps

Across the graph we have partitioned the data by map, and stage where applicable. Within each map/stage each vertical set of points represents one week of matches. Vertically we have the Round Duration, and the colour of each point indicates the Winner of that particular round.

Immediately we can see some patterns. For example in Badwater and Frontier (first two columns) we can see horizontal red lines that correlate with Red wins (successful defence) at each capture point.

We can also see that on Goldrush the density of the Blu dots (Blu wins) in the lower region indicates that Blu often wins stage 1 and 2 very quickly but stage 3 is more challenging. Regular Hoodoo players may be amused by the stray Blu win at the 100 second mark – that's the runaway bomb bug.

This graph confirms some of our intuition but without futher information we can't tell whether games were balanced. Short duration Blu wins on Payload may indicate a steamroll but with so many short duration Blu wins on Goldrush were they all bad games?

Let's look at the same games but plot Kill Difference vs Winner for Payload maps.

Kill Difference vs Winner for Payload maps

Note that in this graph the vertical axis is Red Kills - Blu Kills so negative values indicate that Blu had more kills than Red.

Intuitively we would expect that Blu wins would correlate with Blu having more kills and Red wins correlate with Red having more kills and this is mostly the case but not always. We can see several cases on Frontier where Red wins despite having less kills, and many cases where Blu wins despite having less kills. Overall it seems that for Payload maps Red needs to out deathmatch Blu in order to win.

What's interesting about this graph to me is the vertical spread. Intuitively we would expect close games to have relatively small Kill Differences whereas we may surmise that the Badwater round with a near 150 Red kill advantage was probably very one sided. Perhaps we could draw a line at Kill Difference of 100 and say that all games above this point were one-sided? Similarly we might say Blu wins with Kill Difference below -30 may have been one-sided.

It becomes painfully obvious at this point that neither of these graphs provides quite enough information to decide which games might be bad but there are definite patterns in the data. Before we go on to plotting Round Duration vs Kill Difference by map let's quickly look at the above plots for other map types.

Round Duration vs Winner for Attack/Defend CP maps Round Duration vs Winner for Attack/Defend CP maps

Kill Difference vs Winner for Attack/Defend CP maps Kill Difference vs Winner for Attack/Defend CP maps

These are very similar to their Payload counterparts which makes sense given that both share the format of one team attacking, one team defending.

For the symmetrical maps we get very different patterns:

Round Duration vs Winner for Symmetric CP maps Round Duration vs Winner for Symmetric CP maps

Kill Difference vs Winner for Symmetric CP maps Kill Difference vs Winner for Symmetric CP maps

Round Duration vs Winner for KOTH, PLR, SD & TC maps Round Duration vs Winner for KOTH, PLR, SD & TC maps

Kill Difference vs Winner for KOTH, PLR, SD & TC maps Kill Difference vs Winner for KOTH, PLR, SD & TC maps

As one would probably expect given that the maps are symmetric and the team objectives are also symmetric the plots also look fairly symmetric. SYMMETRY ALL THE THINGS! Blu & Red win in roughly the same durations, and with roughly the same kill differences.

It may be worth noting the spread of round durations for symmetric CP maps, as well as the crazy durations of Hightower vs the other Payload Race maps. I guess people really like Market Gardening.

There's many observations that could be made, comparing maps & game modes but this blog post is already insanely long. :) Please feel free to leave any observations in the comments below!

Map by map breakdown

We then dug into the data map by map, round by round and plot Round Duration vs Kill Difference. We jumped out of Tableau for this part. Tableau is fantastic but I can't afford the $999 license fee and we need metrics that we can monitor over time in order to evaluate SmurfPowder™'s effectiveness.

gsan wrote some Python code to generate plots as we felt by this point that we had a reasonable basis for long term metrics. I'll highlight a couple of examples but the full set is linked at the end of this post.

Note that in this series of graphs we switched from Kill Difference (Red - Blu) to Winning Team Kill Difference. I apologise for this inconsistency but we opted to use the former to highlight symmetry in the earlier graphs but the latter makes more sense when we look at Round Duration vs Kill Difference.

Let's take a look at Badwater, being one of the most popular TF2 maps.

Round Duration vs Winning Team Kill Difference for Badwater Round Duration vs Winning Team Kill Difference for Badwater

Each point represents the winner of the round as before. The solid blue line and solid red lines mark regions which capture our intuitions regarding features of bad games. The larger dots indicate the games we have classified as bad, based on the aforementioned intuitions as well as other heuristics.

We suppose that, given sufficient game rounds, for particular maps/stages some percentile of the kill difference and duration are bad games. For simplicity, we assume that for all maps, all kill differences greater than 95 percentile and all durations less than 20th percentile are bad rounds. We classify the intersection of these as bad rounds. However, it should be noted that there is correlation between kill difference and round duration.

Furthermore, we also consider those rounds in Attack/Defend maps where Red is able to hold the first half of all capture points as bad games.

For Badwater the Blu line captures 2 intuitions. Firstly, short duration games are almost certainly bad – capturing 4 points in under 400 seconds means the cart barely stops, if at all. The second is that as the game goes on (duration increases) we would expect the kill difference to increase but we can see that across all games the kill difference rarely exceeds 50. The Red line captures the intuition that winning at CPs 1 and 2 at any duration is probably okay as long as the kill difference is not large. Red wins at CPs 3 and 4 with larger kill differences than at CPs 1 and 2 may also be okay since the game has gone on longer.

Exactly where the boundaries should lie is debatable but for our purposes precision matters more than recall so we try to be conservative in that respect. We also expect that we will adjust the boundaries as we further study the data and gain additional insights.

As another example let's look at Goldrush stage 3.

Round Duration vs Winning Team Kill Difference for Goldrush stage 3 Round Duration vs Winning Team Kill Difference for Goldrush stage 3

Unlike Badwater this graph captures a single stage (stage 3) of Goldrush. Like Badwater, our intuitions here are that Blu steamrolled Red if they won the round super quickly, or if the kill difference is large. If Red wins with a large kill difference then it's likely the game was one sided but we use a graduated Duration:Kill Difference ratio to allow for larger kill differences in longer games.

These are just two examples and the graphs for each map/round look very different so I encourage you to poke around at the full set.

Toward a single evaluation metric

To figure out if SmurfPowder™ is effective in reducing the number of bad rounds we aggregate all of the above into a single metric: percentage of bad rounds over time.

Smurfy percentage of bad rounds Smurfy percentage of bad rounds

The green shaded area indicates when SmurfPowder™ was active and as yet we do not have enough data to know if it has improved things but we can at least see that it hasn't made things noticeably worse. That's a good start.

What next?

We'll need to wait for SmurfPowder™ to accumulate game results and refine its rankings for individual players, and we'll continue to monitor our bad rounds metric over time.

In the meantime we'll continue to update & further analyse the data that we have and encourage you to do the same.

The raw data produced from log parsing is available here. There are two tab delimited files containing data summarised by server, date, map, and mini-round. Mini-rounds are stages of multi-stage maps such as Dustbowl and Goldrush. all.tsv contains all of the data for Smurfy Fortress beginning October 2012, and all_6month.tsv is the most recent 6 months, both ending 3rd January 2014.

The earlier Tableau graphs above were generated using the 6 month data set but the later Python generated graphs use the full Smurfy set. This was initially because I dun-goofed the processing and didn't want to wait 5 hours to generate the full set but it turns out to be more pleasant for the Tableau graphs as 14 months worth of data for each map would have been too visually dense (or require larger graphs).

The 6 month Tableau workbook is available at Tableau Public.

Please download and play with the data and let us know if you have any suggestions or feedback.

Also, we would love to analyse data from other TF2 communities! If you run TF2 servers and are willing to share your logs with us please get in touch (leave a comment below or email smurfy@mooh.org).

Community & Sustainability

| Comments

After ~14 months running Smurfy Fortress I thought I'd share some of our experience in terms of building a lasting TF2 community. This post is also, in part, prompted by the many recurring threads on reddit regarding premium or donator perks.

Firstly, I'll say that I don't particularly care what other server operators do or don't do but our own players have suggested various perks at various times so this post seeks to share our thoughts around this.

As I said back in November 2012 the goal with Smurfy Fortress was to host servers that I want to play on. Broadly, this means fair & fun games, free of abuse. Above all else, game play is paramount. This isn't only a personal preference but I think it's the key to the sustainability of any gaming community.

Why is community important?

Well, firstly, multiplayer games aren't much fun on your own.

More importantly, anyone can put up a server for $20/month. It's neither expensive nor technically difficult. The only thing that differentiates one server from another is the players… i.e. the community.

Ok, if community is so important, how do you build a community?

One approach is simply to bribe, or buy, players. You would've seen this approach in many forms. e.g. join our server at

This may be okay to kickstart a community but long term this approach is fundamentally flawed as you're essentially paying players to play on your server. Unless you have unlimited funds this is obviously unsustainable, and when you eventually run out of funds your players will migrate to the next "community" offering perks, hats, and whatever else.

Let's think about this another way.

We know that players like to play with their friends. So maybe the easiest way to grow a community is to ensure that the players that we want as part of our community consistently play on our servers. Then, over time, their friends will become regulars, then their friends, and so on. Growing a community this way is essentially free (monetarily, it certainly takes time & effort).

So then the question becomes: what makes players stay on a server?

This seems simple enough. Players leave if they're not having fun.

This brings us back to donator perks. Such perks might be good in terms of fundraising, and are almost certainly fun for the donors, but are terrible for everyone else in your community. If you assume your players aren't completely stupid, the vast majority of players who aren't donors will never play again on your server once they realise they're being raped by players who paid for advantage. Long term, your "community" consists only of donors and new players who don't know any better.

An optimisation of this model is to entice new players to join your servers through giveaways as mentioned earlier. This may be sustainable since the donator perks drive revenue which allows the server operator to continue offering enticements to new players. But this isn't the sort of server I care to operate.

What we've chosen to do with Smurfy is focus on game play.

We started with basic things such as auto-assigned teams (to combat team stacking), and class limits (no one wants to play with 5 snipers, though 12 snipers is hilarious!).

As the community grew abuse became an issue. So we added more admins, and also created the Elder Smurfs. Essentially we give long time regulars some recognition (a server greeting) and the power (!votekick and !votemute) to help us keep the servers clean… for everyone's benefit. Players have asked why Elders don't get other perks, e.g. the ability to have Unusual effects added to their hats. This comes back to our optimisation goal - we want to build a lasting community of friendly players. i.e. we don't want players to play and rack up hours because it will get them perks (any server can offer perks), we want players to play because they enjoy the games & community.

With a scalable way to deal with abuse the main problem we have now is team balance. Although the random assignment we used previously was fair in the sense that no one was allowed to bias the teams, the resulting teams weren't always skill balanced and the game would be one-sided.

A side note – some players are happy with, or think that they are happy with, an easy win. Afterall, a win is a win. But my contention is that what players really want is a challenging win. Consider this – if you play tennis with your best mate and you beat him in straight sets 6-0, 6-0, week after week, how long do you think he'll keep playing? He'll feel terrible, and you don't get any sense of achievement, it's like beating a 5 year old. One of the challenges in making games fun for everyone is remembering that winning is fun but not if it is a walkover.

To resolve this we recently introduced SmurfPowder which ranks players based on whether their teams win or lose, and attempts to pick skill balanced teams based on these rankings. The system needs time to establish rankings so we're unsure how much it will help but we will certainly share our findings when we can.

For the business nerds I see parallels with marketing & product strategy. If your product (games in this case) is shit you'll have to spend a bunch on sales & marketing to sustain your business (community, in our case). OTOH, if your product is good, word of mouth will go far. I think both models are valid from a business viability perspective but I know which one I prefer.

Also, on differentiation – anyone can set up a server and implement any of the technical features mentioned, including the TrueSkill based rankings. Long term, the only differentiator for gaming communities is, unsurprisingly, the community itself.

Populating a TF2 Server

| Comments

One of the hardest things about starting a new TF2 server is getting players to play on it. No one wants to join a server with no players, but if no one joins the server with 0/24 players then how does a server ever get started?

Initially with Smurfy Fortress it was up to the core admin group to get the servers populated. We had all played thousands of hours on other servers so were lucky to have quite large friends lists.

The process goes like this:

  1. Our intrepid server starters join the server via their Favorites list
  2. Friends of theirs join the server
  3. A few randoms will join via the server browser
  4. Once we get to 12+ players Quickplay begins sending players to the server

The exact point at which Quickplay kicks in varies depending on the server reputation.

The crux of it is that without a few players joining the server manually to begin with Quickplay would never kick in, and the server will never be populated.

A significant problem between steps 1-3 and 4 is keeping everyone entertained as sometimes it can take half an hour to get to step 4.

When there's only a handful of players it's very easy for the game to be one sided. All it takes is one good combo and steamroll ensues, causing players to leave to find a more balanced game. Every player is critical at this point because you need to hit the magic threshold for Quickplay to help fill the server.

Fortunately we've built a friendly community on Smurfy Fortress and regular players have devised a number of silly games to play with low numbers of players.

Here's one example known internally as COD Fortress 2:

Others include combinations of:

  • Battle Medic - everyone plays Medic and equips the Blutsauger
  • Tickle Fights - Holiday Punch Heavy, no miniguns, just tickle and taunt kills
  • Trolldier - Soldier using the Rocket Jumper, aiming for Market Gardener or Mantreads kills only
  • Stab Stab Sniper - Huntsman Sniper, going only for Skewer kills
  • Flare Pyro - guess what? Pyro, but you can only use the Flare Gun. Whodathunkit.
  • Melee only

The point of it all is stupid but fun. It's much harder to kill people with these loadouts, and less biased towards skill/experience, so it's not only fun but also non-rage inducing. And it works with any number of players.

The end result is that the servers populate faster as players stick around because they're having fun. We do lose a few players who dislike the fact that people aren't playing seriously but that is actually beneficial to our community as we want players who don't take the game so seriously all the time.

Anti-inflation in the TF2 Economy

| Comments

Today the crafty econonerds at Mann Co. announced that they would retire a bunch of the very first hats introduced to TF2 back in May of 2009.

I remember that day - getting my first Huntsman, arching arrows from base to base in the first stage of Pipeline, and admiring the lucky souls who received the very first Pyro Beanies. Ah nostalgia.

So much has happened in the TF2 economy since - many, many new items, player to player trading, the Steam Workshop, a plethora of 3rd party sites enabling sales of items for real money, and recently the official way to do the same - the Steam Community Market.

But one thing has remained constant - inflation.

With every active player getting 14 new items (0.77 ref) every week by simply playing the game, and a near zero marginal cost of "producing" new items, it's kind of like the Federal Reserve printing money continuously (which, thanks to the "global financial crisis", the real Feds are actually doing). It's the reason why metal prices only ever fall - the price of something given infinite supply is fundamentally zero.

There are several effects of this first step Valve is taking towards limiting item availability. Most traders seem to care about the price effects on these particular items so let's look at that first.

Everyone knows that price is a function of supply & demand.

In the immediate future the demand side for these items will spike due to speculators who feel the price of these items will rise once the supply is limited in future. Also, anyone who wanted these items already has them, or will buy them before they become limited. So one would expect an immediate but perhaps temporary price rise.

But immediately after these items are no longer available I don't foresee any further rise in the price because consumer demand will simply not exist. Very few consumers want these hats which is why their prices are so low, and new demand for these items could only come from new TF2 players.

But there is a more important effect. By cutting off the availability of these items Valve is signalling that it won't allow the economy to debase itself by providing infinite supplies of everything. I think that is more important than the micro effects on the prices of the individual items.

We should see, longer term, that all prices rise due to the implied threat that supply will become limited at some point in future – up til now everyone has worked (read: priced) on the assumption that supply is limitless which is why many items have pathetically low values. There is also the fact that the Mann Co. Store currently sets a price ceiling on every item.

Fun-having players may view this as a negative as the prices of the hats they want will go up. But ultimately it's in everyone's interest that the TF2 economy is strong – Valve relies on it for income to pay the salaries of the engineers & artists who provide game updates, and community item & map makers rely on it for income too.

What Is the Most Popular TF2 Map?

| Comments

Having played TF2 for 5+ years I've been involved in my fair share of debates as to what maps are the "best", "most popular" and so forth. I finally got around to looking at some data.

Here's a graph of the top 20 most played maps on Australian servers for the period 26 November 2012 to 3 December 2012. There's nothing special about this period, I simply picked a roughly 7 day period to look at. This data includes only normal TF2 servers - specifically it excludes MvM and all mods that I know of (such as PropHunt, Dodgeball etc).

The full list of (144!) maps is public.

Firstly - over a 7 day period Aussies racked up an astounding 4.4 million minutes of TF2. We played 8.36 years worth of TF2 in one week. Wowsers. And again - this doesn't include MvM, mods, idle & achievement servers etc - this is purely normal PvP TF2.

Given the number of 24x7 2Forts, Dustbowl, Turbine, Goldrush and Badwater servers it's unsurprising that these are the top 5 most played maps. It is staggering though that 2Forts, Dustbowl and Turbine make up almost 55% of the total play time.

I've never understood the popularity of 2Forts but clearly there is an extremely dedicated sub-population of the Aus TF2 community that enjoys it. Goldrush and Badwater often make up the core of popular fixed rotation servers and clearly that role is well deserved.

After that there's a few surprising results, at least to me. I would not have expected DeGroot Keep to be up in #6 – though we should note the clear gap between Badwater at #5 (6.7%) and DeGroot at #6 with less than half that (2.3%).

I'm surprised to see Egypt and Hightower up so high - I rarely see these maps played but that is the whole point of this analysis – it's clear that our own play style preferences bias our perspective on what maps are the most popular.

Doomsday is another surprise at #10 - most folks I know don't seem to like it but again that goes to show the bias inherent in one's group of friends. KOTH Kong King is also more popular than I expected – of all the KOTH maps I would've thought it's the worst hehe.

Double Cross is lower down than expected given that it's regularly voted in on full rotation servers I've frequented, and that there are a few 24x7 servers around.

As with all stats, the more you look the more questions you have. I'd love to understand the diversity of players across these map types – is there a distinct set of 2Forts devotees? – and are there specific factors that make the top 5 so much more popular than the rest of the maps?

Are these results surprising to you?

Methodology: I poll every server in Australia roughly every 5 minutes so we'll miss things like players leaving/joining between each reading but given that we are looking at total play time over 7 days these factors should even out and the relative values should be about right.

User Friendly TF2 Donation System

| Comments

Collecting donations is an annoying problem for server operators and players alike.

The most obvious method is to simply collect money via a payment system such as PayPal. However that automatically incurs a 2.4% fee + 30c per transaction which becomes 8.4% (32c) of a $5 donation or 5.4% (54c of a $10 donation). It's pretty inefficient, nevermind the annoying UI.

Some GSPs (game server providers) offer solutions such as Multiplay's Clan Pay. That's a step in the right direction but it should be much easier.

The biggest barrier to donating IMHO is the fact that a player has to do it outside of the game interface. And if you've ever run a server you know there's not many opportunities to nicely remind players to donate - MOTDs are routinely ignored and spamming messages during the game is futile because people are there to play, not to read your spam chat messages.

Ideally players should be able to pay directly from the game. If Valve were to get involved they could implement a Flattr type system where players could nominate an amount they are willing to spend each month (say $5/mo) and Valve could distribute it amongst the server(s) the player likes, or play on frequently.

But foregoing Valve's involvement there is a solution that some motivated hackers could rig up. This solution also has the benefit that players don't have to contribute money but can donate items/metal instead. This has the benefit that anyone can contribute, not just those with credit cards, and people can contribute without spending any money - they only need to play the game to get random item drops. Also, it would mean that truly micro donations would be possible - given a refined metal is worth roughly 55c and 18 weapons go into that, a single weapon is worth roughly 3c.

How could this be done?

The server operator needs a way to automatically accept items from players. The bots implemented by TF2WH and scrap.tf (whose SteamBot code is open source BTW) could handle this.

Some plugin code could make such bots initiate trading requests to players. So a player could type !donate into chat and the player would receive a trading request to accept donations.

The other part of the equation is liquidating metal/items to cash. I don't really know of an efficient way to do this though one possibility might be to sell items into TF2WH, convert to Bitcoin via TF2WHX, then sell the Bitcoin to cash. This could all be (in theory) automated.

The incentive for someone to operate this whole donation bot+liquidation as a paid service is that the service operator could take a cut of the donations.

Someone build it! I'll be your first customer.

Why Smurfy Fortress

| Comments

Smurfy Fortress is my attempt at building the game server network that I want to play on.

Most of the Australian TF2 servers are perfectly fine from a technical standpoint - their servers & networks are fantastic and ye olde problems of lag etc are no longer an issue.

The things that did bother me on other game networks were mostly configuration and community related.

Playing on other servers I felt restricted in the efforts I could make in making the servers better to play on. It's hard for admins to know who to listen to, and making changes always involves some risk of not only breaking the server, but also alienating existing community members so admins are understandably reluctant to make frequent changes.

There are 3 key things that I hope to make "better" on the Smurfy Fortress servers.

The first is to listen to feedback, observe player behaviour, act quickly, and observe the results. I'm running the servers as a never ending series of experiments. It's clear after playing TF variants for 15+ years and having been involved in my fair share of theorycrafting that there is no perfect answer. Let's just try things out and see what works, and do it in a tight loop so players don't feel like they're ever stuck with a shitty status quo.

My second goal is for the Smurfy servers to be generally pleasant to play on. There's two aspects to this.

One is removing bad actors. I have no tolerance for players who want to grief, troll, hack etc, and encourage our admins to ban such players. My feeling is that there's no shortage of players who will be nice to each other and by banning those who aren't we establish a co-operative community that will hopefully attract other friendly players.

The second is fair games. Team stacking is a huge problem on many servers. Some servers run gScramble and auto-scramble the teams when an imbalance is detected but finding the right thresholds so that it triggers just enough and not too much is really hard. There isn't a perfect solution but so far we've been trialling auto-assigned teams with good success. At the very least auto-assign prevents deliberate stacking so if you get rolled at least you know it wasn't due to malicious intent by the other team.

Lastly, one thing that has always bugged me about some community run servers is the abuse of admin powers. Apart from arbitrary kicking & banning it also extends to things such as randomly changing maps on a whim. Another is the use of reserved slots for either paying members or admins - this breaks auto-diallers for ordinary players and basically says "we don't value you plebs". I think this is a flawed attitude as you need a reasonably large group of co-operative players to make the game fun long term.

Funding & admining the servers long term is a challenge. The sums of money involved are not huge but I would like to get Smurfy Fortress to a point where it is self-sustaining.

The Beginnings of a TF2 Network

| Comments

It's been almost 2 months since I last posted and that's because I've been working on the Smurfy Fortress servers.

It all started with the release of MvM. Fletcher Dunn from Valve posted a notice saying that a MvM server would require as much CPU as a regular TF server. At that point I had never run a TF2 server so I had no idea how much CPU that actually meant but one thing was obvious - every CPU currently dedicated to a 24 player server could only handle 6 players in MvM.

In other words, at MvM launch the vast majority of players would want to try it out, and unless there was a 4 fold increase in hardware dedicated to MvM Australia wide there was bound to be massive queues for MvM.

The other realisation was that if there's only 6 players and the recommended minimum rate per player was only 30kB/s the peak bandwidth required is roughly 180kB/s or 1.5Mbps. My cable connection is 100/2.4Mbps. Woah, that means it's entirely possible to host a MvM server on my cable!

There didn't seem to be any good guide out there about how much CPU was actually required but keeping in mind that TF2 came out in October 2007 it's fair to assume that one doesn't need to have the most modern overclocked CPU to host a server.

Being a Linux nerd I fired up a KVM on my dev box (a Q9400 quad core) with 1GB of RAM and that was perfectly fine for MvM.

TF2 MvM CPU usage

On the network side an MvM server averages around 600kbps, and peaks at less than 1Mbps outbound.

I've written down the full specifications for an MvM server.

I ran an MvM server on my cable connection for about 2 months and continued to play regular PvP TF2 on the games.on.net servers. Unfortunately in early October all of the GON TF2 servers disappeared. It turns out they were doing a massive platform upgrade.

As a regular 24p TF2 server requires more bandwidth than my cable can handle I began looking at VPSes. I managed to get a couple from GloVine and Exigent to experiment with, and thus Smurfy Fortress was born. It's been an interesting experience finding VPS hosts that allow game servers but I'll save that for another post once I have more data.

Dilbert TF2

| Comments

reddit user Xovaan has created a series of awesome Dilbert-ified TF2 characters:

Dilbert Medic:
Dilbert Medic

Alice Demoman:
Alice Demoman

Dogbert Spy:
Dogbert Spy

World's Smartest Garbageman Engineer:
World's Smartest Gargageman Engineer

Loud Howard Scout:
Loud Howard Scout

Elbonian Soldier:
Elbonian Soldier

Pointy-haired Boss Heavy:
Pointy-haired Boss Heavy

Wally Sniper:
Wally Sniper

Catbert Pyro:
Catbert Pyro

They're hosted here, with Xovaan's permission, for posterity since the original images were posted to Imgur where they will eventually expire. Each image links to the original full sized version.

The original reddit threads are, in chronological order, oldest first:

Australian & NZ TF2 Hosting

| Comments

Being a network & datacentre nerd I wondered who hosts the multitude of game server providers in Australia.

I took the most recent 72 hours of Australian & NZ TF2 data and mapped the server addresses to ASNs using Team Cymru's IP-to-ASN service which gives us a rough idea of who the underlying network providers are. Keep in mind that "networks" in this post refers to the underlying IP networks, not the game networks (e.g. 3FL, GamersUN, games.on.net) that most players are most familiar with.

It's important to note that as this table aggregates servers based only on their IP-to-ASN mapping it hides a lot of the relationships. For example an end user might rent a server from provider X, provider X may in turn be renting the servers "wholesale" from provider Y, who in turn may be leasing datacentre space & bandwidth from provider Z. We're viewing things here mostly from the viewpoint of the Z providers.

The main purpose in generating this table was to get a broad picture of network diversity as it's clear from the length of the Whirlpool list that not every provider could be large enough to justify running their own datacentre and/or routing.

CCNetwork NameServers

By far the largest network in this context is Servers Australia. If you expand their row you'll see that GamersUN is the bulk of the servers hosted there (117 of 167). My understanding is that GamersUN uses Hypernia, and Hypernia hosts with Servers Australia.

One interesting quirk about Servers Australia is that they apparently run a gaming specific network separate to their regular network on which their mostly business customers reside. This seems to be about isolating their business customers from DDoS attacks targeted at their gaming customers.

Back in 2009 iiNet and Hypernia announced a partnership but sometime in 2012 Hypernia migrated many of their customers to Servers Australia. This Whirlpool thread suggests at least some are still at iiNet though as you see from the servers at iiNet today the vast majority are 3FL and games.on.net. However, note that Hypernia and iiNet host many game types, and we're only looking at TF2 data here.

As you go down the list there aren't too many surprises - Telstra hosts mostly GameArena servers, Internode mostly hosts the games.on.net servers, Primus hosts iPGN. These are all game networks owned by the respective parent ISP. However the handful of servers that don't belong to one of these game networks are all customer run servers - presumably either on a DSL or Cable connection.

Who's noticeably missing from this list of ISPs? Optus. They show up as Microplex being the name on the ASN. They killed their games network in 2005 so the handful of servers on their network are presumably run by customers.

Outside of Hypernia and the ISP networks DEDICATEDGAMING.com.au has a nice chunk of the independent community servers.

I couldn't find much out about the mysterious "2 Grantham St". It belongs to Yatcko Data Solutions whose customers include CLOUDCUBE.com.au.

One question that crossed my mind is how big the TF2 server rental market is in AU & NZ. If we take away the ISP run servers: 3FL (32), games.on.net (46), GameArena (31), iPGN (10), Orcon (7) you're left with about 350 servers.

Most providers charge in the realm of $35-50/month for a 24 slot server making the monthly value between $12,250 and $17,500, and the annual value between $140,000 and $210,000. Of course someone like GamersUN would not be paying such prices given that they have 117 servers so this is an overestimate.