The Relevance of Statistics in Myth
Posted: 18 Aug 2015, 03:07
This post is in response the entire Myth community's over-reliance on Myth statistics to gauge individual skill and/or contribution to team wins/losses. We have known for a long time that damage ratios are like porn for some mythers. They get off on those big numbers, but no one has ever done a detailed analysis of whether Myth statistics are really that relevant after all.
Until now.
The idea of the article was triggered by the discussion with Dantski / Switch's mention of myth statistics from the thread linked below:
http://forum.gateofstorms.net/viewtopic ... 1&start=10
The post became a lengthy article as I wanted to go through the subject once in thorough detail, so I felt it deserved its own thread here to start its own separate discussion. Enjoy the read.
---
This might be a good time to bring up a larger discussion about myth statistics. The fact that Switch posted a video about statistics being used in sports I think is a perfect demonstration of the flawed impression so many mythers have of the relevance of myth statistics. We see them used in sports and so we assume they can be just as powerful in myth. Except traditional team sports are very very different games when compared to myth.
There are at least 3 major reasons why statistics in myth are so much less meaningful than they are in sports:
1) They aren't designed to measure the mechanic that decides games won or lost
2) They don't even measure the one thing that they are designed to measure all that well
3) The levels of complexity, abstraction, and sophistication of statistics is very different
Here is the expansion on each of those reasons, one by one:
1) They aren't designed to measure the mechanic that decides games won or lost
Sports have only one "gametype", which is points scored on the scoreboard via the one or few specific methods that define the scoring system. This is the one and only mechanic that decides a game won or lost. At least some of the statistics from each sport measure the contribution to this one gametype very well, either directly as points scored or a ratio of successfully scored / overall attempts, or some other strong contributing factor. Baseball has on-base percentages or hitting percentages, and basketball has field goal percentages, etc. All of these things have direct consequence and high correlation to points scored, and thus games won/lost. No set of statistics is a perfect indicator, but in this case these statistics measure individual contribution to team score very well.
Myth instead has multiple gametypes, and none of the statistics we have are designed to measure any of them all that well. Myth statistics are body count (BC) oriented only, but never has body count been a viable game type. Yes BC is an indirect consequence to the actual gametype being played, but it is a much lower correlation which is fuzzy at best. The relevance of these BC-focused stats also vary considerably from game to game. Sometimes they mean a lot, and sometimes they mean almost nothing at all depending on the map and gametype being played. Territories and lmoth for example are gametypes that are very relevant to body count. Flag rally, and king of the map much less so. The terrain and unit sets matter a lot here too. Sometimes a great play is made where no damage is done, but a key piece of terrain is gained, or a flag is taken, etc. Or else a player made an "assist" by forcing the opponent into a bad position for their teammate to get the easy damage. Yet we have no way to calculate an "assist" in myth, and it is difficult to say how this could be done hypothetically even.
2) They don't even measure the one thing that they are designed to measure all that well
Though myth only has a few statistics designed to measure BC, they don't actually measure BC all that well in the first place. First there is the kill ratio, the most absolutely pointless statistic ever when 1 trow costs 24 points but 1 thrall is 1 point, yet both count as 1 loss or 1 kill. People who bring up kill ratios as if they matter are completely clueless.
Okay but most of us know that kill ratio is useless, so what about damage ratio? We treat damage ratio like its the definitive answer of skill.
While damage ratio is weighted according to trading points there are still some flawed premises that are accepted by this statistic, created by its reliance on trading point values for units:
-That trading point values are perfect representations of each unit's relative value
-That each unit's relative value is the same in every game (values do not change from game to game)
-That each unit's relative value remains the same throughout the game (values do not change in the middle of a game)
Yet we know each of those 3 premises are false:
-Some trading point values are skewed for some unit types making them either undervalued or overvalued. This means some unit types in every trade are almost always maxed (trow, fetch, herons, myrk), and some are almost always eliminated (bre'unor, brigands).
-The actual value of unit types can fluctuate depending on the map, gametype, or other available unit types. Yet the trading values never change. This is proven by the fact that if you took the value of one specific unit type, and isolate all other factors but just change the map, or just the gametype, or just the other available unit types in the trade, the amount you would want to trade for that unit type would likely change.
-If you are down to only 1 arty unit, and you are going against a melee rush, the value of that arty unit has increased. Units in the right place at the right time have more value than those that are in the wrong place at the wrong time during the game. Also some unit types counter others so the opponents trade, or their available units throughout the game matters. Let's say a map has souls and trow, and souls happen to work well as a counter to trow on this particular map/game, but not much else. As soon as the trow that is countered is dead, the value of the souls is diminished. Though they clearly did their job, killing them after the trow is dead matters a lot less than killing them before the trow was dead. Likewise if the opponent did not get the counter to a unit type that was counterable, then the value of those units are increased. Lastly, if you rack up a bunch of damage at the end of the game mopping up an already won battle, that matters a lot less than those who won the key fights at the start or middle of the game.
These flawed premises exist in myth and not in traditional team sports due to the different dynamics of gameplay for Myth. Myth is a complex combat system where you start at full strength and forces dwindle over time as units are lost depending on the efficiency of those battles fought. In sports, the amount of players on each side almost always stays the same. Players are not permanently removed from the field of play (i.e. "killed) giving the opposing team a player (i.e. unit) advantage (except temporarily such as for penalties in hockey). Sports also have the different dynamics of an "offense" trying to score (when a team has the ball/puck, etc.) and a "defense" guarding their goal (when they don't have the ball/puck/etc.) and those roles reversing from each team throughout the game.
Myth is instead more comparable to a tug-of-war. Except it isn't just one tug of war game. Since Myth has several dimensions to its combat system, it is more like several tug-of-war games happening at once with the ropes between each getting entangled and interwoven with one another. And there is a mechanic to make players weaker pulling on the rope (damaged units in myth), or even remove players from tugging on the rope entirely (i.e. killing units).
3) The levels of complexity, abstraction, and sophistication of statistics is very different
Myth is a far more complex and abstract game than any traditional sport. There are a variety of scoring systems (game types), fields to play on (maps), and very different players to use (units). Though our current set of myth statistics are not very relevant measures of contributions to wins, it is hard to imagine how they could be improved upon. Even just trying to measure the BC part of it well would be difficult, the values of the units change so much, even during each game. Even if you wanted to analyze the shit out of a single game and try to give the best values possible encompassing their fluctuation during the game, it still just becomes highly subjective.
Myth is also a tiny old computer game, and sports are big business. Statisticians can make great careers in sports. There has been so much more time, effort, and money thrown behind the statistical science of sports compared to myth. Sports statistics are way more detailed and in-depth, capturing a wide variety of dimensions. They have dozens, perhaps hundreds of different statistics recorded and calculated with massive databases.
We have kill ratios, and damage ratios, that's it.
In conclusion, myth stats do have some correlation but as I said it is fuzzy at best. You can do great, making game-winning plays and still have horrible stats, or have the best dmg ratio and still be the worst player on a team.
Please do not use Myth stats as primary supporting evidence for statements of who did well, or poorly, etc. Instead do more to understand the films when you watch them and use your observation as the primary supporting evidence. Yes it is more subjective, but the trained eye is just a far superior method to measure individual contribution to a team's win or loss than the 1 or 2 meager and flawed stats that we have for this highly abstract game.
Thank you.
Until now.
The idea of the article was triggered by the discussion with Dantski / Switch's mention of myth statistics from the thread linked below:
http://forum.gateofstorms.net/viewtopic ... 1&start=10
The post became a lengthy article as I wanted to go through the subject once in thorough detail, so I felt it deserved its own thread here to start its own separate discussion. Enjoy the read.
---
This might be a good time to bring up a larger discussion about myth statistics. The fact that Switch posted a video about statistics being used in sports I think is a perfect demonstration of the flawed impression so many mythers have of the relevance of myth statistics. We see them used in sports and so we assume they can be just as powerful in myth. Except traditional team sports are very very different games when compared to myth.
There are at least 3 major reasons why statistics in myth are so much less meaningful than they are in sports:
1) They aren't designed to measure the mechanic that decides games won or lost
2) They don't even measure the one thing that they are designed to measure all that well
3) The levels of complexity, abstraction, and sophistication of statistics is very different
Here is the expansion on each of those reasons, one by one:
1) They aren't designed to measure the mechanic that decides games won or lost
Sports have only one "gametype", which is points scored on the scoreboard via the one or few specific methods that define the scoring system. This is the one and only mechanic that decides a game won or lost. At least some of the statistics from each sport measure the contribution to this one gametype very well, either directly as points scored or a ratio of successfully scored / overall attempts, or some other strong contributing factor. Baseball has on-base percentages or hitting percentages, and basketball has field goal percentages, etc. All of these things have direct consequence and high correlation to points scored, and thus games won/lost. No set of statistics is a perfect indicator, but in this case these statistics measure individual contribution to team score very well.
Myth instead has multiple gametypes, and none of the statistics we have are designed to measure any of them all that well. Myth statistics are body count (BC) oriented only, but never has body count been a viable game type. Yes BC is an indirect consequence to the actual gametype being played, but it is a much lower correlation which is fuzzy at best. The relevance of these BC-focused stats also vary considerably from game to game. Sometimes they mean a lot, and sometimes they mean almost nothing at all depending on the map and gametype being played. Territories and lmoth for example are gametypes that are very relevant to body count. Flag rally, and king of the map much less so. The terrain and unit sets matter a lot here too. Sometimes a great play is made where no damage is done, but a key piece of terrain is gained, or a flag is taken, etc. Or else a player made an "assist" by forcing the opponent into a bad position for their teammate to get the easy damage. Yet we have no way to calculate an "assist" in myth, and it is difficult to say how this could be done hypothetically even.
2) They don't even measure the one thing that they are designed to measure all that well
Though myth only has a few statistics designed to measure BC, they don't actually measure BC all that well in the first place. First there is the kill ratio, the most absolutely pointless statistic ever when 1 trow costs 24 points but 1 thrall is 1 point, yet both count as 1 loss or 1 kill. People who bring up kill ratios as if they matter are completely clueless.
Okay but most of us know that kill ratio is useless, so what about damage ratio? We treat damage ratio like its the definitive answer of skill.
While damage ratio is weighted according to trading points there are still some flawed premises that are accepted by this statistic, created by its reliance on trading point values for units:
-That trading point values are perfect representations of each unit's relative value
-That each unit's relative value is the same in every game (values do not change from game to game)
-That each unit's relative value remains the same throughout the game (values do not change in the middle of a game)
Yet we know each of those 3 premises are false:
-Some trading point values are skewed for some unit types making them either undervalued or overvalued. This means some unit types in every trade are almost always maxed (trow, fetch, herons, myrk), and some are almost always eliminated (bre'unor, brigands).
-The actual value of unit types can fluctuate depending on the map, gametype, or other available unit types. Yet the trading values never change. This is proven by the fact that if you took the value of one specific unit type, and isolate all other factors but just change the map, or just the gametype, or just the other available unit types in the trade, the amount you would want to trade for that unit type would likely change.
-If you are down to only 1 arty unit, and you are going against a melee rush, the value of that arty unit has increased. Units in the right place at the right time have more value than those that are in the wrong place at the wrong time during the game. Also some unit types counter others so the opponents trade, or their available units throughout the game matters. Let's say a map has souls and trow, and souls happen to work well as a counter to trow on this particular map/game, but not much else. As soon as the trow that is countered is dead, the value of the souls is diminished. Though they clearly did their job, killing them after the trow is dead matters a lot less than killing them before the trow was dead. Likewise if the opponent did not get the counter to a unit type that was counterable, then the value of those units are increased. Lastly, if you rack up a bunch of damage at the end of the game mopping up an already won battle, that matters a lot less than those who won the key fights at the start or middle of the game.
These flawed premises exist in myth and not in traditional team sports due to the different dynamics of gameplay for Myth. Myth is a complex combat system where you start at full strength and forces dwindle over time as units are lost depending on the efficiency of those battles fought. In sports, the amount of players on each side almost always stays the same. Players are not permanently removed from the field of play (i.e. "killed) giving the opposing team a player (i.e. unit) advantage (except temporarily such as for penalties in hockey). Sports also have the different dynamics of an "offense" trying to score (when a team has the ball/puck, etc.) and a "defense" guarding their goal (when they don't have the ball/puck/etc.) and those roles reversing from each team throughout the game.
Myth is instead more comparable to a tug-of-war. Except it isn't just one tug of war game. Since Myth has several dimensions to its combat system, it is more like several tug-of-war games happening at once with the ropes between each getting entangled and interwoven with one another. And there is a mechanic to make players weaker pulling on the rope (damaged units in myth), or even remove players from tugging on the rope entirely (i.e. killing units).
3) The levels of complexity, abstraction, and sophistication of statistics is very different
Myth is a far more complex and abstract game than any traditional sport. There are a variety of scoring systems (game types), fields to play on (maps), and very different players to use (units). Though our current set of myth statistics are not very relevant measures of contributions to wins, it is hard to imagine how they could be improved upon. Even just trying to measure the BC part of it well would be difficult, the values of the units change so much, even during each game. Even if you wanted to analyze the shit out of a single game and try to give the best values possible encompassing their fluctuation during the game, it still just becomes highly subjective.
Myth is also a tiny old computer game, and sports are big business. Statisticians can make great careers in sports. There has been so much more time, effort, and money thrown behind the statistical science of sports compared to myth. Sports statistics are way more detailed and in-depth, capturing a wide variety of dimensions. They have dozens, perhaps hundreds of different statistics recorded and calculated with massive databases.
We have kill ratios, and damage ratios, that's it.
In conclusion, myth stats do have some correlation but as I said it is fuzzy at best. You can do great, making game-winning plays and still have horrible stats, or have the best dmg ratio and still be the worst player on a team.
Please do not use Myth stats as primary supporting evidence for statements of who did well, or poorly, etc. Instead do more to understand the films when you watch them and use your observation as the primary supporting evidence. Yes it is more subjective, but the trained eye is just a far superior method to measure individual contribution to a team's win or loss than the 1 or 2 meager and flawed stats that we have for this highly abstract game.
Thank you.