jzahir

In economics there’s something called the Edgeworth box which attempts to illustrate how two people with different production possibilities might trade with each other.

I’ve always maintained that a similar construct could be used to illustrate how two people with slightly different idea/word-meaning values might communicate (i.e. trade and barter meanings).

This theory neatly explains how spectacularly I fail to communicate with programmers.  It’s because I completely lack any meanings or ideas that they want to trade in.

In the game industry of Seattle, everyone integrates easily with the same geek-chic humor we’ve had since the early 1990’s but programmers quickly separate themselves from the non-linear thinking of non-programmers.  When they saw us data wonks floundering because our managers keep asking for different slices of the data, 5 programmers slammed the same sticky note to the Scrum board: Automate finding patterns in the data.  At that point, I had nothing of value to add to the conversation.

My abstractions about player values and the equilibrium of the game’s economy solicited the same sympathetic disdain they show artists debating the color schemes of a game.  They quickly disappear into the safety of their code.  (Meanwhile the concept artists are drawing ridiculously chibi-headed caricatures of the programmers as zombie fodder for their next project.)

The product they came up with is what is hailed as the newly minted discipline of Data Science.  The application of apps to data from apps to build better apps from.

But instead of actually improving the outcomes it simply puts the mantra of “fail cheaply and fail often” on steroids.  Although you can’t derive any moments from data that comes from a Cauchy-generating process, you can subject it to a power series that will quickly reveal…you guessed it, something to find patterns in.

One of the places I saw this clearly was in the balance of soft and hard currencies in a builder game we were designing.

In the early 2000’s everybody was enamored with agent model simulations that could simulate the data you’d get with agents that a finite set of rule-based behaviors.  I don’t know why we thought this was so special.  Game designers have been using simulations on board games forever.

For games that monetized based on currency shortages, however, you had to build separate simulators for currencies so that you wouldn’t accidentally nerf or buff a virtual good that you just spent the last 5 levels increasing the value of.  In mid-2012 Joris Dormans created an online simulator called Machinations to do just that.  You constructed sources and sinks of each currency and a few other logical conditions to build an animation of the flows along with a graphic that simulated what you would see from an instrumented game in Beta.  Now with a model of the mechanics, a par sheet of probabilities and well constructed on-boarding I should be able to balance the economy right?  Well…except for those pesky players who refused to play as scripted.

No matter how well we balanced the economy in the builder game, results came back from tests in Canada that players were not making choices that could be classed as strategies or player-types.

When our programmer got hold of Joris’ open source program, he immediately bolted it to a genetic algorithm to maximize the flow of resources (including In App Purchases). This would seem the right thing to do as a programmer but it ignores the fact that entrepreneurs and gamers live by:  One plays to beat expected values, not to meet them.

Once again, the charts produced by Machinations that most closely matched actual data were those with the fattest tails of a distribution (Cauchy).

No matter how many ways the problem was ‘diagnosed’ player retention dropped (with a freemium ‘Thud!’) right at the point the player met a paywall.

Now it was time to bring out the big guns of behavior analysis…psychology.

The absolute genius of game designers is that they understand economics at a very sophisticated and subtle level.  What they do to manipulate people’s learned value of engagement is only now being appreciated by educators, social scientists and economists as a real tool.

This is what truly excited me about the possibilities of this industry.  How did these geniuses build up a sense of value in a virtual good such that people would willingly spend millions of dollars to possess?  and then brag about it?  The artificial creation of value is what lies at the heart of economic growth and collapse and these guys had mastered it into an art.  Literally, an art.

That’s when I met the marketing department.

For us managers, marketers and feather merchants, artistry is just another asset to be exploited.  Our main ‘sploit’ seems to be to convince the game designers that we have hidden knowledge of the players that they do not.  We do this by leveraging apophenia.

Apophenia is the tendency of humans to see patterns in random.  It is not just a comfortable delusion…It’s a savanna-bred imperative for survival and tops the agenda of most business meetings.

In the case of the game industry it works like this: Take any given population of players and say they’re all motivated to act differently.  Let’s say we actually graph their likelihood to spend money on something over time and find that the curves those ‘likelihoods’ trace includes every shape from a ski slope to a bell curve to a wall.  The deeper we go, the more random we find people’s preferences, motivations and willingness to spend money are any given point in time.

Now take a game in the Apple or Google store and solicit random downloads from among this population and graph the likelihood they’ll spend on level 7 of your game.  You’ll get a bell curve…but a bell curve with really fat tails.

You can convolve a random selection from a set of random distributions (with varying σ), but you’re no longer able to derive their moments (mean, standard deviation etc.).  Instead the best description of their central tendency is by what astronomer’s use to describe the brightness of stars that appear fuzzy around the edges: Full width, half maximum (FWHM).

The next day you’ll get a different random selection of distributions (even from the same players) but everyone staring at the chart will clearly see a middle and call it average.  Stats classes do us a severe dis-service by stating everything in terms of normal distributions because forevermore, we’ll use the mean of a normal distribution (think “bell curve”)  for the expected value and central tendency of any set of measurements.

So what happens when we design changes in the economy of a games points/currencies and power based on a mistaken distribution of player behavior?  BAM!  SPLAT! Game over!

When the product owners completely change of direction in the game at every scrum retrospective (with the predictability of a Roomba) you can be pretty sure you’re not measuring expected values. But the apophenia persists and we’re sure we’d have a hit, if we could only expose it!

When the first dozen diagnostics don’t change outcomes, managers double down on their ability to see patterns in the data and recursively make the situation worse.  They’ll ask their data analysts for increasingly obscure cross tabulations to feed an endless cycle of “AHA!  I knew it!…wait, that doesn’t work?!  Gimme….” moments.

So what happens to insight analysts in this environment?  How do they maintain their worth in the face of a habit of failed recommendations?

Here is where I met the fourth leg of the video game industry…the programmers.

A metaphor I think is useful here is the story Sylvie and Bruno by Lewis Carroll.  In it a king orders the court cartographers to make maps of increasing accuracy until they finally have a 1-1 scale map of everything in the kingdom.  The hero’s quest is to convince the king he’ll win a Darwin Award if he unrolls the map.

There is a constant debate among those of us who assay life as to just how closely our models of reality have to agree with empirical data in order to be useful.  Unfortunately, despite George Box’s hopeful assertion, models are only ever useful for deceiving oneself into believing that natural law has anything to do with human behavior.

The first wrong model I used was a taxonomy  for classifying player types developed by Richard Bartle in 1990.  In Bartle’s taxonomy there are four basic types of players in games.  Achievers, Explorers, Socializers and Killers.  I was definitely the explorer he describes as players for whom “The real fun comes only from discovery and making the most complete set of maps in existence.”  I should’ve known better than trust the ramblings of the self-proclaimed “Wizards of MUDS” but it turns out Bartles accurately predicted my final score as an “Explorer” in the game of Freemium game design.

You can imagine what happens to characters like me when the game industry is dominated by people whose ludo type is Killer.  According to Bartle, “Killers use words like: ‘Ha!’, ‘Coward!’, ‘Die!’ and ‘Die, Die! Die!’ (Killers are people of few words).  “

It’s amazing how fast you find out that being an explorer is dominated by any other strategy.  The trick is in how fast you learn to change your game.

The learning process in game play is nicely described by theories of Q-learning or re-enforcement learning models.  The problem is that those models assume a well defined Markov decision chain that every rational player would follow…and of course All players are rational.

Right.

At about the same time Bartle was deciding how people played Dungeon’s and Dragons, Nigel Howard was doing the same thing for international arms talks.  Only Howard didn’t assume rationality.

Howard’s Drama theory holds that as one learns the game they are only holding their definition of the game as a provisional assumption.  If they get enough evidence that the game isn’t what they thought it was, they’ll re-define the game to better suit the data.  Very Bayes.  The result of this was that players would fall into predictable Markov decision processes only so long as there was no divergence from expected outcomes.  As soon as something unexpected happened, the game would change.  And nothing’s more predictable than the unexpected in the game industry.

In 2011 Seattle hosted Casual Connect, a conference of casual game developers, at which everyone was told that any company without several statisticians on staff would be at a competitive disadvantage against teams that could see into the hearts of its players.

Everyone got the message and I truly believe that, but for that message, I would not have been hired by a game company.

The problem was that they were expecting this guy, while I thought I was there to make the most complete maps of player behavior in existence.

What happens in a 2-player game when each player is given the rules to completely different games?

If they have the expectation that everyone is playing the same game, the person with the fewest words moans “Lame” and struts to the next machine in the arcade.

The thing is, that because they are learning and adapting, the way they play the next game is different than how they would have played in the absence of analytics.

They’re haunted by the possibility that within the crystal ball of analytics there lies the super-power to make people like you…even if only for cosplay at a conference.

Most students of economics mistake concepts like “maximizing utility” and “marginal cost” as clinical, dose-response calculations.  They are not.  They are examples of the absurd heuristics we use to hide how completely pointless and self deceptive our ability to value something is.  We can create value out of nothing and invest it with nothing less than our identity or the worth of a nation.

At no point in my career has this been clearer than when I helped design game economies for video games.

My brief (and completely lackluster) encounter with this industry started in the summer of 2011 when a High School friend of mine who taught economics at Stanford and I were B.S.ing about an emerging business model called “freemium” games and concluded that this would be the perfect petri dish for experimenting with consumer behavior in controlled economies.  In freemium games a person could download a casual game onto a mobile device and play for free but soon after starting, players were offered the “opportunity” to make “In App” purchases to enhance their game play.

I made a few inquiries and later that year I was approached by a company in Redmond Washington to help design economies in their portfolio of freemium games.

To get me oriented in the world of game design they sent me Game Design Documents from some of their most successful titles and deconstructions of their competitor’s games.  These read like Cliff Notes of a Cirque du Soleil performance.  It described with great precision the colors, fonts, layout of the screen, action and concept but there was very little describing the intended consequences of engagement.  I was left very impressed but not at all sure about what.

I finally got a clue about a month later when a game producer gave me a copy of Mihaly Csikszentmihalyi’s book “Flow”, with a quickly scribbled sticky note “The Bible of Game design” stuck to it.  In it, the author describes the psychology of flow and how the value of play becomes “autotelic” or and end-in-itself when a person is in the zone, so to speak.  This was not the source of value I was expecting and certainly not the source of value taught in economics courses.

It was while reading this explication of play that I began my first deconstruction of game economies with a newly released title called Clash of Clans.  I created 14 different players (using several thousands of the company’s dollars) and tried several different game-play techniques in order to map:

1. The pricing and points structure
2. The rate of flow of both the hard and soft currencies in the game
3. Networks and interactions of clans.
4. Subgame deconstructions
5. Subgame portfolio analysis

In 2012 I followed this up with deconstructions of two other new releases: Rage of Bahamut (A card collecting game) and CSR Racer (a Drag Racing game).

At the same time I was doing this, I was also building my own ludography of games that were being released into the excessively hot and competitive market of iPhone freemium games in 2012 and 2013.

The sequence of events that were my introduction to the gaming world is important.  In games this is called on-boarding and it turns out to be one of the most important phases for determining a game’s success or failure.

So many of us kids that went to the Freemium “Rave” came away with scars and embarrassments that never fully healed.  Some of them were funny.  There were the producers with what I called the GDC swagger.  GDC is an annual cabal of conspirators in the game industry where everyone went to show off.  There were two indicators that you were looking at a producer.  The first was that a number of groupies were following them like a plague of gnats.  They were almost theatrical caricatures of 21st century geek-chic as they managed to Volks-strut down an Isle while pontificating to no one in particular.  The second was that they would, at arbitrary moments, make an exaggerated point and wink of recognition at random people along the way who would just look confused and mouth “Who the hell was that?”

Some of the scars were more serious.  Using the as-yet-untapped resources of following a person’s every move and gesture on an iPhone and then asking us Analysts “What information can we take from this to create an addiction…Without the addict realizing it?”.

Suddenly Mihaly Csikszentmihalyi was Walter White from Breaking Bad and I was one of his doomed business partners.

This get rich quick scheme has pretty much run its course and is ripe for some deconstruction.  While many of those who know how to Volks-strut will undoubtedly spin this much better than I can, mine will demonstrate just how twisted economic thinking can be.  I am going to attempt to model my deconstruction in the same way I would any game.  It begins with the players…not the game.